AVS3-based pixel-by-pixel motion estimation method, device and electronic device

Abstract

The present application discloses a pixel-by-pixel motion estimation method, device and electronic device based on AVS3. The method includes: performing pixel-by-pixel motion estimation on the first coding unit CU set in the current video frame to obtain the pixel-by-pixel motion vector of each first CU in the first CU set; dividing each first CU in the first CU set according to a binary tree partitioning method or an enhanced quadtree partitioning method to obtain a sub-CU corresponding to each first CU; when it is determined that the discreteness of the integer pixel motion vector of the first CU is less than a preset threshold, the pixel-by-pixel motion vector of the first CU is used as the pixel-by-pixel motion vector of each sub-CU in the first CU; when it is determined that the discreteness of the IMV of the first CU is greater than a preset threshold, performing pixel-by-pixel motion estimation on each sub-CU in the first CU to obtain the pixel-by-pixel motion vector of each sub-CU. The present application solves the technical problem of low efficiency of pixel-by-pixel motion estimation in the related art.

Description

Technical Field

[0001] The present application relates to the field of video coding and decoding, and in particular to a pixel-by-pixel motion estimation method, device and electronic device based on AVS3. Background Art

[0002] AVS3 is the third generation standard developed by the Audio Video Coding Standard (AVS) working group in China. It is designed to adapt to a variety of application scenarios, such as Ultra High Definition (UHD) TV broadcasting, Virtual Reality (VR), and video surveillance. Based on AVS2, AVS3 adopts a more flexible block partitioning structure to support videos of various resolutions. In addition to supporting Quad-Tree (QT) partitioning, AVS3 also supports Binary Tree (BT) and Extended Quad-Tree (EQT) partitioning, and AVS3 expands the size of the Largest Coding Unit (LCU) from 64x64 to 128x128, which will lead to a sharp increase in the complexity of coding unit block partitioning. For sub-pixel motion estimation of video frames, due to the increased complexity of coding unit partitioning in AVS3, sub-pixel motion estimation is less efficient and takes more time. Summary of the invention

[0003] The embodiments of the present application provide a sub-pixel motion estimation method, device and electronic device based on AVS3 to at least solve the technical problem of low sub-pixel motion estimation efficiency in the related art.

[0004] According to one aspect of an embodiment of the present application, a sub-pixel motion estimation method based on AVS3 is provided, including: performing sub-pixel motion estimation on a first coding unit CU set in a current video frame to obtain a sub-pixel motion vector of each first CU in the first CU set; wherein the shape of each first CU in the first CU set is a square; dividing each first CU of the first CU set according to a binary tree partitioning method or an enhanced quadtree partitioning method to obtain a sub-CU corresponding to each first CU; when it is determined that the discreteness of the integer pixel motion vector of the first CU is less than a preset threshold, using the sub-pixel motion vector of the first CU as the sub-pixel motion vector of each sub-CU in the first CU; the discreteness is used to indicate the degree of dispersion of the integer pixel motion vectors of multiple sub-CUs in the first CU; when it is determined that the discreteness of the integer pixel motion vector IMV of the first CU is greater than the preset threshold, performing sub-pixel motion estimation on each sub-CU in the first CU to obtain the sub-pixel motion vector of each sub-CU.

[0005] According to another aspect of an embodiment of the present application, a sub-pixel motion estimation device based on AVS3 is also provided, including: an estimation unit, which performs sub-pixel motion estimation on a first coding unit CU set in a current video frame to obtain a sub-pixel motion vector of each first CU in the first CU set; wherein the shape of each first CU in the first CU set is a square; a division unit, which is used to divide each first CU of the first CU set according to a binary tree division method or an enhanced quadtree division method to obtain a sub-CU corresponding to each first CU; a first replacement unit, which is used to use the sub-pixel motion vector of the first CU as the sub-pixel motion vector of each sub-CU in the first CU when it is determined that the discreteness of the integer pixel motion vector of the first CU is less than a preset threshold; the discreteness is used to indicate the degree of dispersion of the integer pixel motion vectors of multiple sub-CUs in the first CU; a first acquisition unit, which is used to perform sub-pixel motion estimation on each sub-CU in the first CU to obtain the sub-pixel motion vector of each sub-CU when it is determined that the discreteness of the IMV of the first CU is greater than a preset threshold.

[0006] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned AVS3-based pixel-by-pixel motion estimation method when running.

[0007] According to another aspect of an embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to execute the AVS3-based pixel-by-pixel motion estimation method through the computer program.

[0008] In an embodiment of the present application, pixel-by-pixel motion estimation is performed on a first coding unit CU set in a current video frame to obtain a pixel-by-pixel motion vector of each first CU in the first CU set; wherein the shape of each first CU in the first CU set is a square; each first CU in the first CU set is divided according to a binary tree partitioning method or an enhanced quadtree partitioning method to obtain a sub-CU corresponding to each first CU; when it is determined that the discreteness of the integer pixel motion vector of the first CU is less than a preset threshold, the pixel-by-pixel motion vector of the first CU is used as the pixel-by-pixel motion vector of each sub-CU in the first CU; the discreteness is used to indicate the degree of dispersion of the integer pixel motion vectors of multiple sub-CUs in the first CU; when it is determined that the discreteness of the IMV of the first CU is greater than the preset threshold, pixel-by-pixel motion estimation is performed on each sub-CU in the first CU to obtain the pixel-by-pixel motion vector of each sub-CU. Since whether to perform pixel-by-pixel motion estimation on the sub-unit of the coding unit is determined according to the discreteness of the integer-pixel motion vector of the coding unit, and there is no need to perform pixel-by-pixel estimation on each coding unit and sub-unit, not only the efficiency of pixel-by-pixel motion estimation can be improved, but also the pixel-by-pixel motion estimation time can be reduced, saving system computing resources, thereby solving the technical problem of low efficiency in determining the prediction mode of the CU in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0010] Figure 1 is a schematic diagram of an application environment of an optional AVS3-based pixel-by-pixel motion estimation method according to an embodiment of the present invention;

[0011] Figure 2 is a schematic diagram of an application environment of another optional AVS3-based pixel-by-pixel motion estimation method according to an embodiment of the present invention;

[0012] Figure 3 is a flowchart of an optional AVS3-based pixel-by-pixel motion estimation method according to an embodiment of the present invention;

[0013] Figure 4 is a schematic diagram of coding unit division according to an embodiment of the present invention;

[0014] Figure 5 is a schematic diagram of an integer pixel motion vector of a coding unit according to an embodiment of the present invention;

[0015] Figure 6 is a schematic diagram of integer pixel motion vectors of another coding unit according to an embodiment of the present invention;

[0016] Figure 7 is a flowchart of another optional sub-pixel motion estimation method based on AVS3 according to an embodiment of the present invention;

[0017] Figure 8 is a schematic structural diagram of an optional AVS3-based pixel-by-pixel motion estimation device according to an embodiment of the present invention;

[0018] Fig. 9 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0020] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0021] According to one aspect of an embodiment of the present invention, a sub-pixel motion estimation method based on AVS3 is provided. Optionally, as an optional implementation, the above sub-pixel motion estimation method based on AVS3 can be but is not limited to being applied to the following: Figure 1In the hardware environment shown. The hardware environment includes: a terminal device 102 for human-computer interaction with a user, a network 104, and a server 106. A user 108 can perform human-computer interaction with the terminal device 102, and a pixel-by-pixel motion estimation application client based on AVS3 is running in the terminal device 102. The above-mentioned terminal device 102 includes a human-computer interaction screen 1022, a processor 1024 and a memory 1026. The human-computer interaction screen 1022 is used to present an interface for video frame processing; the processor 1024 is used to obtain the first coding unit CU set in the current video frame. The memory 1026 is used to store the first coding unit CU set and the pixel-by-pixel motion vector of each first CU in the first CU set.

[0022] In addition, the server 106 includes a database 1062 and a processing engine 1064, wherein the database 1062 is used to store the first coding unit CU set and the sub-pixel motion vector of each first CU in the first CU set. The processing engine 1064 is used to perform sub-pixel motion estimation on the first coding unit CU set in the current video frame to obtain the sub-pixel motion vector of each first CU in the first CU set; wherein each first CU in the first CU set is in a square shape; each first CU in the first CU set is divided according to a binary tree partitioning method or an enhanced quadtree partitioning method to obtain a sub-CU corresponding to each first CU; when it is determined that the discreteness of the integer pixel motion vector of the first CU is less than a preset threshold, the sub-pixel motion vector of the first CU is used as the sub-pixel motion vector of each sub-CU in the first CU; the discreteness is used to indicate the dispersion degree of the integer pixel motion vectors of multiple sub-CUs in the first CU; when it is determined that the discreteness of the IMV of the first CU is greater than the preset threshold, the sub-pixel motion estimation is performed on each sub-CU in the first CU to obtain the sub-pixel motion vector of each sub-CU.

[0023] As another optional implementation, the above-mentioned pixel-by-pixel motion estimation method based on AVS3 in this application can be applied to Figure 2 In. Figure 2 As shown, human-computer interaction can be performed between the user 202 and the user equipment 204. The user equipment 204 includes a memory 206 and a processor 208. In this embodiment, the user equipment 204 can refer to but is not limited to performing the operations performed by the above-mentioned terminal device 102 to obtain the sub-pixel motion vector of each sub-CU.

[0024] Optionally, the terminal device 102 and the user device 204 may be, but are not limited to, terminals such as mobile phones, tablet computers, laptop computers, and PCs, and the network 104 may include, but is not limited to, a wireless network or a wired network. The wireless network includes: WIFI and other networks that implement wireless communication. The wired network may include, but is not limited to: a wide area network, a metropolitan area network, and a local area network. The server 106 may include, but is not limited to, any hardware device that can perform computing. The server may be a single server, or a server cluster consisting of multiple servers, or a cloud server. The above is only an example, and no limitation is made to this in this embodiment.

[0025] In the related art, AVS3 performs video encoding at the LCU level, and Fractional Motion Estimation (FME) is a separate pipeline structure in the pipeline architecture of the AVS3 encoder. Before the LCU-level fractional motion estimation structure, integer-pixel motion estimation (IME) is also a separate pipeline structure in the pipeline architecture of the AVS3 encoder.

[0026] Optionally, in one or more embodiments, Figure 3 As shown, the above-mentioned pixel-by-pixel motion estimation method based on AVS3 includes:

[0027] S302, performing pixel-by-pixel motion estimation on a first coding unit CU set in a current video frame to obtain a pixel-by-pixel motion vector of each first CU in the first CU set; wherein each first CU in the first CU set is in a square shape.

[0028] In the embodiment of the present invention, the first CU in the first coding unit set may be a coding unit at the LCU level, or one or more CU blocks obtained by dividing the LCU in a quadtree partitioning manner. Figure 4 As shown, the size of the LCU is 64*64, and the size of the first coding unit in the first coding unit set can be 64*64, 32*32, 16*16, 8*8 and 4*4. The above is only an example, and the embodiment of the present invention does not make any limitation to this.

[0029] S304: Divide each first CU of the first CU set according to a binary tree partitioning method or an enhanced quadtree partitioning method to obtain a sub-CU corresponding to each first CU.

[0030] In the embodiment of the present invention, Figure 5As shown in , the first CU of the first CU set is divided according to the binary tree division method to obtain Figure 5 The two sub-CUs corresponding to the first CU shown in (a); Figure 5 As shown, the first CU of the first CU set is divided according to the enhanced quadtree division method to obtain Figure 5 The four sub-CUs corresponding to each first CU shown in (b).

[0031] S306: When it is determined that the discreteness of the integer pixel motion vector of the first CU is less than a preset threshold, the sub-pixel motion vector of the first CU is used as the sub-pixel motion vector of each sub-CU in the first CU; the discreteness is used to indicate the degree of dispersion of the integer pixel motion vectors of multiple sub-CUs in the first CU.

[0032] In the embodiment of the present invention, Figure 5 As shown, when it is determined that the discreteness of the integer pixel motion vector IMV of the first CU is less than the preset threshold, that is, the integer pixel motion vectors of the multiple sub-CUs after the first CU is divided are relatively concentrated, and the integer pixel motion vectors of the sub-CUs are not much different. At this time, the integer pixel motion vector IMV of the first CU can be used as the IMV of each sub-CU in the first CU.

[0033] Preferably, the sub-pixel motion vector of the replaced sub-CU is replaced by the sub-pixel motion vector of the replacement CU block that just wraps the replaced sub-CU block. Figure 6 As shown in the figure, the sub-pixel motion vector of the 8x4 CU block is replaced by the sub-pixel motion vector of the 8x8 CU block that just wraps it, rather than being replaced by the sub-pixel motion vector of the 16x16 CU block.

[0034] S308: When it is determined that the discreteness of the IMV of the first CU is greater than a preset threshold, perform pixel-by-pixel motion estimation on each sub-CU in the first CU to obtain a pixel-by-pixel motion vector of each sub-CU.

[0035] In an embodiment of the present invention, when the discreteness of the integer-pixel motion vector (IMV) of the first CU is greater than a preset threshold, it means that the integer-pixel motion vectors of the multiple sub-CUs after the first CU is divided are relatively dispersed, and the integer-pixel motion vectors of the sub-CUs are quite different. At this time, the integer-pixel motion vector IMV of the first CU cannot be used as the IMV of each sub-CU in the first CU, but each sub-CU needs to be further subjected to pixel-by-pixel motion estimation.

[0036] In an embodiment of the present application, pixel-by-pixel motion estimation is performed on the first coding unit CU set in the current video frame to obtain the pixel-by-pixel motion vector of each first CU in the first CU set; wherein, the shape of each first CU in the first CU set is a square; each first CU in the first CU set is divided according to a binary tree partitioning method or an enhanced quadtree partitioning method to obtain a sub-CU corresponding to each first CU; when it is determined that the discreteness of the integer pixel motion vector IMV of the first CU is less than a preset threshold, the IMV of the first CU is used as the IMV of each sub-CU in the first CU; the discreteness is used to indicate the degree of dispersion of the integer pixel motion vectors of multiple sub-CUs in the first CU; when it is determined that the discreteness of the IMV of the first CU is greater than the preset threshold, pixel-by-pixel motion estimation is performed on each sub-CU in the first CU to obtain the pixel-by-pixel motion vector of each sub-CU. Since whether to perform pixel-by-pixel motion estimation on the sub-unit of the coding unit is determined according to the discreteness of the integer-pixel motion vector of the coding unit, and there is no need to perform pixel-by-pixel estimation on each coding unit and sub-unit, it can not only improve the efficiency of pixel-by-pixel motion estimation, reduce the time of pixel-by-pixel motion estimation, and save system computing resources, but also solve the technical problem of low efficiency in determining the prediction mode of CU in the related technology.

[0037] In one or more embodiments, the above-mentioned pixel-by-pixel motion estimation method based on AVS3 further includes:

[0038] Partition each of the sub-coding units by a quadtree, a binary tree, or an enhanced quadtree to obtain a second coding unit;

[0039] The sub-pixel motion vector of the sub-coding unit is used as the sub-pixel motion vector of the second coding unit.

[0040] Specifically, for example, after the square 16*16 coding unit is divided into enhanced quadtree EQT, the obtained 8*8 sub-coding unit is divided into BT, and two 8*4 coding units are obtained. The sub-pixel motion vectors of these two 8*4 coding units are replaced by the sub-pixel motion vectors of the 8*8 sub-coding unit.

[0041] In one or more embodiments, the first coding unit includes one or more maximum coding units LCU and / or coding units obtained by dividing the LCU in a quadtree partitioning manner, and the step S302 of performing pixel-by-pixel motion estimation on the first coding unit CU set in the current video frame to obtain the pixel-by-pixel motion vector of each first CU in the first CU set includes at least one of the following:

[0042] Perform pixel-by-pixel motion estimation on the one or more maximum coding units LCU to obtain a pixel-by-pixel motion vector of each maximum coding unit LCU.

[0043] In the embodiment of the present invention, Figure 4 As shown, taking the size of the maximum coding unit LCU as 64*64 as an example, here, the LCU with a size of 64*64 is subjected to pixel-by-pixel motion estimation to obtain the pixel-by-pixel motion vector of the current LCU.

[0044] The coding units obtained by dividing the LCU in a quadtree division manner are subjected to pixel-by-pixel motion estimation to obtain pixel-by-pixel motion vectors of the CU obtained after the division.

[0045] In the embodiment of the present invention, Figure 4 As shown, taking the size of the maximum coding unit LCU as 64*64 as an example, here, the size of 64*64 is divided by quadtree partitioning to obtain 16 coding units CU of 16*16 size, and then the obtained coding units are subjected to pixel-by-pixel motion estimation to obtain the pixel-by-pixel motion vector of the 16*16 size CU divided by quadtree partitioning.

[0046] In one or more embodiments, before using the fractional pixel motion vector of the first CU as the fractional pixel motion vector of each sub-CU in the first CU when determining that the discreteness of the integer pixel motion vector of the first CU is less than a preset threshold, the method further includes:

[0047] When the first CU is divided into a plurality of sub-CUs in a binary tree manner, the discreteness of the integer pixel motion vector IMV of the first CU is determined by the following formula:

[0048] ;

[0049] in, Indicates the discreteness of the integer pixel motion vector IMV of the first CU; Represents the displacement value of the integer pixel motion vector of the i-th sub-CU on the x-axis, Represents the displacement value of the ith sub-CU integer pixel motion vector on the y-axis, Represents the average displacement value of the sub-CU of the first CU on the x-axis, represents the average displacement value of the sub-CU of the first CU on the y-axis, and n represents the number of sub-coding units of the first coding unit.

[0050] In the embodiment of the present invention, Figure 5 As shown in , the first CU of the first CU set is divided according to the binary tree division method to obtain Figure 5 The two sub-CUs corresponding to the first CU shown in (a); the discreteness of the integer pixel motion vector IMV of the first CU is determined When it is less than the preset threshold, the integer pixel motion vector is IMV ( , ) sub-CU, and another integer pixel motion vector IMV( , )The pixel-by-pixel motion vectors of the sub-CU are replaced by the pixel-by-pixel motion vectors of the first CU.

[0051] In one or more embodiments, before using the fractional pixel motion vector of the first CU as the fractional pixel motion vector of each sub-CU in the first CU when determining that the discreteness of the integer pixel motion vector of the first CU is less than a preset threshold, the method further includes:

[0052] When the first CU is divided into a plurality of sub-CUs in an enhanced quadtree manner, the discreteness of the integer pixel motion vector IMV of the first CU is determined by the following formula:

[0053] ;

[0054] in, Indicates the discreteness of the integer pixel motion vector IMV of the first CU; Represents the displacement value of the integer pixel motion vector of the i-th sub-CU on the x-axis, Represents the displacement value of the ith sub-CU integer pixel motion vector on the y-axis, Represents the average displacement value of the sub-CU of the first CU on the x-axis, represents the average displacement value of the sub-CU of the first CU on the y-axis, and n represents the number of sub-coding units of the first coding unit.

[0055] Specifically, Figure 5 As shown, the first CU of the first CU set is divided according to the enhanced quadtree division method to obtain Figure 5 In (b), each first CU corresponds to four sub-CUs. The discreteness of the integer pixel motion vector IMV of the first CU is determined. When it is less than the preset threshold, the integer pixel motion vector IMV ( , ), IMV( , ), IMV( , ), IMV( , )The sub-pixel motion vectors of the four sub-CUs are all replaced by the integer-pixel motion vector IMV of the first CU.

[0056] In one or more embodiments, before performing pixel-wise motion estimation on the first coding unit CU set in the current video frame, it includes: performing integer-pixel motion estimation on the current video frame to obtain integer-pixel motion vector information of the first coding unit CU set.

[0057] In the embodiment of the present invention, AVS3 performs video encoding at the maximum coding unit LCU level pipeline, and sub-pixel motion estimation is a separate first-level pipeline structure in the AVS3 encoder pipeline architecture. Before the sub-pixel motion estimation structure at the LCU level, the whole pixel motion estimation is also a separate first-level pipeline structure in the AVS3 encoder pipeline architecture.

[0058] In one or more embodiments, the size of the LCU is 64x64.

[0059] Based on the above embodiments, in one or more embodiments, the above pixel-by-pixel motion estimation method based on AVS3 includes the following steps:

[0060] First, select the CU block method for sub-pixel motion estimation

[0061] In the process of presetting within the video frame, the integer pixel motion estimation IME obtains the integer pixel motion vector information of all possible partitioned CUs, including the integer pixel motion vector information of the CU blocks obtained by the binary tree (BT) partitioning method, the quad tree (QT) and the enhanced quad tree (EQT) partitioning method.

[0062] (1) First, perform pixel-by-pixel motion estimation on the LCU block and all CU blocks obtained only by QT partitioning, that is, perform pixel-by-pixel motion estimation only on CU blocks of 64x64, 32x32, 16x16, 8x8, and 4x4 sizes. Figure 4 The CU block of size 16x16 obtained only by the QT partitioning method is shown in FIG.

[0063] (2) Then, for LCU and CU blocks obtained by QT partitioning, that is, square CU blocks, it is necessary to determine whether pixel-by-pixel motion estimation is required for CU blocks obtained by BT or EQT partitioning. Whether pixel-by-pixel motion estimation is required for square CU blocks that can be partitioned by BT or EQT depends on the discreteness and threshold of their integer pixel motion vectors (IMVs).

[0064] IMV discreteness refers to the dispersion degree of the integer pixel motion vectors of the divided CU. If their integer pixel motion vectors are too dispersed, each divided CU needs to perform sub-pixel motion estimation on the current integer pixel motion vector to improve the accuracy of the motion vector. If their integer pixel motion vectors are relatively concentrated, the direction and displacement of the motion vectors of the divided sub-CU blocks are consistent with those of their parent CU blocks. The sub-pixel motion vectors of the parent CU blocks can be directly used to replace the sub-pixel motion vectors of the divided sub-CU blocks without the need for sub-pixel motion estimation of the sub-CU blocks.

[0065] For square CU blocks with sizes between 8x8 and 64x64, the judgment formula for pixel-by-pixel motion estimation of the CU blocks obtained after BT division is as follows:

[0066] ;

[0067] in, and Respectively refer to the division The displacement value of the integer pixel motion vector of the CU block on the x-axis and y-axis, and Represent their means on the x-axis and y-axis respectively, Represents the threshold value.

[0068] Similarly, for square CU blocks with sizes between 16x16 and 64x64, the CU blocks obtained after EQT division need to perform pixel-by-pixel motion estimation, which is determined by the discreteness and threshold of their IMV. The formula is as follows:

[0069]

[0070] Among them, among them, Indicates the discreteness of the integer pixel motion vector IMV of the first CU; Represents the displacement value of the integer pixel motion vector of the i-th sub-CU on the x-axis, Represents the displacement value of the ith sub-CU integer pixel motion vector on the y-axis, Represents the average displacement value of the sub-CU of the first CU on the x-axis, Represents the average displacement value of the sub-CU of the first CU on the y-axis; Represents the threshold value.

[0071] Figure 5 The whole pixel motion vector diagram of the sub-CU blocks obtained after BT or EQT block division for the current square CU block is shown. When the IMV discreteness of the sub-CU blocks divided by BT or EQT is greater than the threshold, pixel-by-pixel motion estimation is required for these divided sub-CU blocks.

[0072] In the embodiment of the present invention, sub-pixel motion vectors of all possible CU blocks are obtained by using a sub-pixel motion vector replacement method.

[0073] The sub-pixel motion vector replacement solution adopted in the embodiment of the present invention needs to follow two principles:

[0074] (1) The sub-pixel motion vectors of the alternative parties are all square CU sub-pixel motion vectors;

[0075] (2) The sub-pixel motion vector of the replaced CU block is replaced by the sub-pixel motion vector of the replacement CU block that just wraps the replaced CU block.

[0076] like Figure 6 As shown in the figure, the sub-pixel motion vector of the 8x4 CU block is replaced by the sub-pixel motion vector of the 8x8 CU block that just wraps it, rather than being replaced by the sub-pixel motion vector of the 16x16 CU block.

[0077] In one or more embodiments, Figure 7 As shown, the above-mentioned pixel-by-pixel motion estimation method based on AVS3 includes the following steps:

[0078] S702, performing pixel-by-pixel motion estimation on a square CU with a size from 4*4 to 64*64;

[0079] S704, performing BT and EQT division on the square CU that meets the division condition, and determining whether the IMV dispersion of the divided CU exceeds a threshold;

[0080] S706, determining whether the IMV discreteness of the divided square CU exceeds a threshold, and if so, performing pixel-by-pixel motion estimation on the divided CU;

[0081] S708: Sub-pixel motion vector replacement is required for the CU block for which sub-pixel motion estimation has not been performed.

[0082] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0083] According to another aspect of the embodiment of the present application, there is also provided an AVS3-based pixel-by-pixel motion estimation device for implementing the above-mentioned AVS3-based pixel-by-pixel motion estimation method. Figure 8As shown, the device comprises:

[0084] The estimation unit 802 performs pixel-by-pixel motion estimation on the first coding unit CU set in the current video frame to obtain a pixel-by-pixel motion vector of each first CU in the first CU set; wherein each first CU in the first CU set is square in shape.

[0085] In the embodiment of the present invention, the first coding unit in the first coding unit set is an LCU, or a CU block obtained by dividing the LCU according to a quadtree partitioning method. Figure 4 As shown, the size of the LCU is 64*64, and the size of the first coding unit in the first coding unit set can be 64*64, 32*32, 16*16, 8*8 and 4*4. The above is only an example, and the embodiment of the present invention does not make any limitation to this.

[0086] The partitioning unit 804 is configured to partition each first CU of the first CU set according to a binary tree partitioning method or an enhanced quadtree partitioning method to obtain a sub-CU corresponding to each first CU.

[0087] In the embodiment of the present invention, Figure 5 As shown in , the first CU of the first CU set is divided according to the binary tree division method to obtain Figure 5 The two sub-CUs corresponding to the first CU shown in (a); Figure 5 As shown, the first CU of the first CU set is divided according to the enhanced quadtree division method to obtain Figure 5 The four sub-CUs corresponding to each first CU shown in (b).

[0088] The first replacement unit 806 is used to use the sub-pixel motion vector of the first CU as the sub-pixel motion vector of each sub-CU in the first CU when it is determined that the discreteness of the integer pixel motion vector of the first CU is less than a preset threshold; the discreteness is used to indicate the degree of dispersion of the integer pixel motion vectors of multiple sub-CUs in the first CU.

[0089] In the embodiment of the present invention, Figure 5 As shown, when it is determined that the discreteness of the integer pixel motion vector IMV of the first CU is less than the preset threshold, that is, the integer pixel motion vectors of the multiple sub-CUs after the first CU is divided are relatively concentrated, and the integer pixel motion vectors of the sub-CUs are not much different. At this time, the integer pixel motion vector IMV of the first CU can be used as the IMV of each sub-CU in the first CU.

[0090] Preferably, the sub-pixel motion vector of the replaced sub-CU is replaced by the sub-pixel motion vector of the replacement CU block that just wraps the replaced sub-CU block. Figure 6 As shown in the figure, the sub-pixel motion vector of the 8x4 CU block is replaced by the sub-pixel motion vector of the 8x8 CU block that just wraps it, rather than being replaced by the sub-pixel motion vector of the 16x16 CU block.

[0091] The first acquisition unit 808 is used to perform pixel-by-pixel motion estimation on each sub-CU in the first CU to obtain the pixel-by-pixel motion vector of each sub-CU when it is determined that the discreteness of the IMV of the first CU is greater than a preset threshold.

[0092] In an embodiment of the present invention, when the discreteness of the IMV of the first CU is greater than a preset threshold, it means that the integer pixel motion vectors of the multiple sub-CUs after the first CU is divided are relatively dispersed, and the integer pixel motion vectors of the sub-CUs are quite different. At this time, the integer pixel motion vector IMV of the first CU cannot be used as the IMV of each sub-CU in the first CU, and each sub-CU needs to be individually estimated in terms of pixel motion.

[0093] In an embodiment of the present application, pixel-by-pixel motion estimation is performed on the first coding unit CU set in the current video frame to obtain the pixel-by-pixel motion vector of each first CU in the first CU set; wherein, the shape of each first CU in the first CU set is a square; each first CU in the first CU set is divided according to a binary tree partitioning method or an enhanced quadtree partitioning method to obtain a sub-CU corresponding to each first CU; when it is determined that the discreteness of the integer pixel motion vector IMV of the first CU is less than a preset threshold, the IMV of the first CU is used as the IMV of each sub-CU in the first CU; the discreteness is used to indicate the degree of dispersion of the integer pixel motion vectors of multiple sub-CUs in the first CU; when it is determined that the discreteness of the IMV of the first CU is greater than the preset threshold, pixel-by-pixel motion estimation is performed on each sub-CU in the first CU to obtain the pixel-by-pixel motion vector of each sub-CU. Since whether to perform pixel-by-pixel motion estimation on the sub-unit of the coding unit is determined according to the discreteness of the integer-pixel motion vector of the coding unit, and there is no need to perform pixel-by-pixel estimation on each coding unit and sub-unit, it can not only improve the efficiency of pixel-by-pixel motion estimation, reduce the time of pixel-by-pixel motion estimation, and save system computing resources, but also solve the technical problem of low efficiency in determining the prediction mode of CU in the related technology.

[0094] In one or more embodiments, the above-mentioned pixel-by-pixel motion estimation device based on AVS3 further includes:

[0095] a re-dividing unit, configured to obtain a second coding unit by dividing each of the sub-coding units by a quadtree partitioning method, a binary tree partitioning method or an enhanced quadtree partitioning method;

[0096] The second replacement unit is configured to use the sub-pixel motion vector of the sub-coding unit as the sub-pixel motion vector of the second coding unit.

[0097] In one or more embodiments, the first coding unit includes one or more maximum coding units LCU and / or coding units obtained by dividing the LCU in a quadtree partitioning manner; the estimation unit 802 includes at least one of the following:

[0098] A first estimation module, configured to perform pixel-by-pixel motion estimation on the one or more maximum coding units LCU to obtain a pixel-by-pixel motion vector of each maximum coding unit LCU;

[0099] The second estimation module is used to perform pixel-by-pixel motion estimation on the coding units obtained by dividing the LCU in a quadtree division manner, and obtain the pixel-by-pixel motion vector of the CU obtained after the division.

[0100] In one or more embodiments, the above-mentioned pixel-by-pixel motion estimation device based on AVS3 further includes:

[0101] The first determining unit is configured to determine the discreteness of the integer pixel motion vector IMV of the first CU by using the following formula when the first CU is divided into a plurality of sub-CUs in a binary tree manner:

[0102]

[0103] in, Indicates the discreteness of the integer pixel motion vector IMV of the first CU; Represents the displacement value of the integer pixel motion vector of the i-th sub-CU on the x-axis, Represents the displacement value of the ith sub-CU integer pixel motion vector on the y-axis, Represents the average displacement value of the sub-CU of the first CU on the x-axis, represents the average displacement value of the sub-CU of the first CU on the y-axis, and n represents the number of sub-coding units of the first coding unit.

[0104] In one or more embodiments, the above-mentioned pixel-by-pixel motion estimation device based on AVS3 further includes:

[0105] The second determining unit is configured to determine the discreteness of the integer pixel motion vector IMV of the first CU by using the following formula when the first CU is divided into a plurality of sub-CUs in a quadtree manner:

[0106]

[0107] Among them, among them, Indicates the discreteness of the integer pixel motion vector IMV of the first CU; Represents the displacement value of the integer pixel motion vector of the i-th sub-CU on the x-axis, Represents the displacement value of the ith sub-CU integer pixel motion vector on the y-axis, Represents the average displacement value of the sub-CU of the first CU on the x-axis, represents the average displacement value of the sub-CU of the first CU on the y-axis, and n represents the number of sub-coding units of the first coding unit.

[0108] In one or more embodiments, the above-mentioned pixel-by-pixel motion estimation device based on AVS3 further includes:

[0109] The second acquisition unit is used to perform integer pixel motion estimation on the current video frame to obtain integer pixel motion vector information of the first coding unit CU set.

[0110] In one or more embodiments, in the above-mentioned AVS3-based pixel-by-pixel motion estimation device, the size of the LCU is 64x64.

[0111] According to another aspect of the embodiment of the present application, an electronic device for implementing the above-mentioned pixel-by-pixel motion estimation method based on AVS3 is also provided. The electronic device may be Figure 1 The terminal device or server shown in the figure. This embodiment is described by taking the electronic device as a server as an example. Fig. 9 As shown, the electronic device includes a memory 902 and a processor 904. The memory 902 stores a computer program, and the processor 904 is configured to execute the steps in any of the above method embodiments through the computer program.

[0112] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0113] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0114] S1, performing pixel-by-pixel motion estimation on a first coding unit CU set in a current video frame to obtain a pixel-by-pixel motion vector of each first CU in the first CU set; wherein each first CU in the first CU set is in a square shape;

[0115] S2, dividing each first CU of the first CU set according to a binary tree partitioning method or an enhanced quadtree partitioning method to obtain a sub-CU corresponding to each first CU;

[0116] S3, when it is determined that the discreteness of the integer pixel motion vector of the first CU is less than a preset threshold, using the sub-pixel motion vector of the first CU as the sub-pixel motion vector of each sub-CU in the first CU; the discreteness is used to indicate the dispersion degree of the integer pixel motion vectors of multiple sub-CUs in the first CU;

[0117] S4: When it is determined that the discreteness of the IMV of the first CU is greater than a preset threshold, perform pixel-by-pixel motion estimation on each sub-CU in the first CU to obtain a pixel-by-pixel motion vector of each sub-CU.

[0118] Alternatively, a person skilled in the art may understand that: Fig. 9 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, and other terminal devices. Fig. 9 The electronic device and the electronic equipment described above are not limited in structure. Fig. 9 More or fewer components (such as network interfaces, etc.) as shown in, or with Fig. 9 Different configurations are shown.

[0119] Among them, the memory 902 can be used to store software programs and modules, such as the program instructions / modules corresponding to the pixel-by-pixel motion estimation method and device based on AVS3 in the embodiment of the present application. The processor 904 executes various functional applications and data processing by running the software programs and modules stored in the memory 902, that is, to implement the above-mentioned pixel-by-pixel motion estimation method based on AVS3. The memory 902 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 902 may further include a memory remotely arranged relative to the processor 904, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 902 can be specifically, but not limited to, used to store the pixel-by-pixel motion vectors of the first CU. As an example, such as Fig. 9 As shown, the memory 902 may include, but is not limited to, the estimation unit 802, the division unit 804, the first replacement unit 806, and the first acquisition unit 808 in the AVS3-based sub-pixel motion estimation device. In addition, other module units in the AVS3-based sub-pixel motion estimation device may also be included but are not limited to, which will not be described in detail in this example.

[0120] Optionally, the transmission device 906 is used to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one example, the transmission device 906 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 906 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0121] In addition, the electronic device further includes: a display 908 for displaying the sub-pixel motion vector of the first CU; and a connection bus 910 for connecting various module components in the electronic device.

[0122] In other embodiments, the terminal device or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes through network communication. Among them, the nodes may form a peer-to-peer (P2P, Peer To Peer) network, and any form of computing device, such as a server, terminal and other electronic devices, may become a node in the blockchain system by joining the peer-to-peer network.

[0123] In one or more embodiments, the present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-mentioned pixel-by-pixel motion estimation method based on AVS3. The computer program is configured to execute the steps of any of the above-mentioned method embodiments when it is run.

[0124] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0125] S1, performing pixel-by-pixel motion estimation on a first coding unit CU set in a current video frame to obtain a pixel-by-pixel motion vector of each first CU in the first CU set; wherein each first CU in the first CU set is in a square shape;

[0126] S2, dividing each first CU of the first CU set according to a binary tree partitioning method or an enhanced quadtree partitioning method to obtain a sub-CU corresponding to each first CU;

[0127] S3, when it is determined that the discreteness of the integer pixel motion vector of the first CU is less than a preset threshold, using the sub-pixel motion vector of the first CU as the sub-pixel motion vector of each sub-CU in the first CU; the discreteness is used to indicate the dispersion degree of the integer pixel motion vectors of multiple sub-CUs in the first CU;

[0128] S4: When it is determined that the discreteness of the IMV of the first CU is greater than a preset threshold, perform pixel-by-pixel motion estimation on each sub-CU in the first CU to obtain a pixel-by-pixel motion vector of each sub-CU.

[0129] Optionally, in this embodiment, a person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium can include: a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc. The serial numbers of the above embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0130] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which may be personal computers, servers or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0131] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0132] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0133] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0134] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0135] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A pixel-by-pixel motion estimation method based on AVS3, characterized in that: include: Perform pixel-by-pixel motion estimation on a first coding unit set in the current video frame to obtain a pixel-by-pixel motion vector of each first coding unit in the first coding unit set; wherein each first coding unit in the first coding unit set is in a square shape; Divide each first coding unit of the first coding unit set according to a binary tree partitioning method or an enhanced quadtree partitioning method to obtain a sub-coding unit corresponding to each first coding unit; When it is determined that the discreteness of the integer pixel motion vector of the first coding unit is less than a preset threshold, the sub-pixel motion vector of the first coding unit is used as the sub-pixel motion vector of each sub-coding unit in the first coding unit; the discreteness is used to indicate the dispersion degree of the integer pixel motion vectors of multiple sub-coding units in the first coding unit; When it is determined that the discreteness of the integer pixel motion vector of the first coding unit is greater than a preset threshold, sub-pixel motion estimation is performed on each sub-coding unit in the first coding unit to obtain the sub-pixel motion vector of each sub-coding unit.

2. The method according to claim 1, characterized in that The method further comprises: Partition each of the sub-coding units by a quadtree, a binary tree, or an enhanced quadtree to obtain a second coding unit; The sub-pixel motion vector of the sub-coding unit is used as the sub-pixel motion vector of the second coding unit.

3. The method according to claim 1, characterized in that The first coding unit includes one or more maximum coding units LCU and / or coding units obtained by dividing the LCU in a quadtree partitioning manner, and performing pixel-by-pixel motion estimation on a set of first coding units CU in the current video frame to obtain a pixel-by-pixel motion vector of each first coding unit in the set of first coding units includes at least one of the following: Perform pixel-by-pixel motion estimation on the one or more maximum coding units LCU to obtain a pixel-by-pixel motion vector of each maximum coding unit LCU; Perform pixel-by-pixel motion estimation on the coding units obtained by dividing the maximum coding unit LCU in a quadtree division manner to obtain pixel-by-pixel motion vectors of the coding units obtained after the division.

4. The method according to claim 1, characterized in that: Before taking the sub-pixel motion vector of the first coding unit as the sub-pixel motion vector of each sub-coding unit in the first coding unit when it is determined that the discreteness of the integer-pixel motion vector of the first coding unit is less than a preset threshold, the method further includes: When the first coding unit is divided into a plurality of sub-coding units in a binary tree manner, the discreteness of the integer pixel motion vector of the first coding unit is determined by the following formula: in, Indicates the discreteness of the integer pixel motion vector of the first coding unit; Represents the displacement value of the integer pixel motion vector of the i-th sub-coding unit on the x-axis, Represents the displacement value of the integer pixel motion vector of the i-th sub-coding unit on the y-axis, represents the mean value of the displacement values ​​of the sub-coding units of the first coding unit on the x-axis, represents the average of the displacement values ​​of the sub-coding units of the first coding unit on the y-axis, and n represents the number of the sub-coding units of the first coding unit.

5. The method according to claim 1, characterized in that Before taking the sub-pixel motion vector of the first coding unit as the sub-pixel motion vector of each sub-coding unit in the first coding unit when it is determined that the discreteness of the integer-pixel motion vector of the first coding unit is less than a preset threshold, the method further includes: When the first coding unit is divided into a plurality of sub-coding units in an enhanced quadtree manner, the discreteness of the integer pixel motion vector IMV of the first coding unit is determined by the following formula: in, Indicates the discreteness of the integer pixel motion vector of the first coding unit; Represents the displacement value of the integer pixel motion vector of the i-th sub-coding unit on the x-axis, Represents the displacement value of the integer pixel motion vector of the i-th sub-coding unit on the y-axis, represents the mean value of the displacement values ​​of the sub-coding units of the first coding unit on the x-axis, represents the average of the displacement values ​​of the sub-coding units of the first coding unit on the y-axis, and n represents the number of the sub-coding units of the first coding unit.

6. The method according to claim 1, characterized in that Before performing pixel-by-pixel motion estimation on the first coding unit set in the current video frame, the method includes: Perform integer pixel motion estimation on the current video frame to obtain integer pixel motion vector information of the first coding unit set.

7. The method according to claim 3, characterized in that The size of the maximum coding unit LCU is 64x64.

8. A pixel-by-pixel motion estimation device based on AVS3, characterized in that: include: an estimating unit, configured to perform pixel-by-pixel motion estimation on a first coding unit set in a current video frame to obtain a pixel-by-pixel motion vector of each first coding unit in the first coding unit set; wherein each first coding unit in the first coding unit set is in a square shape; a partitioning unit, configured to partition each first coding unit of the first coding unit set according to a binary tree partitioning method or an enhanced quadtree partitioning method to obtain a sub-coding unit corresponding to each first coding unit; a first replacing unit, configured to use the sub-pixel motion vector of the first coding unit as the sub-pixel motion vector of each sub-coding unit in the first coding unit when it is determined that the discreteness of the integer-pixel motion vector of the first coding unit is less than a preset threshold; the discreteness is used to indicate the dispersion degree of the integer-pixel motion vectors of multiple sub-coding units in the first coding unit; The first acquisition unit is used to perform pixel-by-pixel motion estimation on each sub-coding unit in the first coding unit to obtain the pixel-by-pixel motion vector of each sub-coding unit when it is determined that the discreteness of the integer pixel motion vector of the first coding unit is greater than a preset threshold.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 7 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images