Inter-frame prediction motion search judgment method, system and device

Abstract

The invention provides an inter-frame prediction motion search judgment method, system and device, and the method comprises the steps: judging the proportion of a bit number part used by a coding motion vector of an optimal motion vector obtained through integer pixel search in a total rate distortion optimization value under a certain search precision; and deciding whether an integer pixel search process under other search precision needs to be skipped or not.

Description

Technical Field

[0001] The present invention belongs to the field of image and video data compression processing, and in particular to an inter-frame prediction motion search and decision method, system and device. Background Art

[0002] Motion search and motion estimation are key components in video compression technology. They are mainly used in inter-frame prediction of video sequences. The purpose is to reduce redundant information between frames, thereby achieving the purpose of compressing the amount of video data. Motion search refers to the process of finding the best matching block in the inter-frame prediction process and finding the vector (Motion Vector) pointing from the current block to the best matching block. This process is usually performed in the encoder to find a reference frame block that is closest to the current frame to be encoded. In this way, the difference between the current frame and the reference frame (called residual) can be effectively represented instead of directly encoding the entire frame, which can significantly reduce the amount of encoded data.

[0003] Taking the AVS3 encoder as an example, the motion search process includes: Block division: divide the current frame into smaller coding units (for example, 16x16 pixels). Determine the search strategy: select the applicable motion search algorithm (such as full search, diamond search, hexagonal search, grid search, etc.) according to the characteristics of the coding unit (such as CU size, the time domain reference level of the CU, etc.). Determine the search accuracy: determine the accuracy used in the motion search process. Generally speaking, all accuracies will be tried one by one starting from high accuracy. If the rate-distortion optimization value corresponding to the optimal motion vector of the search result under a certain accuracy meets the preset threshold condition, it will trigger early termination and no lower accuracy will be tried. Motion search: According to the search strategy and search accuracy, a series of motion vectors (motion vectors) are obtained. The position of the candidate block is determined by the motion vector, and the difference between the current block and the candidate block (such as SAD-Sum of Absolute Differences, SSD-Sum of Squared Differences, etc.) is calculated as the distortion corresponding to the motion vector. At the same time, the number of bits required to encode the motion vector is calculated. The rate-distortion optimization value corresponding to the motion vector is obtained based on the distortion, the number of bits and the Lagrange multiplier. Select the best match: Among the rate-distortion optimization values ​​corresponding to a series of motion vectors, select the motion vector corresponding to the smallest rate-distortion optimization value as the optimal motion vector obtained by the motion search.

[0004] When the existing AVS3 encoder performs motion search and uses multiple rounds of searches with different search precisions, sometimes the process of searching for the optimal motion vector under different search precisions is extremely similar, resulting in long encoding time and low efficiency. Summary of the invention

[0005] The present invention aims to solve the problem that the encoding time of the existing encoder is long when performing motion search.

[0006] The present invention provides an inter-frame prediction motion search and decision method, system and device.

[0007] An inter-frame prediction motion search and decision method, comprising:

[0008] Step 1, obtaining the search accuracy, search mode and search step of the encoder during the motion search process, setting the corresponding terminator in the coding unit according to the search mode and search step, and setting all terminators to A when the first search accuracy is used in each coding unit;

[0009] Step 2: The encoder uses the first search accuracy to perform integer pixel search, obtains the optimal motion vector corresponding to the first search accuracy, the rate-distortion optimization value corresponding to the optimal motion vector, and the product of the number of bits used to encode the optimal motion vector and the Lagrange multiplier, and saves the above three data; calculates the ratio of the obtained product to the obtained rate-distortion optimization value, and if the ratio meets the preset requirements, sets the terminator corresponding to the search mode and the search step of the first integer pixel search to B;

[0010] Step 3, perform integer pixel search according to the next search accuracy, first judge the search mode and search step length, if it is the search mode and search step length that have been used in the previous search process, and the corresponding early termination flag is B, then do not perform the integer pixel search under the current search accuracy, use the saved optimal motion vector as the optimal motion vector of the current integer pixel search, use the saved rate-distortion optimization value as the rate-distortion optimization value of the current integer pixel search, and use the optimal motion vector as the search starting point to perform a search with a lower step length; otherwise, obtain the optimal motion vector corresponding to the current search accuracy, the rate-distortion optimization value corresponding to the optimal motion vector, and the product of the number of bits used to encode the optimal motion vector and the Lagrange multiplier, save the above three data, if the ratio of the obtained product to the obtained rate-distortion optimization value meets the preset requirements, set the search mode and the termination flag corresponding to the search step length of the integer pixel search for the current search accuracy to B;

[0011] Step 4: Repeat step 3 until all search precisions complete the integer pixel search.

[0012] Preferably, in step 2, the rate-distortion optimization value is obtained by summing the product of the number of bits consumed by encoding the motion vector and the Lagrange multiplier and the distortion value of the current coding unit.

[0013] Preferably, the current coding unit distortion value includes SAD or STAD or MSE or SSE or SATD or any combination of the above values.

[0014] Preferably, in step 1, the search step size includes 1, 2 and 4.

[0015] Preferably, in step 1, the search methods of the integer pixel search include full search, hexagon search, diamond search and grid search.

[0016] Preferably, in step 1, the search precision includes 1 / 4 pixel precision, 1 / 2 pixel precision, 1 pixel precision, 2 pixel precision and 4 pixel precision, and the integer pixel search of each precision is performed in order from high precision to low precision.

[0017] An inter-frame prediction motion search and decision system is used to implement the inter-frame prediction motion search and decision method, including: a statistical module, used to obtain all search accuracies that can be tried in the motion search process, the search methods and search steps of all integer pixel searches; a decision module, used to calculate the product of the number of bits used by the optimal motion vector and the Lagrange multiplier; a matching module, used to determine the final search result.

[0018] An inter-frame prediction motion search and decision device comprises a storage device and a processing device, wherein the storage device is used to store a program for implementing the inter-frame prediction motion search and decision method, and the processing device comprises the encoder.

[0019] Compared with the prior art, the present invention has the following advantages and effects:

[0020] The present invention determines the proportion of the number of bits used in the coded motion vector of the optimal motion vector obtained by integer pixel search under a certain search accuracy to the total rate distortion optimization value, and decides whether it is necessary to skip the integer pixel search process under other search accuracy, thereby reducing the number of RDOs in the motion search stage and saving encoding time. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0022] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0024] like Figure 1 As shown, an inter-frame prediction motion search and decision method includes:

[0025] Step 1, obtaining the search accuracy, search mode and search step of the encoder during the motion search process, setting the corresponding terminator in the coding unit according to the search mode and search step, and setting all terminators to A when the first search accuracy is used in each coding unit;

[0026] Step 2: The encoder uses the first search accuracy to perform integer pixel search, obtains the optimal motion vector corresponding to the first search accuracy, the rate-distortion optimization value corresponding to the optimal motion vector, and the product of the number of bits used to encode the optimal motion vector and the Lagrange multiplier, and saves the above three data; calculates the ratio of the obtained product to the obtained rate-distortion optimization value, and if the ratio meets the preset requirements, sets the terminator corresponding to the search mode and the search step of the first integer pixel search to B;

[0027] Step 3, perform integer pixel search according to the next search accuracy, first judge the search mode and search step length, if it is the search mode and search step length that have been used in the previous search process, and the corresponding early termination flag is B, then do not perform the integer pixel search under the current search accuracy, use the saved optimal motion vector as the optimal motion vector of the current integer pixel search, use the saved rate-distortion optimization value as the rate-distortion optimization value of the current integer pixel search, and use the optimal motion vector as the search starting point to perform a search with a lower step length; otherwise, obtain the optimal motion vector corresponding to the current search accuracy, the rate-distortion optimization value corresponding to the optimal motion vector, and the product of the number of bits used to encode the optimal motion vector and the Lagrange multiplier, save the above three data, if the ratio of the obtained product to the obtained rate-distortion optimization value meets the preset requirements, set the search mode and the termination flag corresponding to the search step length of the integer pixel search for the current search accuracy to B;

[0028] Step 4: Repeat step 3 until all search precisions complete the integer pixel search.

[0029] In a specific embodiment, in step 1, the search accuracy includes at least one or more of 1 / 4 pixel accuracy, 1 / 2 pixel accuracy, 1 pixel accuracy, 2 pixel accuracy and 4 pixel accuracy. The search mode of integer pixel search includes at least one or more of full search, hexagon search, diamond search and grid search. The search step size includes at least one or more of 1, 2 and 4, wherein a search step size of 1 represents that when searching for integer pixels, the motion vector is changed with 1 pixel as the minimum unit, a search step size of 2 represents that when searching for integer pixels, the motion vector is changed with 2 pixels as the minimum unit, and similarly, a search step size of n represents that when searching for integer pixels, the motion vector is changed with n pixels as the minimum unit.

[0030] In a specific implementation, in step 1, the applicable motion search method and search accuracy range are selected according to conditions such as the block size and the temporal reference level of the coding unit. Full search has the largest number of search points and the best search results, but it takes the longest time. Diamond search is simpler than full search, and hexagonal search is simpler than diamond search. Therefore, in order to save encoding time, the encoder generally uses diamond search or hexagonal search during the encoding process if a frame image is not referenced by other images (that is, when the temporal level is relatively high). If the image is referenced by many other images (that is, when the temporal level is relatively low), full search should be used. If the current block size is relatively large, full search will also be time-consuming, so for blocks with relatively large sizes, simpler diamond search or hexagonal search should also be used. For example, by setting some thresholds, when the temporal level L<threshold C and the block size is also less than another threshold, full search is used, otherwise diamond search is used. When C<temporal level L<threshold D, diamond search is used, and when the temporal level L>D, hexagonal search is used.

[0031] In a specific implementation, A is 0 and B is 1. The rate-distortion optimization value is calculated as follows: RD_Cost=Bit_cnt*λ+Dist, where RD_Cost is the rate-distortion optimization value, Bit_cnt is the number of bits used to encode the motion vector, λ is the Lagrange multiplier, which is determined by the qp of the macroblock where the current coding unit is located, and Dist is the distortion value of the previous coding unit and the prediction unit.

[0032] In a specific implementation, in step 1, the encoder will try three search precisions, namely 1 / 4 pixel precision, 1 / 2 pixel precision and 1 pixel precision, in the process of motion search. And when trying each pixel precision, a full search with a search step of 1 will be performed to obtain the optimal integer pixel motion vector, and all early termination flags will be set to 0 when each CU tries the first search precision. Step 2, the encoder first tries 1 / 4 pixel precision, and the current coding unit obtains the optimal integer pixel motion vector and the rate-distortion optimization value corresponding to the optimal integer pixel motion vector through a full search with a step of 1, and calculates the product of the number of bits used to encode the motion vector and the Lagrange multiplier. Step 3, the preset threshold is 10%, and the ratio of the product obtained in step 2 to the rate-distortion optimization value obtained in step 2 is 5%, which is less than the preset threshold, and the early termination flag corresponding to the full search with a step of 1 is set to 1, and the optimal integer pixel motion vector and the rate-distortion optimization value of the optimal integer pixel motion vector are saved at the same time. Step 4. When trying 1 / 2 pixel accuracy, a full search with a step size of 1 is also required. First, determine whether the early termination flag corresponding to the full search with a step size of 1 is 1. In this case, it is 1, so skip the full search with a step size of 1. Use the previously saved optimal integer pixel motion vector as the optimal motion vector obtained by this integer pixel search, use the previously saved rate-distortion optimization value as the optimal rate-distortion optimization value obtained by this integer pixel search, and then use the optimal motion vector of the integer pixel as the search starting point for sub-pixel search.

[0033] In a specific implementation, in step 1, the encoder will sequentially try three search precisions, namely 1 / 4 pixel precision, 1 / 2 pixel precision and 1 pixel precision, during the motion search process. And when trying each pixel precision, a full search with a search step of 1 will be performed first to obtain the optimal integer pixel motion vector, and all identifiers will be set to 0 when trying the first search precision for each CU. In step 2, the encoder first tries 1 / 4 pixel precision, and the current coding unit obtains the optimal integer pixel motion vector and the rate-distortion optimization value corresponding to the optimal integer pixel motion vector through a full search with a step of 1, and calculates the product of the number of bits used to encode the motion vector and the Lagrange multiplier. In step 3, the preset threshold is 5%, and the ratio of the product obtained in step 2 to the rate-distortion optimization value obtained in step 2 is calculated to be 10%. If this ratio is greater than the preset threshold, the early termination identifier corresponding to the full search with a step of 1 will not be set to 1. Step 4, when trying 1 / 2 pixel accuracy, a full search with a step size of 1 is also required. First, determine whether the early termination flag corresponding to the full search with a step size of 1 is 1. In this case, if it is not 1, the current coding unit does not skip the full search with a step size of 1, executes the search and obtains the optimal integer pixel motion vector. When trying to search with an accuracy of 1, the full search with a step size of 1 is not skipped.

[0034] In a specific implementation, in step 1, the encoder will sequentially try three search precisions, namely, 1 pixel precision, 2 pixel precision, and 4 pixel precision, in the process of motion search. And when trying each pixel precision, a diamond search with a search step of 4 will be performed to obtain the local optimal motion vector, and all identifiers will be set to 0 when each CU tries the first search precision. Step 2, the encoder first tries 1 pixel precision, and the current coding unit obtains the local optimal motion vector and the rate-distortion optimization value corresponding to the local optimal motion vector through a diamond search with a step of 4, and calculates the product of the number of bits used to encode the motion vector and the Lagrange multiplier. Step 3, the preset threshold is 10%, and the ratio of the product obtained in step 2 to the rate-distortion optimization value obtained in step 2 is 5%, which is less than the preset threshold. The early termination identifier corresponding to the diamond search with a step of 4 is set to 1, and the local optimal motion vector and the rate-distortion optimization value of the local optimal motion vector are saved at the same time. Step 4. When trying 2-pixel accuracy, a diamond search with a step size of 4 is also required. First, determine whether the early termination flag corresponding to the diamond search with a step size of 4 is 1. In this case, it is 1, so skip the diamond search with a step size of 4. Use the previously saved local optimal motion vector as the local optimal motion vector obtained by this diamond search with a step size of 4. Use the previously saved rate-distortion optimization value as the optimal rate-distortion optimization value obtained by this search, and then use the optimal motion vector as the search starting point to perform a search with a lower step size.

[0035] In a specific implementation, in step 1, the encoder will sequentially try three search precisions, namely 1 pixel precision, 2 pixel precision and 4 pixel precision, in the process of motion search. And when trying each pixel precision, a diamond search with a search step of 4 will be performed to obtain a local optimal motion vector first, and all identifiers will be set to 0 when each CU tries the first search precision. In step 2, the encoder first tries 1 pixel precision, and the current coding unit obtains a local optimal motion vector and a rate-distortion optimization value corresponding to the local optimal motion vector through a diamond search with a step of 4, and calculates the product of the number of bits used to encode the motion vector and the Lagrange multiplier. In step 3, the preset threshold is 5%, and the ratio of the product obtained in step 2 to the rate-distortion optimization value obtained in step 2 is 10%. If this ratio is greater than the preset threshold, the early termination identifier corresponding to the full search with a step of 4 will not be set to 1. Step 4, when trying 2-pixel accuracy, a diamond search with a step size of 4 is also required. First, determine whether the early termination flag corresponding to the full search with a step size of 4 is 1. In this case, it is not 1, and the diamond search with a step size of 4 is not skipped. The search is performed to obtain the local optimal motion vector. When the subsequent search accuracy is tried to be 1, the diamond search with a step size of 4 is not skipped.

[0036] An inter-frame prediction motion search and decision system is used to implement the inter-frame prediction motion search and decision method, including: a statistical module, used to obtain all search accuracies that can be tried in the motion search process, the search methods and search steps of all integer pixel searches; a decision module, used to calculate the product of the number of bits used by the optimal motion vector and the Lagrange multiplier; a matching module, used to determine the final search result.

[0037] An inter-frame prediction motion search and decision device comprises a storage device and a processing device, wherein the storage device is used to store a program for implementing the inter-frame prediction motion search and decision method, and the processing device comprises the encoder.

[0038] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for inter-frame prediction motion search and decision, characterized in that: include: Step 1, obtaining the search accuracy, search mode and search step of the encoder during the motion search process, setting the corresponding terminator in the coding unit according to the search mode and search step, and setting all terminators to A when the first search accuracy is used in each coding unit; Step 2: The encoder uses the first search accuracy to perform integer pixel search, obtains the optimal motion vector corresponding to the first search accuracy, the rate-distortion optimization value corresponding to the optimal motion vector, and the product of the number of bits used to encode the optimal motion vector and the Lagrange multiplier, and saves the above three data; calculates the ratio of the obtained product to the obtained rate-distortion optimization value, and if the ratio meets the preset requirements, sets the terminator corresponding to the search mode and the search step of the first integer pixel search to B; Step 3: Perform integer pixel search according to the next search accuracy. First, determine the search mode and search step size. If the search mode and search step size have been used in the previous search process, and the early termination flag corresponding to the used search mode and search step size is B, then the integer pixel search under the current search accuracy is not performed. The saved optimal motion vector is used as the optimal motion vector of the current integer pixel search, and the saved rate-distortion optimization value is used as the rate-distortion optimization value of the current integer pixel search. The optimal motion vector is used as the search starting point to perform a search with a lower step size. Otherwise, the optimal motion vector corresponding to the current search precision, the rate-distortion optimization value corresponding to the optimal motion vector, and the product of the number of bits used to encode the optimal motion vector and the Lagrange multiplier are obtained, and the above three data are saved. If the ratio of the obtained product to the obtained rate-distortion optimization value meets the preset requirements, the search mode for integer pixel search of the current search precision and the terminator corresponding to the search step length are set to B; Step 4: Repeat step 3 until all search precisions complete the integer pixel search.

2. The inter-frame prediction motion search and decision method according to claim 1, characterized in that: In step 2, the rate-distortion optimization value is obtained by summing the product of the number of bits consumed by encoding the motion vector and the Lagrange multiplier and the distortion value of the current coding unit.

3. The inter-frame prediction motion search and decision method according to claim 1, characterized in that: The current coding unit distortion value includes SAD or STAD or MSE or SSE or SATD or any combination of the above values.

4. The inter-frame prediction motion search and decision method according to claim 1, characterized in that: In step 1, the search step lengths include 1, 2 and 4.

5. The inter-frame prediction motion search and decision method according to claim 1, characterized in that: In step 1, the search methods of the integer pixel search include full search, hexagon search, diamond search and grid search.

6. The inter-frame prediction motion search and decision method according to claim 1, characterized in that: In step 1, the search precision includes 1 / 4 pixel precision, 1 / 2 pixel precision, 1 pixel precision, 2 pixel precision and 4 pixel precision, and the whole pixel search of each precision is performed in order from high precision to low precision.

7. An inter-frame prediction motion search and decision system, used to implement the inter-frame prediction motion search and decision method according to any one of claims 1 to 6, characterized in that: include: The statistical module is used to obtain the search accuracy, the search method and the search step size of the integer pixel search during the motion search process; the decision module is used to calculate the product of the number of bits used by the optimal motion vector and the Lagrange multiplier; the matching module is used to determine the final search result.

8. An inter-frame prediction motion search and decision device, characterized in that: It comprises a storage device and a processing device, wherein the storage device is used to store a program for implementing the inter-frame prediction motion search and decision method described in any one of claims 1-6, and the processing device comprises an encoder described in any one of claims 1-6.

Images