Motion estimation method, device, electronic device and storage medium

Abstract

The present application discloses a motion estimation method, device and electronic device, the method comprising: using a preset integer pixel search algorithm to search for an initial matching block that matches a prediction block in a coding frame from a reference frame; obtaining a candidate block set corresponding to the initial matching block from the reference frame, wherein the candidate block set includes at least one candidate block, and the candidate block is composed of integer pixels located within a preset range of the initial matching block; obtaining a target matching block according to the initial matching block and the candidate block set. The method can effectively reduce the complexity of motion estimation under the premise of limited coding quality loss, so as to improve the overall coding speed of video coding processing.

Description

Technical Field

[0001] The present disclosure relates to the technical field of video encoding, and more specifically, to a motion estimation method, device, electronic device, and computer-readable storage medium. Background Art

[0002] The purpose of video coding (also known as video compression) is to eliminate redundant information between video sequences. In the process of video coding, prediction is required. Prediction is divided into intra-frame prediction and inter-frame prediction. The most critical technology in inter-frame prediction is motion estimation (ME, Motion Estimation). Motion estimation refers to the process of searching for the best matching block in the reference frame for the current prediction block to be predicted in the current coding frame, and determining the motion vector (MV, Motion Vector) between the current prediction block and the best matching block. In the process of video coding, motion estimation is used to find a matching block that matches the prediction block, and the prediction block is encoded based on the matching block, which can effectively reduce the temporal correlation between video frames and greatly improve the efficiency of video compression.

[0003] Motion estimation is very complex in the entire video encoding process. Therefore, how to effectively reduce the complexity of motion estimation under the premise of limited encoding quality loss to improve the overall encoding speed of video encoding has become a problem that needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present disclosure provide a motion estimation method, an apparatus, an electronic device, and a storage medium, which can effectively reduce the complexity of motion estimation under the premise of limited coding quality loss.

[0005] In a first aspect of the present disclosure, a motion estimation method is provided, the method comprising:

[0006] Searching for an initial matching block matching the prediction block in the coding frame from the reference frame using a preset integer pixel search algorithm;

[0007] Acquire a candidate block set corresponding to the initial matching block from the reference frame, wherein the candidate block set includes at least one candidate block, and the candidate block is composed of integer pixels located within a preset range of the initial matching block;

[0008] A target matching block is obtained according to the initial matching block and the candidate block set.

[0009] According to a second aspect of the present disclosure, there is further provided a motion estimation device, comprising:

[0010] An initial matching block search module, used to search for an initial matching block matching the prediction block in the coding frame from the reference frame using a preset integer pixel search algorithm;

[0011] A candidate block set acquisition module, configured to acquire a candidate block set corresponding to the initial matching block from the reference frame, wherein the candidate block set includes at least one candidate block, and the candidate block is composed of integer pixels located within a preset range of the initial matching block;

[0012] The target matching block obtaining module is used to obtain a target matching block according to the initial matching block and the candidate block set.

[0013] According to a third aspect of the present disclosure, there is also provided an electronic device, including:

[0014] A memory for storing executable instructions;

[0015] The processor is used to operate the electronic device to execute the method according to the first aspect of the present disclosure under the control of the executable instructions.

[0016] According to a fourth aspect of the present disclosure, a computer-readable storage medium is further provided, wherein the computer-readable storage medium stores a computer program that can be read and executed by a computer, and the computer program is used to execute the method according to the first aspect of the present disclosure when read and executed by the computer.

[0017] The beneficial effect of the present disclosure is that, unlike the prior art, when performing motion estimation, it is necessary to first search for an initial matching block matching the prediction block from the reference frame based on an integer pixel search algorithm, and then search for a target matching block from the sub-pixel candidate blocks corresponding to the initial matching block based on a sub-pixel search algorithm, the motion estimation method provided by the embodiment of the present disclosure, after the electronic device obtains the initial matching block, it is not necessary to first perform a sub-pixel search process, but by obtaining at least one candidate block composed of integer pixels within a preset range of the initial matching block from the reference frame, and based on the candidate block set including the at least one candidate block and the initial matching block, the target matching block can be obtained under the premise of limited coding quality loss. This method can reduce the complexity of motion estimation and improve the overall coding speed of video coding processing.

[0018] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0020] Figure 1 It is a first framework schematic diagram of an existing motion estimation method.

[0021] Figure 2 It is a schematic diagram of searching for matching blocks with integer pixels.

[0022] Figure 3 Schematic diagram of a second framework of an existing motion estimation method.

[0023] Figure 4 It is a schematic diagram of constructing sub-pixels.

[0024] Figure 5 It is a flowchart of the motion estimation method provided by an embodiment of the present disclosure.

[0025] Figure 6 It is a principle block diagram of a motion estimation device provided by an embodiment of the present disclosure.

[0026] Figure 7 It is a schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0028] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0029] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0030] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0031] The terms "first", "second", etc. in the specification and claims of the present disclosure are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure can be implemented in an order other than those illustrated or described herein.

[0032] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0033] <Method Example>

[0034] The video sequences in the video data usually have strong correlation in the time dimension, that is, the contents between multiple consecutive video frames usually have strong similarities. Therefore, when transmitting video data, in order to improve the transmission speed and reduce bandwidth consumption, the video data is usually encoded and compressed based on video coding technology to reduce the temporal correlation between video sequences and reduce the amount of video data. Existing video coding technology usually divides the video frame into several coding units (CU) of preset size, and based on intra-frame prediction or inter-frame prediction technology, removes redundant information in the coding unit to effectively compress the video data, where the coding unit can also be called an image block.

[0035] Generally, in the inter-frame prediction phase of video coding, a motion estimation algorithm is generally used to search for a matching block from a reference frame that best matches the prediction block to be predicted in the current coding frame, so as to obtain the motion vector of the prediction block relative to the matching block and encode the prediction block. Figure 1 , which is a first schematic diagram of the existing motion estimation method. Figure 1 As shown, the existing motion estimation algorithm generally includes an integer pixel search submodule and a sub-pixel search submodule.

[0036] The whole pixel search submodule is used to search the matching block that best matches the prediction block in the current frame using the whole pixel search algorithm from the reference frame. Generally, the sum of absolute difference (SAD) can be used as the distortion measurement criterion for whole pixel search. Figure 2 As shown, for the prediction block to be predicted in the current frame, the matching block that best matches the prediction block can be searched from the search range shown in the black dotted box of the reference frame, and the motion vector between the two can be obtained to encode the prediction block, where the motion vector represents the displacement of the prediction block and the matching block in the horizontal and vertical directions. For example, MV(-1,1) means that the matching block is shifted 1 pixel to the left in the horizontal direction and 1 pixel down in the vertical direction relative to the prediction block. Common integer pixel search algorithms generally include: full search algorithm, diamond search algorithm, hexagonal search algorithm, cross diamond search algorithm, cross hexagonal search algorithm, etc. Among them, the full search algorithm has the highest accuracy, but also the highest complexity, and the complexity in the encoder accounts for a very large proportion. Generally, the full search algorithm is not used in a large-scale search, and other search algorithms are fast motion estimation methods, which allow the complexity of motion estimation to be greatly reduced under the premise of reducing a certain accuracy.

[0037] Please see Figure 3 , which is a second schematic diagram of the existing motion estimation method. Figure 3As shown in the figure, the sub-pixel search submodule is used to improve the accuracy of motion estimation. Based on the matching block obtained by the whole pixel search submodule, the sub-pixel is constructed based on the original whole pixel by interpolation method, so as to perform sub-pixel search at fine-grained pixel precision such as 1 / 2 pixel precision and 1 / 4 pixel precision in the vicinity of the matching block obtained by the whole pixel search submodule, so as to obtain the module of the optimal matching block. Please refer to Figure 4 , which is a schematic diagram of constructing sub-pixels, Figure 4 The circle in the middle is the pixel point at the original integer pixel position, and the triangle is the pixel point at the constructed sub-pixel position, which is generated by integer pixel interpolation.

[0038] Usually in the pixel search submodule, after constructing Figure 4 After the sub-pixel point shown, the matching cost can be calculated based on the interpolated pixel value and the pixel value of the prediction block to search for the optimal sub-pixel position according to the matching cost, wherein the matching cost here can be the absolute error sum of the pixel values ​​in the sub-pixel matching block and the prediction block, or can also be the sum of the absolute values ​​of the coefficients after Hadamard transformation (SATD, Sum of Absolute TransformedDifference). Since the interpolation process in the sub-pixel search submodule involves more multiplication operations, its complexity is relatively high, which also leads to the high complexity of the sub-pixel search process.

[0039] In order to reduce the complexity of the motion estimation process and improve the overall encoding speed of the video encoding process, the embodiment of the present disclosure provides a motion estimation method. Figure 5 , which is a flow chart of the motion estimation method provided by the embodiment of the present disclosure. The method can be implemented by an electronic device, which can be a terminal device, for example, a mobile phone, a tablet computer, etc.; or, the electronic device can also be a server, for example, a blade server, a rack server, etc., which is not particularly limited here.

[0040] like Figure 5 The method of this embodiment may include the following steps S5100-S5300, which are described in detail below.

[0041] Step S5100: Use a preset integer pixel search algorithm to search for an initial matching block in the reference frame that matches the prediction block in the coding frame.

[0042] The coding frame refers to the video frame and / or image frame currently to be coded.

[0043] The prediction block is any one of the multiple coding units obtained after splitting the coding frame into coding units of preset sizes. In the embodiments of the present disclosure, the coding units can be image blocks of 4 pixels * 4 pixels, 8 pixels * 8 pixels and 16 pixels * 16 pixels, and there is no special limitation here.

[0044] A reference frame is a video frame and / or image frame that is related to the content in a coding frame.

[0045] The initial matching block is a coding unit searched from a reference frame based on a preset integer pixel search algorithm and having the smallest distortion value with the prediction block, wherein the distortion value between the prediction block and the initial matching block represents the matching cost between the two, and the distortion value can be obtained by calculating the absolute error sum between the reference frame and the initial matching block.

[0046] In one embodiment, the preset integer pixel search algorithm may be at least one of a full search algorithm, a diamond search algorithm, a hexagonal search algorithm, a cross diamond search algorithm, and a cross hexagonal search algorithm, which is not particularly limited here.

[0047] Step S5200: acquiring a candidate block set corresponding to the initial matching block from the reference frame, wherein the candidate block set includes at least one candidate block, and the candidate block is composed of integer pixels located within a preset range of the initial matching block.

[0048] In the prior art, after searching for the initial matching block with the lowest matching cost with the prediction block from the reference frame based on the integer pixel search algorithm, in order to improve the accuracy of the matching block, it is usually interpolated based on the integer pixels in the reference frame to obtain sub-pixels, and under different sub-pixel precisions, the target matching block with a lower matching cost, that is, the optimal matching block, is searched based on the interpolated sub-pixels. However, due to the high processing complexity of interpolating the sub-pixels, the existing motion estimation algorithm directly performs sub-pixel search after the integer pixel search, which often makes the overall complexity of motion estimation higher and takes longer time.

[0049] In the process of implementing the present application, the inventors found that in some cases, the first matching cost between the initial matching block and the prediction block obtained according to the whole pixel search process, and the second matching cost between the target matching block and the prediction block obtained according to the sub-pixel search process, are often close to each other, which means that in this case, the initial matching block can actually be used directly as the target matching block without the need for subsequent sub-pixel search processing. Therefore, in an embodiment of the present disclosure, a motion estimation method is provided, which, in the motion estimation process, after obtaining the initial matching block according to the whole pixel search process, constructs an evaluation index for determining whether to perform subsequent sub-pixel search processing, so as to achieve the purpose of reducing the complexity of motion estimation by terminating the subsequent sub-pixel search processing in advance under the premise of limited coding quality loss.

[0050] In one embodiment, obtaining a set of candidate blocks corresponding to the initial matching block from the reference frame includes: obtaining an initial motion vector between a prediction block and the initial matching block; and obtaining the set of candidate blocks from the reference frame according to the initial motion vector.

[0051] The initial motion vector indicates the horizontal and vertical displacement of the prediction block and the initial matching block. For example, MV(-1,1) indicates that the matching block is horizontally shifted 1 pixel to the left and vertically shifted 1 pixel downward relative to the prediction block.

[0052] After obtaining the initial motion vector, based on the initial motion vector, the candidate motion vectors of N*M candidate blocks near the prediction block and the initial matching block can be calculated. According to the candidate motion vector, the candidate block composed of integer pixels in the reference frame and located within a preset range of the initial matching block can be easily determined, where N and M are both positive integers.

[0053] For example, when the initial motion vector is MV(-1,1), image blocks in the reference frame whose corresponding MVs with respect to the prediction block are (-2,0), (-1,0), (0,0), (-2,1), (-1,1), (0,1), (-2,2), (-1,2) and (0,2) can be used as candidate blocks to construct a candidate block set.

[0054] Step S5300: Obtain a target matching block according to the initial matching block and the candidate block set.

[0055] In one embodiment, obtaining a target matching block according to the initial matching block and the candidate block set includes: obtaining a target reference value according to the candidate block set, wherein the target reference value represents the discrete degree of the matching cost between the candidate blocks in the candidate block set and the prediction block; and determining that the initial matching block is the target matching block when the target reference value is less than a preset threshold.

[0056] In this embodiment, obtaining a target matching block according to the initial matching block and the candidate block set includes: obtaining a distortion value set by calculating a distortion value between each candidate block in the candidate block set and the prediction block, wherein the distortion value represents a matching cost between the corresponding candidate block and the prediction block; and obtaining the target reference value according to the distortion value set.

[0057] The distortion value between the candidate block and the predicted block can be obtained by calculating the absolute error value between the two.

[0058] The target reference value may be any one of the variance, mean square error, and the difference between the maximum value and the minimum value of all distortion values ​​in the distortion value set.

[0059] It should be noted that, in a specific implementation, the above-mentioned preset threshold is greater than 0, and the specific value of the preset threshold is different depending on the calculation method of the target reference value.

[0060] For example, in the case where the target reference value is the mean square error of all distortion values ​​in a distortion value set, the preset value can be the same value for coding units of any size; and in the case where the target reference value is the variance of all distortion values ​​in the distortion value set, since the variances of coding units of different sizes cannot be compared, the preset threshold needs to be linearly adjusted according to the number of pixels in the coding unit.

[0061] Specifically, after obtaining the initial matching block in step S5100 and obtaining the candidate block set consisting of candidate blocks located near the initial matching block in step S5200, the distortion value set can be obtained by respectively calculating the distortion value between each candidate block and the prediction block.

[0062] Through the distortion value set, the discrete degree of the matching cost between other image blocks near the initial matching block, that is, the candidate block and the prediction block, can be counted. If the discrete degree is small, it means that the matching costs between the integer pixel candidate blocks near the initial matching block are basically close. Even if the sub-pixel search process is performed, the result may not be better than the initial matching block. Therefore, the sub-pixel search process can be terminated in advance and the initial matching block is used as the target matching block. If the discrete degree of the distortion value in the distortion value set is large, it means that the matching cost difference between the candidate blocks composed of integer pixels near the initial matching block is large. At this time, the sub-pixel search process can be continued to ensure the accuracy of the result.

[0063] That is, in one embodiment, obtaining the target matching block according to the initial matching block and the candidate block set further includes: when the target reference value is not less than the preset threshold, searching from the reference frame using a preset pixel search algorithm according to the initial matching block to obtain the target matching block.

[0064] It should be noted that, in one embodiment, after obtaining the target matching block, the method further includes: calculating a target motion vector between the prediction block and the target matching block; and encoding the prediction block according to the target motion vector and the target matching block.

[0065] According to the above description, the motion estimation method provided by the embodiment of the present disclosure, after using a preset integer pixel search algorithm to obtain an initial matching block corresponding to a prediction block, obtains at least one candidate block located within a preset range of the initial matching block and composed of integer pixels in a reference frame, and by counting the discrete degree of the distortion value between the at least one candidate block and the prediction block, a target reference value for measuring whether the sub-pixel search process can be terminated in advance can be obtained. With the target reference value as an evaluation indicator, the sub-pixel search process can be terminated in advance in some cases, so as to achieve a technical effect of significantly reducing the complexity of motion estimation under the premise of limited coding loss quality.

[0066] After the motion estimation algorithm provided by the embodiment of the present disclosure is used for actual video encoding processing, experimental data show that the method can improve the video encoding speed by 1% on average and only cause a 0.01% objective compression efficiency loss. That is, the method can improve the encoding speed of the entire video encoding process without almost reducing the objective performance of the video encoding process and maintaining consistent subjective picture quality, thereby increasing the usage of computing resources on the server side and saving server costs. It can also improve the video decoding speed to reduce the CPU usage of the client and improve the user experience.

[0067] <Device Example>

[0068] Corresponding to the above method embodiment, in this embodiment, a motion estimation device is also provided, such as Figure 6 As described above, the apparatus 100 may include an initial matching block search module 110 , a candidate block set acquisition module 120 , and a target matching block acquisition module 130 .

[0069] The initial matching block search module 110 is used to search for an initial matching block that matches a prediction block in a coding frame from a reference frame using a preset integer pixel search algorithm; the candidate block set acquisition module 120 is used to acquire a candidate block set corresponding to the initial matching block from the reference frame, wherein the candidate block set includes at least one candidate block, and the candidate block is composed of integer pixels located within a preset range of the initial matching block; the target matching block acquisition module 130 is used to acquire a target matching block based on the initial matching block and the candidate block set.

[0070] <Equipment Embodiment>

[0071] In this embodiment, an electronic device is also provided. Figure 7 As described above, the electronic device 200 may further include a processor 220 and a memory 210, wherein the memory 210 is used to store executable instructions; the processor 220 is used to operate the electronic device according to the control of the instructions to execute the method according to any embodiment of the present disclosure.

[0072] The electronic device 200 may be a terminal device, or may also be a server, which is not limited here.

[0073] <Medium Example>

[0074] Corresponding to the above method embodiments, in this embodiment, a computer-readable storage medium is also provided, which stores a computer program that can be read and executed by a computer, and the computer program is used to execute the method described in any of the above embodiments of the present disclosure when read and executed by the computer.

[0075] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of an instruction, and the module, a program segment or a part of an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0076] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present disclosure is defined by the appended claims.

Claims

1. A motion estimation method, characterized in that: include: Searching for an initial matching block matching the prediction block in the coding frame from the reference frame using a preset integer pixel search algorithm; Acquire a candidate block set corresponding to the initial matching block from the reference frame, wherein the candidate block set includes at least one candidate block, and the candidate block is composed of integer pixels located within a preset range of the initial matching block; According to the initial matching block and the candidate block set, a target matching block is obtained, The step of obtaining a target matching block according to the initial matching block and the candidate block set includes: Obtaining a set of distortion values ​​by calculating a distortion value between each candidate block in the set of candidate blocks and the prediction block, wherein the distortion value represents a matching cost between the corresponding candidate block and the prediction block, and the distortion value is obtained by calculating an absolute error value between the corresponding candidate block and the prediction block; Obtaining a target reference value according to the distortion value set, wherein the target reference value represents a discrete degree of a matching cost between a candidate block in the candidate block set and the prediction block; When the target reference value is less than a preset threshold, the initial matching block is determined as the target matching block.

2. The method according to claim 1, characterized in that The target reference value is any one of the variance, mean square error, maximum value and minimum value difference of all distortion values ​​in the distortion value set.

3. The method according to claim 1, characterized in that The step of acquiring a candidate block set corresponding to the initial matching block from the reference frame includes: Obtaining an initial motion vector between the prediction block and the initial matching block; The candidate block set is obtained from the reference frame according to the initial motion vector.

4. The method according to claim 1, characterized in that: The step of obtaining a target matching block according to the initial matching block and the candidate block set further includes: In the case that the target reference value is not less than the preset threshold, the target matching block is searched from the reference frame using a preset pixel-wise search algorithm according to the initial matching block to obtain the target matching block.

5. The method according to claim 1, characterized in that After obtaining the target matching block, the method further includes: Calculating a target motion vector between the prediction block and the target matching block; The prediction block is encoded according to the target motion vector and the target matching block.

6. A motion estimation device, characterized in that: include: An initial matching block search module, used to search for an initial matching block matching the prediction block in the coding frame from the reference frame using a preset integer pixel search algorithm; A candidate block set acquisition module, configured to acquire a candidate block set corresponding to the initial matching block from the reference frame, wherein the candidate block set includes at least one candidate block, and the candidate block is composed of integer pixels located within a preset range of the initial matching block; A target matching block obtaining module is used to obtain a target matching block according to the initial matching block and the candidate block set. The target matching block obtaining module is specifically used for: Obtaining a set of distortion values ​​by calculating a distortion value between each candidate block in the set of candidate blocks and the prediction block, wherein the distortion value represents a matching cost between the corresponding candidate block and the prediction block, and the distortion value is obtained by calculating an absolute error value between the corresponding candidate block and the prediction block; Obtaining a target reference value according to the distortion value set, wherein the target reference value represents a discrete degree of a matching cost between a candidate block in the candidate block set and the prediction block; When the target reference value is less than a preset threshold, the initial matching block is determined as the target matching block.

7. An electronic device, characterized in that: include: A memory for storing executable instructions; A processor is used to operate the electronic device to execute the method according to any one of claims 1 to 5 under the control of the instruction.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that can be read and executed by a computer, and the computer program is used to execute the method according to any one of claims 1 to 5 when being read and executed by the computer.

Images