A motion estimation method, device and computer equipment

Abstract

The application discloses a motion estimation method, device and computer equipment. The method comprises the following steps: determining a target coding block in a target video frame; performing motion vector prediction on the target coding block by using an advanced motion vector prediction method to obtain a predicted motion vector of the target coding block; selecting a plurality of reference coding blocks of the target coding block from history coding blocks of a video sequence to which the target video frame belongs and which have been completed coding, wherein the history coding blocks comprise coding blocks that are not adjacent to the target coding block in time domain and coding blocks that are not adjacent to the target coding block in space domain; obtaining a motion vector of each reference coding block in the plurality of reference coding blocks, and taking the obtained motion vectors of the plurality of reference coding blocks as a plurality of candidate motion vectors of the target coding block; taking a pixel position pointed by a motion vector with the minimum rate-distortion cost in the plurality of candidate motion vectors and the predicted motion vector as a search starting point, and performing motion estimation on the target coding block by using the search starting point, so that the efficiency of motion estimation can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a motion estimation method, apparatus, and computer equipment. Background Technology

[0002] Computer technology has now been deeply integrated into all aspects of life. To meet the needs of daily life and improve quality of life, the development of computer technology has placed particular emphasis on digital image processing technology in the field of digital imaging. Digital image processing technology can be applied to video processing scenarios. In most video processing scenarios, motion estimation is required to further encode the video file. Motion estimation refers to searching for the image content that best matches the target coding block from the reference image, and motion estimation usually requires the use of a search starting point. Current motion estimation methods typically rely on advanced motion vector prediction methods to determine the search starting point for further video processing. However, the search starting point determined by current advanced motion vector prediction methods has low accuracy in situations such as motion occlusion and scene transitions, resulting in low efficiency in motion estimation. Therefore, how to improve the efficiency of motion estimation has become a current research hotspot. Summary of the Invention

[0003] This application provides a motion estimation method, apparatus, and computer device that can improve the efficiency of motion estimation.

[0004] On one hand, embodiments of this application provide a motion estimation method, including:

[0005] Determine the target coding block in the target video frame from which motion estimation needs to be performed;

[0006] An advanced motion vector prediction method is used to predict the motion vectors of the target coding block to obtain the predicted motion vectors of the target coding block.

[0007] Determine the historical coding blocks that have been encoded in the video sequence to which the target video frame belongs, and select multiple reference coding blocks of the target coding block from the historical coding blocks. The historical coding blocks include: coding blocks that are not temporally adjacent to the target coding block, and coding blocks that are not spatially adjacent to the target coding block.

[0008] The motion vector of each of the plurality of reference coding blocks is obtained, and the obtained motion vectors of the plurality of reference coding blocks are used as the plurality of candidate motion vectors of the target coding block;

[0009] From the plurality of candidate motion vectors and the predicted motion vectors, the pixel position pointed to by the motion vector with the minimum rate-distortion cost is selected as the search starting point, and the motion estimation of the target coding block is performed using the search starting point.

[0010] Furthermore, embodiments of this application provide a motion estimation device, including:

[0011] The determination unit is used to determine the target coding block in the target video frame that currently needs to undergo motion estimation.

[0012] The prediction unit is used to perform motion vector prediction on the target coding block using an advanced motion vector prediction method to obtain the predicted motion vector of the target coding block;

[0013] The determining unit is further configured to determine the historical coding blocks that have been encoded in the video sequence to which the target video frame belongs, and select a plurality of reference coding blocks of the target coding block from the historical coding blocks. The historical coding blocks include: coding blocks that are not temporally adjacent to the target coding block, and coding blocks that are not spatially adjacent to the target coding block.

[0014] The acquisition unit is used to acquire the motion vector of each of the plurality of reference coding blocks, and to use the acquired motion vectors of the plurality of reference coding blocks as the plurality of candidate motion vectors of the target coding block;

[0015] The processing unit is configured to select the pixel position pointed to by the motion vector with the minimum rate-distortion cost from the plurality of candidate motion vectors and the predicted motion vectors as the search starting point, and use the search starting point to perform motion estimation on the target coding block.

[0016] Furthermore, embodiments of this application also provide a computer device, including:

[0017] A processor, the processor being adapted to implement one or more computer programs;

[0018] A computer storage medium storing one or more computer programs adapted to be loaded by the processor and executed by the above-described motion estimation method.

[0019] In another aspect, embodiments of this application also provide a computer storage medium storing one or more computer programs, which are adapted to be loaded by a processor and executed by the above-described motion estimation method.

[0020] In this embodiment, when determining the search starting point, the computer device references not only the predicted motion vector measured using advanced motion vector prediction methods, but also motion vectors determined from the non-adjacent temporal domain and the non-adjacent spatial domain of the target coding block. This allows the computer device to utilize motion information from both non-adjacent temporal and spatial domains, greatly enriching the range of motion vectors available for reference. Determining the search starting point based on these abundant reference vectors effectively avoids situations where the computer device can only determine the search starting point from adjacent spatial or temporal domains, such as motion occlusion or scene transitions, thus preventing low accuracy of the search starting point. In other words, this embodiment improves the accuracy of the search starting point. Higher accuracy results in fewer search points for motion estimation, reducing the workload and efficiency of motion estimation. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of a target video frame and coded block provided in an embodiment of this application;

[0023] Figure 2a This is a schematic diagram of a pixel position that is not adjacent to the spatial domain of the target coding block, provided in an embodiment of this application.

[0024] Figure 2b This is a schematic diagram of a pixel position that is not temporally adjacent to the target coding block, provided in an embodiment of this application.

[0025] Figure 3 This is a schematic flowchart of a motion estimation method provided in an embodiment of this application;

[0026] Figure 4 This is a schematic diagram of a target coding block and an auxiliary coding block provided in an embodiment of this application;

[0027] Figure 5 This is a schematic flowchart of another motion estimation method provided in the embodiments of this application;

[0028] Figure 6 This is a schematic diagram of the structure of a motion estimation device provided in an embodiment of this application;

[0029] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the methods provided in the embodiments of this application, the technical methods in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be noted that the specific embodiments described in this application are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the various embodiments of this application without creative effort are within the scope of protection of this application.

[0031] To determine a highly accurate search starting point, thereby enabling computer devices to perform motion estimation more efficiently based on the search starting point, this application provides two motion estimation methods, each of which can be executed by a computer device. In one embodiment, the general principle of a motion estimation method proposed in this application is as follows: After the computer device determines the target coded block in the target video frame that currently requires motion estimation, it uses an advanced motion vector prediction method to predict the motion vector of the target coded block to obtain the predicted motion vector of the target coded block. In addition, the computer device will also determine the historical coded blocks that have been encoded in the video sequence to which the target video frame belongs, and select multiple reference coded blocks of the target coded block from the historical coded blocks. The historical coded blocks include at least: coded blocks that are not temporally adjacent to the target coded block, and coded blocks that are not spatially adjacent to the target coded block. Furthermore, the computer device uses the motion vector of each reference coding block as multiple candidate motion vectors for the target coding block, and selects the motion vector with the minimum rate-distortion cost from the multiple candidate motion vectors and the predicted motion vectors, so that the computer device can use the pixel position pointed to by the motion vector as the search starting point, and then use the search starting point to perform motion estimation on the target coding block.

[0032] Because the motion vectors referenced by the computer device when determining the search starting point include not only the predicted motion vectors measured using advanced motion vector prediction methods, but also the motion vectors of the coding blocks in the non-adjacent temporal domain and the non-adjacent spatial domain of the target coding block, the reference to motion information in both non-adjacent temporal and spatial domains greatly enriches the motion vectors available to the computer device. Determining the search starting point based on these abundant referenced motion vectors effectively avoids situations where the computer device can only determine the search starting point from adjacent spatial or temporal domains in cases of motion occlusion and scene transitions, thus preventing low accuracy of the search starting point. In other words, the embodiments of this application can improve the accuracy of the search starting point. Higher accuracy of the search starting point results in fewer search points for the computer device during motion estimation. Fewer search points mean less workload for the computer device in performing motion estimation, thereby improving the efficiency of motion estimation.

[0033] In this context, the target coding block refers to the coding block in the target video frame that currently requires motion estimation. Besides the target coding block, the target video frame may also include one or more coding blocks of the same size, with no restrictions on their position. The target video frame refers to the video frame in a video sequence that currently requires video encoding processing. For example, the target video frame and its included coding blocks can be as follows: Figure 1 As shown, Figure 1 The left image represents a video frame. In the right image, the gray area marked with 101 indicates a coding block, which consists of all the pixels covered by this gray area. Similarly, the gray area marked with 102 can also be a coding block. Coding block 101 and coding block 102 have the same size. In practical implementation, motion estimation techniques are used when encoding the target video frame. The basic idea of ​​motion estimation is to divide each frame of the video sequence into many non-overlapping macroblocks (i.e., coding blocks), assuming that the displacement of all pixels within a macroblock is the same. Then, within a given specific search range from each macroblock to a reference frame, the block most similar to the current block (i.e., the matching block) is found according to certain matching criteria. The relative displacement between the matching block and the current block is called the motion vector. Therefore, performing motion estimation on a coding block can also be understood as searching for the matching block of the coding block. When searching for a matching block in a target coded block, the computer device employs a search start point. This search start point is essentially a motion vector used to determine the pixel position from which the computer device begins its search for a matching block. Therefore, an accurate search start point can effectively reduce the time spent by the computer device searching for a matching block, thereby improving the efficiency of motion estimation.

[0034] In one embodiment, based on the inventive concept that utilizing motion information in non-adjacent temporal and spatial domains during motion estimation can improve the accuracy of the search starting point used in motion estimation, thereby improving the efficiency of motion estimation, this application embodiment also provides another motion estimation method. This method specifies that when performing motion estimation for a target coded block, an advanced motion vector prediction method is first used to predict the motion vectors of the target coded block, obtaining the predicted motion vectors of the target coded block. Furthermore, the computer device determines multiple candidate motion vectors, and the determination of the candidate motion vectors references the non-adjacent spatial and temporal domains of the target coded block, as well as the already encoded coded blocks of the target video frame where the target coded block is located. After determining the predicted motion vectors and multiple candidate motion vectors, the computer device uses the pixel position indicated by the motion vector with the lowest rate-distortion cost among the predicted motion vectors and all determined candidate motion vectors as the search starting point, to achieve motion estimation of the target coded block based on the search starting point. Specifically, the computer device determines candidate motion vectors in the following ways: (1) in the target video frame, determining the motion vector of at least one coded block that has been encoded as a candidate motion vector; (2) in a reference video frame of the target video frame, determining the pixel position that is not temporally adjacent to the target coded block, and determining the motion vector from at least one temporally non-adjacent pixel position as a candidate motion vector; (3) in the target video frame, determining the pixel position that is not spatially adjacent to the target coded block, and determining the motion vector from at least one spatially non-adjacent pixel position as a candidate motion vector.

[0035] The locations of pixels that are not temporally adjacent to the target coding block can be found in [reference needed]. Figure 2a To understand. Assume. Figure 2a The gray area A in the image represents the auxiliary coding block corresponding to the target coding block in the reference video frame. Figure 2a In this context, pixel positions marked with numbers 1-6 are those temporally adjacent to the target coding block, while pixel positions marked with numbers 7-31 are those non-temporally adjacent to the target coding block. Therefore, when the computer device determines a motion vector as a candidate motion vector from at least one temporally non-adjacent pixel position, the selected non-temporally non-adjacent pixel position can be any one or more of the multiple pixel positions marked with numbers 7-31, or it can be one or more pixel positions selected sequentially according to their distance from the target coding block from farthest to nearest or from nearest to farthest. This embodiment does not impose any limitations on this. Furthermore, pixel positions non-temporally adjacent to the target coding block can be found in [reference needed]. Figure 2b At this point, let's assume Figure 2b If gray area B in the diagram represents the target coded block, then... Figure 2bThe pixel positions marked by numbers 1-5 are the pixel positions adjacent to the spatial domain of the target coding block, while the pixel positions marked by numbers 6-18 are the pixel positions not adjacent to the spatial domain of the target coding block.

[0036] In the motion estimation method proposed in this application, the computer equipment used can be a terminal device, a server, or a complex device composed of both a terminal device and a server. This application does not impose specific limitations on this. The terminal device can be, for example, a smartphone, tablet, desktop computer, or intelligent robot. It can also run video playback applications to encode video files. Other applications can also run on the terminal device, such as image processing applications, map navigation applications, and music playback applications. The server can be, for example, an independent physical server, a server cluster consisting of multiple physical servers, a distributed system, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, and cloud storage. The server can establish a communication connection with the client, which can be a client with video playback or video transmission capabilities. The server then provides background services such as video encoding and motion estimation for the client.

[0037] Please see Figure 3 , Figure 3 This is a schematic flowchart illustrating a motion estimation method provided in an embodiment of this application. The motion estimation method can be executed by the aforementioned computer device, and as... Figure 3 As shown, the method includes at least steps S301-S305:

[0038] S301. Determine the target coding block in the target video frame that currently requires motion estimation.

[0039] In one embodiment, the computer device can trigger the determination of a target video frame and a target coded block upon receiving a corresponding computer instruction. Specifically, upon receiving a video encoding instruction or motion estimation instruction, the computer device can acquire one or more video sequences indicated by the instruction, and then select a video frame from the video sequence that has not yet completed encoding processing as the target video frame. The target video frame includes at least one coded block that has not yet completed encoding processing, and the computer device can select any coded block from at least one coded block as the target coded block. The video encoding instruction or motion estimation instruction can be generated by the computer device itself, or it can be sent to the computer device by other devices that have established a communication connection with the computer device; this embodiment does not impose any limitations on this.

[0040] S302. Use advanced motion vector prediction technology to predict the motion vectors of the target coding block to obtain the predicted motion vectors of the target coding block.

[0041] In a specific embodiment, the Advanced Motion Vector Prediction (AMVP) method refers to the new MV (Motion Vector) prediction technique proposed in H.265 (HEVC, High Efficiency Video Coding). The AMVP method primarily utilizes the correlation between spatial and temporal motion vectors to predict motion vectors for a target coding block, obtaining predicted motion vectors. The number of predicted motion vectors can be one or more. In one implementation, the predicted motion vectors can be MVPs (Motion Vector Prediction values) selected from the spatial and temporal candidate lists established by the computer device using the AMVP method for the target coding block. In another implementation, the predicted motion vectors can also include all motion vectors from both the spatial and temporal candidate lists established by the computer device using the AMVP method.

[0042] S303. Determine the historical coding blocks that have been encoded in the video sequence to which the target video frame belongs, and select multiple reference coding blocks of the target coding block from the historical coding blocks. The historical coding blocks include: coding blocks that are not temporally adjacent to the target coding block, and coding blocks that are not spatially adjacent to the target coding block.

[0043] In one embodiment, a historical coded block refers to a coded block that has completed encoding processing within a historical time period, or a coded block whose matching block has been determined. There are multiple historical coded blocks. Specifically, among these multiple historical coded blocks, at least one is not temporally adjacent to the target coded block, and at least one is not spatially adjacent to the target coded block. That is, when determining historical coded blocks, the computer device needs to determine not only the coded blocks not temporally adjacent to the target coded block, but also the coded blocks not spatially adjacent to the target coded block. Optionally, the coded blocks not temporally adjacent include: coded blocks whose position is not adjacent to the auxiliary coded block corresponding to the target coded block in a video frame temporally adjacent to the target video frame. "Not adjacent to the auxiliary coded block" means that the coded block and the auxiliary coded block do not have overlapping regions or boundary lines in the spatial domain (i.e., pixel domain). In other words, in a reference video frame, a coded block that is not spatially adjacent to the auxiliary coded block can be considered a coded block not temporally adjacent to the target coded block. An auxiliary coding block refers to a coding block in a reference video frame whose position is the same as that of the target coding block in the target video frame. Temporally adjacent to the target video frame means that the playback time of the target video frame is consecutive in the video sequence. For example, if the target video frame is the i-th video frame in the video sequence, where i is a positive integer greater than 1, then the (i-1)-th and (i+1)-th video frames can both be temporally adjacent to the target video frame. Optionally, non-temporally adjacent coding blocks may also include coding blocks in video frames that are not temporally adjacent to the target video frame. In the embodiments of this application, the video frame used to determine non-temporally adjacent coding blocks can be called the reference video frame of the target video frame. The following is combined with... Figure 4 The following example illustrates non-adjacent coding blocks in the time domain.

[0044] Figure 4 In the image, the two images marked with 40 represent the target video frame and the reference video frame. In this example, the reference video frame is the frame following the target video frame. To facilitate the reader's understanding of the auxiliary coded blocks in the reference frame, [details omitted]. Figure 4 The target video frame and reference video frame in the image are marked with auxiliary lines (such as...). Figure 4 (The dashed line in the middle) gives the following result: Figure 4The two images marked with 41 are shown in the diagram. In these two images, the gray area marked with 411, the area marked with 412, the area marked with 413, and the area marked with 414 can each be considered a coding block, hereinafter referred to as coding block 411, coding block 412, coding block 413, and coding block 414. Assuming the target coding block is coding block 411, since the position of coding block 413 in the reference video is the same as the position of coding block 411 in the target video frame, the corresponding auxiliary coding block in the reference video frame is coding block 413. Furthermore, since the positions of coding block 414 and coding block 413 in the reference video frame are not adjacent, coding block 414 can be considered a temporally non-adjacent coding block of the target coding block. Similarly, assuming the target coding block is coding block 412, its corresponding auxiliary coding block in the reference video frame is coding block 414, and a temporally non-adjacent coding block is coding block 413. It should be noted that in practical applications, the size of a coding block that is not temporally adjacent to the target coding block may be different from the size of the target coding block, and the computer device does not add auxiliary lines to the target video frame or the reference video frame during motion estimation. Figure 4 The auxiliary lines marked in the text are merely for the reader's convenience and should not be regarded as a limitation on the embodiments of this application.

[0045] S304. Obtain the motion vector of each reference coding block in multiple reference coding blocks, and use the obtained motion vectors of multiple reference coding blocks as multiple candidate motion vectors of the target coding block.

[0046] In one embodiment, when the historical encoded block is a fully encoded block, the computer device can use the motion vector used during encoding of the reference encoded block as a candidate motion vector when acquiring the motion vector of the reference encoded block. When the historical encoded block is a encoded block for which a matching block has been found, the computer device can use the search starting point used when searching for the matching block as a candidate motion vector of the target encoded block when acquiring the motion vector of the reference encoded block.

[0047] In another embodiment, to enrich the candidate motion vectors of the target coding block, the computer device may also obtain one or more motion vectors from a reference coding block as candidate motion vectors for the target coding block. Specifically, the computer device may select one or more candidate motion vectors from the candidate motion vectors used for motion estimation of the reference coding block as candidate motion vectors for the target coding block. The selection of these one or more candidate motion vectors may be random or sequentially selected in ascending order of rate-distortion cost; this embodiment does not impose any limitations on this. Furthermore, the computer device may also use the predicted motion vectors of the reference coding block predicted using an advanced motion vector prediction method as candidate motion vectors for the target coding block.

[0048] S305. From multiple candidate motion vectors and predicted motion vectors, select the pixel position pointed to by the motion vector with the minimum rate-distortion cost as the search starting point, and use the search starting point to perform motion estimation on the target coding block.

[0049] In one embodiment, the purpose of motion estimation of a target coded block by a computer device is to determine the motion vector between the target coded block and its matching block. The matching block is obtained by searching using a search starting point according to a preset search rule. The preset search rule can be used to specify the search range and search algorithm, etc. The search algorithm includes, but is not limited to, full search, TZSearch algorithm, etc. Taking the TZSearch algorithm as an example, in this embodiment, the steps of using this algorithm to perform motion estimation for the target coded block may include (1)-(5). Since this algorithm is a mature algorithm in the prior art, the principle of this algorithm will not be further described in this embodiment. Only its application in this embodiment will be described as follows:

[0050] (1) Determine the starting point of the search.

[0051] (2) Start searching the diamond template within the search range with a step size of 1. The step size increases in the form of an integer power of 2. Select the point (i.e., pixel position) with the minimum rate distortion cost as the search result of this step.

[0052] (3) If the step size corresponding to the optimal point obtained in step (2) is 1, then two-point search needs to be performed around the point. The main purpose is to supplement the search of points around the optimal point that have not yet been searched.

[0053] (4) If the step size corresponding to the optimal point selected in step (2) is greater than a certain threshold, then take the optimal point as the center and perform a full search within a certain range to determine the optimal point.

[0054] (5) Starting from the optimal point obtained in step (4), repeat steps (2)-(4) to refine the search until the optimal points obtained in two adjacent refinement searches are consistent. The motion vector obtained at this time is the motion vector of the final determined target coding block.

[0055] In this embodiment, when determining the search starting point, the computer device references not only the predicted motion vector measured using advanced motion vector prediction methods, but also motion vectors determined from the non-adjacent temporal domain and the non-adjacent spatial domain of the target coding block. This allows the computer device to utilize motion information from both non-adjacent temporal and spatial domains, greatly enriching the range of motion vectors available for reference. Determining the search starting point based on these abundant reference vectors effectively avoids situations where the computer device can only determine the search starting point from adjacent spatial or temporal domains, such as motion occlusion or scene transitions, thus preventing low accuracy of the search starting point. In other words, this embodiment improves the accuracy of the search starting point. Higher accuracy results in fewer search points for motion estimation, reducing the workload and efficiency of motion estimation.

[0056] Please see Figure 5 , Figure 5 This is a schematic flowchart illustrating another motion estimation method provided in an embodiment of this application. This motion estimation method can be executed by the aforementioned computer device, and as... Figure 5 As shown, the method includes at least steps S501-S506:

[0057] S501. Determine the target coding block in the target video frame that currently requires motion estimation.

[0058] S502. Use advanced motion vector prediction technology to predict the motion vectors of the target coding block and obtain the predicted motion vectors of the target coding block.

[0059] In one embodiment, the specific implementation of steps S501-S502 can be found in the relevant embodiments of steps S301 and S302, and will not be repeated here.

[0060] S503. Determine the historical coding blocks that have been encoded in the video sequence to which the target video frame belongs, and select multiple reference coding blocks of the target coding block from the historical coding blocks. The historical coding blocks include: coding blocks that are not temporally adjacent to the target coding block, and coding blocks that are not spatially adjacent to the target coding block.

[0061] In one embodiment, when determining historically encoded blocks that have been encoded in the video sequence to which the target video frame belongs, the computer device needs to determine encoded blocks that are not temporally adjacent to the target encoded block, and encoded blocks that are not spatially adjacent to the target encoded block. Specifically, to determine encoded blocks that are not temporally adjacent to the target encoded block, the computer device can first determine at least one video frame adjacent to the target video frame in the video sequence to which the target video frame belongs, and then select a reference video frame for the target video frame from among the at least one adjacent video frames. Specifically, the computer device can determine encoded video frames from among the at least one adjacent video frames, and use one or more of these encoded video frames as reference video frames to obtain one or more reference video frames. After obtaining the reference video frames, the computer device can determine the auxiliary encoded blocks corresponding to the target encoded block from at least one encoded block contained in each reference video frame, and then use the encoded blocks in the corresponding reference video frames that are not positionally adjacent to the target encoded block as encoded blocks that are not temporally adjacent to the target encoded block. In one feasible implementation, the reference video frames can also be one or more encoded video frames in the video sequence to which the target video frame belongs; this embodiment does not limit this. Furthermore, for determining the coding blocks that are not spatially adjacent to the target coding block, the computer device can first determine one or more candidate coding blocks that have been encoded in the target video frame, and then take the candidate coding blocks that are not spatially adjacent to the target coding block as the coding blocks that are not spatially adjacent to the target coding block. The following continues in conjunction with... Figure 2b The following example illustrates a coding block that is not adjacent to the spatial domain of the target coding block. Figure 2b In the diagram, gray area B represents the target coding block, and the squares marked by numbers 1-18 are the completed coding blocks (i.e., candidate coding blocks) included in the target video frame. Among them, the squares marked by numbers 1-5 are coding blocks that are spatially adjacent to the target coding block, and correspondingly, the squares marked by numbers 6-18 are coding blocks that are not spatially adjacent to the target coding block.

[0062] In one embodiment, the computer device may select multiple reference coding blocks from historical coding blocks to select a target coding block in one or more of the following (1)-(3):

[0063] (1) The computer device randomly selects at least one coding block as a reference coding block from coding blocks that are not adjacent to the target coding block in the time domain and coding blocks that are not adjacent to the target coding block in the spatial domain, and obtains multiple reference coding blocks.

[0064] (2) The computer device numbers each historical coding block according to a preset numbering rule, and obtains at least one historical coding block with a preset number as a reference coding block from coding blocks that are not temporally adjacent to the target coding block and coding blocks that are not spatially adjacent to the target coding block, so as to obtain multiple reference coding blocks. The preset numbering rule can be: the historical coding blocks are numbered in ascending order of their distance from the target coding block. For example, assuming historical coding block A is the coding block closest to the target coding block among all historical coding blocks, then historical coding block A is numbered 1.

[0065] (3) From the coding blocks that are not temporally adjacent to the target coding block, determine the coding blocks whose distance to the auxiliary coding block is greater than or equal to a preset distance, and select at least one coding block from the determined coding blocks as a reference coding block. In addition, from the coding blocks that are not spatially adjacent to the target coding block, determine the coding blocks whose distance to the target coding block is greater than or equal to a preset distance, and select at least one coding block from the determined coding blocks as a reference coding block, and then use all the selected reference coding blocks as multiple reference coding blocks selected by the computer device.

[0066] In one embodiment, in addition to selecting reference coding blocks from coding blocks that are not temporally adjacent to the target coding block and coding blocks that are not spatially adjacent to the target coding block, the computer device may also select at least one reference coding block from coding blocks that have been encoded in the target video frame. For example, the computer device may select reference coding blocks from coding blocks that have been encoded in the target video frame in a random selection manner.

[0067] S504. Obtain the motion vector of each reference coding block in multiple reference coding blocks, and use the obtained motion vectors of multiple reference coding blocks as multiple candidate motion vectors of the target coding block.

[0068] In one embodiment, when the computer device selects the above three methods to determine multiple candidate motion vectors, before obtaining the motion vector of each reference coding block, the computer device can first perform deduplication processing on all the obtained reference coding blocks to obtain the motion vector of each reference coding block from the deduplicated multiple reference coding blocks, as multiple candidate motion vectors of the target coding block. Deduplication processing effectively avoids the computer device performing related processing based on duplicate motion vectors, thereby reducing the workload of the computer device and effectively saving computer resources such as storage resources and computing resources.

[0069] S505. Calculate the rate-distortion cost of each candidate motion vector using the rate-distortion cost function, and calculate the rate-distortion cost of the predicted motion vector.

[0070] S506. Take the pixel position pointed to by the motion vector corresponding to the calculated minimum rate distortion cost as the search starting point.

[0071] In one embodiment, the computer device may employ a rate-distortion cost function to calculate the rate-distortion cost of each candidate motion vector and the rate-distortion cost of the predicted motion vector, thereby using the pixel position indicated by the motion vector corresponding to the calculated minimum rate-distortion cost as the search starting point. The rate-distortion cost function can be represented by Equation 1 below, where Cost represents the rate-distortion cost (or Lagrange cost), D represents the sum of absolute errors between the target coding block and the corresponding reference coding block, R represents the number of bits required to encode the index of the advanced motion vector prediction list, R1 represents the number of bits required to encode the difference between the motion vectors in the candidate list and the motion vector of the MVP, and λ represents the Lagrange factor in motion estimation.

[0072] Cost = D + λ * (R + R1) (Equation 1)

[0073] In this embodiment, when determining the search starting point, the computer device references not only the predicted motion vector measured using advanced motion vector prediction methods, but also motion vectors determined from the non-adjacent temporal domain and the non-adjacent spatial domain of the target coding block. This allows the computer device to utilize motion information from both non-adjacent temporal and spatial domains, greatly enriching the range of motion vectors available for reference. Determining the search starting point based on these abundant reference vectors effectively avoids situations where the computer device can only determine the search starting point from adjacent spatial or temporal domains, such as motion occlusion or scene transitions, thus preventing low accuracy of the search starting point. In other words, this embodiment improves the accuracy of the search starting point. Higher accuracy results in fewer search points for motion estimation, reducing the workload and efficiency of motion estimation.

[0074] Based on the aforementioned embodiments of the motion estimation method, this application also discloses a motion estimation device, which can be a computer program (including program code) running on the aforementioned computer device. This motion estimation device can perform actions such as... Figure 3 as well as Figure 5 For the motion estimation method shown, please refer to [link / reference]. Figure 6 The motion estimation device may include at least: a determination unit 601, a prediction unit 602, an acquisition unit 603, and a processing unit 604.

[0075] The determining unit 601 is used to determine the target coding block in the target video frame that currently needs to perform motion estimation;

[0076] The prediction unit 602 is used to perform motion vector prediction on the target coding block using an advanced motion vector prediction method to obtain the predicted motion vector of the target coding block;

[0077] The determining unit 601 is further configured to determine the historical coding blocks that have been encoded in the video sequence to which the target video frame belongs, and select a plurality of reference coding blocks of the target coding block from the historical coding blocks. The historical coding blocks include: coding blocks that are not temporally adjacent to the target coding block, and coding blocks that are not spatially adjacent to the target coding block.

[0078] The acquisition unit 603 is used to acquire the motion vector of each of the plurality of reference coding blocks, and to use the acquired motion vectors of the plurality of reference coding blocks as the plurality of candidate motion vectors of the target coding block;

[0079] The processing unit 604 is configured to select the pixel position pointed to by the motion vector with the minimum rate-distortion cost from the plurality of candidate motion vectors and the predicted motion vectors as the search starting point, and use the search starting point to perform motion estimation on the target coding block.

[0080] In one implementation, the determining unit 601 may specifically be used to perform:

[0081] Determine at least one video frame in the video sequence to which the target video frame belongs that is adjacent to the target video frame, and select a reference video frame for the target video frame from the at least one adjacent video frame;

[0082] An auxiliary coding block corresponding to the target coding block is determined from at least one coding block contained in the reference video frame, wherein the position of the target coding block in the reference video frame is the same as the position of the target coding block in the target video frame;

[0083] The coding blocks in the reference video frame that are not adjacent to the target coding block in position are taken as the coding blocks that are not adjacent to the target coding block in the time domain.

[0084] In another implementation, the determining unit 601 can also be specifically used to perform:

[0085] Determine the video frame that has been encoded in at least one adjacent video frame;

[0086] One or more video frames from the already encoded video frames are used as the reference video frames.

[0087] In another implementation, the determining unit 601 can also be specifically used to perform:

[0088] One or more candidate coded blocks that have been encoded are identified in the target video frame;

[0089] Among the one or more candidate coding blocks, the candidate coding block that is not adjacent to the target coding block in position is taken as the coding block that is not adjacent to the target coding block in spatial domain.

[0090] In another implementation, the determining unit 601 can also be specifically used to perform:

[0091] At least one coding block is randomly selected from the coding blocks that are not temporally adjacent to the target coding block and the coding blocks that are not spatially adjacent to the target coding block as the reference coding block, thereby obtaining the plurality of reference coding blocks.

[0092] In another implementation, the determining unit 601 can also be specifically used to perform:

[0093] From the coding blocks that are not temporally adjacent to the target coding block, determine the coding blocks whose distance from the auxiliary coding block is greater than or equal to a preset distance, and select at least one coding block from the determined coding blocks as a reference coding block;

[0094] From the coding blocks that are not adjacent to the target coding block in the spatial domain, determine the coding blocks whose distance from the target coding block is greater than or equal to a preset distance, and select at least one coding block from the determined coding blocks as a reference coding block.

[0095] In another implementation, the processing unit 604 may specifically be used to perform:

[0096] The rate-distortion cost of each candidate motion vector is calculated using a rate-distortion cost function, and the rate-distortion cost of the predicted motion vector is also calculated.

[0097] The pixel position pointed to by the motion vector corresponding to the calculated minimum rate distortion cost is used as the search starting point.

[0098] In another implementation, the determining unit 601 can also be specifically used to perform:

[0099] Identify the encoded blocks that have been fully encoded in the target video frame;

[0100] Randomly select at least one coded block from the coded blocks that have been encoded in the target video frame as a reference coded block;

[0101] The plurality of reference coding blocks and at least one randomly selected reference coding block are deduplicated to obtain a plurality of deduplicated reference coding blocks;

[0102] When acquiring the motion vector of each reference coding block in the plurality of reference coding blocks, the acquisition unit 603 may specifically perform the following:

[0103] Obtain the motion vector of each reference coding block in the multiple reference coding blocks after deduplication.

[0104] According to another embodiment of this application, Figure 6 The units in the motion estimation device shown are based on logical functional division. These units can be individually or entirely combined into one or more other units, or some of these units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. In other embodiments of this application, the motion estimation device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and multiple units can collaborate to achieve them.

[0105] According to another embodiment of this application, the following can be achieved by running on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), a device capable of performing operations such as... Figure 3 and Figure 5 The computer program (including program code) involved in each step of the method shown is used to construct, for example... Figure 6 The motion estimation apparatus shown herein, and the motion estimation method for implementing the embodiments of this application, are described. The computer program may be recorded on, for example, a computer storage medium, loaded onto the aforementioned computer device via the computer storage medium, and run therein.

[0106] In this embodiment, when determining the search starting point, the motion estimation device references not only the predicted motion vector measured using advanced motion vector prediction methods, but also motion vectors determined from the non-adjacent temporal domain and the non-adjacent spatial domain of the target coding block. This allows the motion estimation device to utilize motion information from both non-adjacent temporal and spatial domains, greatly enriching the range of motion vectors available for reference. Determining the search starting point based on these abundant reference motion vectors effectively avoids situations where the motion estimation device can only determine the search starting point from adjacent spatial or temporal domains in cases of motion occlusion or scene switching, thus preventing low accuracy of the search starting point. In other words, this embodiment improves the accuracy of the search starting point. Higher accuracy of the search starting point results in fewer search points for the motion estimation device, reducing its workload and improving its efficiency.

[0107] Based on the descriptions of the above method and device embodiments, this application also provides a computer device. Please refer to [link to relevant documentation]. Figure 7 The computer device includes at least a processor 701 and a computer storage medium 702, and the processor 701 and the computer storage medium 702 of the computer device can be connected by a bus or other means.

[0108] The aforementioned computer storage medium 702 is a memory device in a computer device used to store programs and data. It is understood that the computer storage medium 702 can include both the built-in storage medium of the computer device and extended storage media supported by the computer device. The computer storage medium 702 provides storage space for storing the operating system of the computer device. Furthermore, this storage space also stores one or more computer programs suitable for loading and execution by the processor 701. These computer programs can be one or more program codes. It should be noted that the computer storage medium can be high-speed RAM or non-volatile memory, such as at least one disk storage device; optionally, it can also be at least one storage medium located remotely from the aforementioned processor. The processor 701 (or CPU (Central Processing Unit)) is the computing and control core of the computer device, suitable for implementing one or more computer programs, specifically suitable for loading and executing one or more computer programs to achieve corresponding method flows or corresponding functions.

[0109] In one embodiment, processor 701 may load and execute one or more computer programs stored in computer storage medium 702 to achieve the aforementioned related functions. Figure 3 The corresponding method steps in the illustrated method embodiment; in specific implementation, one or more computer programs in the computer storage medium 702 are loaded by the processor 701 and executed as follows:

[0110] Determine the target coding block in the target video frame from which motion estimation needs to be performed;

[0111] An advanced motion vector prediction method is used to predict the motion vectors of the target coding block to obtain the predicted motion vectors of the target coding block.

[0112] Determine the historical coding blocks that have been encoded in the video sequence to which the target video frame belongs, and select multiple reference coding blocks of the target coding block from the historical coding blocks. The historical coding blocks include: coding blocks that are not temporally adjacent to the target coding block, and coding blocks that are not spatially adjacent to the target coding block.

[0113] The motion vector of each of the plurality of reference coding blocks is obtained, and the obtained motion vectors of the plurality of reference coding blocks are used as the plurality of candidate motion vectors of the target coding block;

[0114] From the plurality of candidate motion vectors and the predicted motion vectors, the pixel position pointed to by the motion vector with the minimum rate-distortion cost is selected as the search starting point, and the motion estimation of the target coding block is performed using the search starting point.

[0115] In one implementation, the processor 701 may specifically be used to load and execute:

[0116] Determine at least one video frame in the video sequence to which the target video frame belongs that is adjacent to the target video frame, and select a reference video frame for the target video frame from the at least one adjacent video frame;

[0117] An auxiliary coding block corresponding to the target coding block is determined from at least one coding block contained in the reference video frame, wherein the position of the target coding block in the reference video frame is the same as the position of the target coding block in the target video frame;

[0118] The coding blocks in the reference video frame that are not adjacent to the target coding block in position are taken as the coding blocks that are not adjacent to the target coding block in the time domain.

[0119] In another implementation, the processor 701 can also be specifically used to load and execute:

[0120] Determine the video frame that has been encoded in at least one adjacent video frame;

[0121] One or more video frames from the already encoded video frames are used as the reference video frames.

[0122] In another implementation, the processor 701 can also be specifically used to load and execute:

[0123] One or more candidate coded blocks that have been encoded are identified in the target video frame;

[0124] Among the one or more candidate coding blocks, the candidate coding block that is not adjacent to the target coding block in position is taken as the coding block that is not adjacent to the target coding block in spatial domain.

[0125] In another implementation, the processor 701 can also be specifically used to load and execute:

[0126] At least one coding block is randomly selected from the coding blocks that are not temporally adjacent to the target coding block and the coding blocks that are not spatially adjacent to the target coding block as the reference coding block, thereby obtaining the plurality of reference coding blocks.

[0127] In another implementation, the processor 701 can also be specifically used to load and execute:

[0128] From the coding blocks that are not temporally adjacent to the target coding block, determine the coding blocks whose distance from the auxiliary coding block is greater than or equal to a preset distance, and select at least one coding block from the determined coding blocks as a reference coding block;

[0129] From the coding blocks that are not adjacent to the target coding block in the spatial domain, determine the coding blocks whose distance from the target coding block is greater than or equal to a preset distance, and select at least one coding block from the determined coding blocks as a reference coding block.

[0130] In another implementation, the processor 701 can also be specifically used to load and execute:

[0131] The rate-distortion cost of each candidate motion vector is calculated using a rate-distortion cost function, and the rate-distortion cost of the predicted motion vector is also calculated.

[0132] The pixel position pointed to by the motion vector corresponding to the calculated minimum rate distortion cost is used as the search starting point.

[0133] In another implementation, the processor 701 can also be specifically used to load and execute:

[0134] Identify the encoded blocks that have been fully encoded in the target video frame;

[0135] Randomly select at least one coded block from the coded blocks that have been encoded in the target video frame as a reference coded block;

[0136] The plurality of reference coding blocks and at least one randomly selected reference coding block are deduplicated to obtain a plurality of deduplicated reference coding blocks;

[0137] When the processor 701 acquires the motion vector of each of the plurality of reference coding blocks, it can specifically load and execute:

[0138] Obtain the motion vector of each reference coding block in the multiple reference coding blocks after deduplication.

[0139] In this embodiment, when determining the search starting point, the computer device references not only the predicted motion vector measured using advanced motion vector prediction methods, but also motion vectors determined from the non-adjacent temporal domain and the non-adjacent spatial domain of the target coding block. This allows the computer device to utilize motion information from both non-adjacent temporal and spatial domains, greatly enriching the range of motion vectors available for reference. Determining the search starting point based on these abundant reference vectors effectively avoids situations where the computer device can only determine the search starting point from adjacent spatial or temporal domains, such as motion occlusion or scene transitions, thus preventing low accuracy of the search starting point. In other words, this embodiment improves the accuracy of the search starting point. Higher accuracy results in fewer search points for motion estimation, reducing the workload and efficiency of motion estimation.

[0140] This application also provides a computer storage medium storing one or more computer programs corresponding to the above-described motion estimation method. When one or more processors load and execute the one or more computer programs, the motion estimation method described in the embodiments can be implemented, and will not be repeated here. The beneficial effects of using the same method will also not be repeated here. It is understood that the computer program can be deployed on one or more devices capable of communicating with each other.

[0141] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer storage medium, and when executed, it can include the processes of the embodiments of the motion estimation method described above. The computer storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0142] The above-disclosed embodiments are merely partial embodiments of this application and should not be construed as limiting the scope of this application. Those skilled in the art will understand that implementing all or part of the above embodiments and making equivalent changes in accordance with the claims of this application still fall within the scope of this invention.

Claims

1. A motion estimation method, characterized in that, include: Determine the target coding block in the target video frame from which motion estimation needs to be performed; An advanced motion vector prediction method is used to predict the motion vectors of the target coding block to obtain the predicted motion vectors of the target coding block. Determine the historical coding blocks that have been encoded in the video sequence to which the target video frame belongs, and select multiple reference coding blocks of the target coding block from the historical coding blocks. The historical coding blocks include: coding blocks that are not temporally adjacent to the target coding block, and coding blocks that are not spatially adjacent to the target coding block. The motion vector of each of the plurality of reference coding blocks is obtained, and the obtained motion vectors of the plurality of reference coding blocks are used as the plurality of candidate motion vectors of the target coding block; From the plurality of candidate motion vectors and the predicted motion vectors, the pixel position pointed to by the motion vector with the minimum rate-distortion cost is selected as the search starting point, and the target coding block is motion estimated using the search starting point; The methods for determining the coding block that is not temporally adjacent to the target coding block include: Determine at least one video frame in the video sequence to which the target video frame belongs that is adjacent to the target video frame, and select a reference video frame for the target video frame from the at least one adjacent video frame; An auxiliary coding block corresponding to the target coding block is determined from at least one coding block contained in the reference video frame, wherein the position of the auxiliary coding block in the reference video frame is the same as the position of the target coding block in the target video frame; The coding block in the reference video frame that is not adjacent to the auxiliary coding block is taken as the coding block that is not temporally adjacent to the target coding block.

2. The method according to claim 1, characterized in that, The step of selecting a reference video frame for the target video frame from at least one adjacent video frame includes: Determine the video frame that has been encoded in at least one adjacent video frame; One or more video frames from the already encoded video frames are used as the reference video frames.

3. The method according to claim 1, characterized in that, The method for determining the coding block that is not spatially adjacent to the target coding block: One or more candidate coded blocks that have been encoded are identified in the target video frame; Among the one or more candidate coding blocks, the candidate coding block that is not adjacent to the target coding block in position is taken as the coding block that is not adjacent to the target coding block in spatial domain.

4. The method according to any one of claims 1-3, characterized in that, The step of selecting multiple reference coding blocks from the historical coding blocks to select the target coding block includes: At least one coding block is randomly selected from the coding blocks that are not temporally adjacent to the target coding block and the coding blocks that are not spatially adjacent to the target coding block as the reference coding block, thereby obtaining the plurality of reference coding blocks.

5. The method according to claim 1 or 2, characterized in that, The step of selecting multiple reference coding blocks from the historical coding blocks to select the target coding block includes: From the coding blocks that are not temporally adjacent to the target coding block, determine the coding blocks whose distance from the auxiliary coding block is greater than or equal to a preset distance, and select at least one coding block from the determined coding blocks as a reference coding block; From the coding blocks that are not adjacent to the target coding block in the spatial domain, determine the coding blocks whose distance from the target coding block is greater than or equal to a preset distance, and select at least one coding block from the determined coding blocks as a reference coding block.

6. The method according to claim 1, characterized in that, The step of selecting the pixel position pointed to by the target motion vector as the search starting point from the at least one candidate motion vector and the predicted motion vector includes: The rate-distortion cost of each candidate motion vector is calculated using a rate-distortion cost function, and the rate-distortion cost of the predicted motion vector is also calculated. The pixel position pointed to by the motion vector corresponding to the calculated minimum rate distortion cost is used as the search starting point.

7. The method according to claim 1, characterized in that, The method further includes: Identify the encoded blocks that have been fully encoded in the target video frame; Randomly select at least one coded block from the coded blocks that have been encoded in the target video frame as a reference coded block; The plurality of reference coding blocks and at least one randomly selected reference coding block are deduplicated to obtain a plurality of deduplicated reference coding blocks; The step of obtaining the motion vector of each of the plurality of reference coding blocks includes: Obtain the motion vector of each reference coding block in the multiple reference coding blocks after deduplication.

8. A motion estimation device, characterized in that, include: The determination unit is used to determine the target coding block in the target video frame that currently needs to undergo motion estimation. The prediction unit is used to perform motion vector prediction on the target coding block using an advanced motion vector prediction method to obtain the predicted motion vector of the target coding block; The determining unit is further configured to determine the historical coding blocks that have been encoded in the video sequence to which the target video frame belongs, and select a plurality of reference coding blocks of the target coding block from the historical coding blocks. The historical coding blocks include: coding blocks that are not temporally adjacent to the target coding block, and coding blocks that are not spatially adjacent to the target coding block. The acquisition unit is used to acquire the motion vector of each of the plurality of reference coding blocks, and to use the acquired motion vectors of the plurality of reference coding blocks as the plurality of candidate motion vectors of the target coding block; The processing unit is configured to select the pixel position pointed to by the motion vector with the minimum rate-distortion cost from the plurality of candidate motion vectors and the predicted motion vectors as the search starting point, and use the search starting point to perform motion estimation on the target coding block; The methods for determining the coding block that is not temporally adjacent to the target coding block include: Determine at least one video frame in the video sequence to which the target video frame belongs that is adjacent to the target video frame, and select a reference video frame for the target video frame from the at least one adjacent video frame; An auxiliary coding block corresponding to the target coding block is determined from at least one coding block contained in the reference video frame, wherein the position of the auxiliary coding block in the reference video frame is the same as the position of the target coding block in the target video frame; The coding block in the reference video frame that is not adjacent to the auxiliary coding block is taken as the coding block that is not temporally adjacent to the target coding block.

9. A computer device, characterized in that, include: A processor, the processor being adapted to implement one or more computer programs; A computer storage medium storing one or more computer programs adapted to be loaded by the processor and executed as described in claims 1-7.

10. A computer storage medium, characterized in that, The computer storage medium stores one or more computer programs, which are adapted to be loaded by a processor and executed as described in claims 1-7.

Images