Matching point search method, device, storage medium and electronic device

Abstract

Embodiments of the present invention provide a matching point search method, apparatus, storage medium, and electronic device, wherein the method includes: determining a first pixel point included in a reference frame image that matches a target pixel point included in a target frame image; when a first loss value between a motion vector corresponding to the first pixel point and a motion vector of the target pixel point is greater than a predetermined threshold, correcting the first pixel point to obtain a second pixel point, wherein the degree of matching between the second pixel point and the target pixel point is higher than the degree of matching between the first pixel point and the target pixel point; and searching for an optimal matching point included in the reference frame image with the highest degree of matching with the target pixel point, with the second pixel point as the center. The present invention solves the problem of poor matching point search results in the related art, thereby improving the accuracy of the matching points found.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and in particular, to a matching point search method, device, storage medium, and electronic device. Background Art

[0002] Video coding technology has been widely used in fields such as digital television, the internet, and wireless multimedia. Motion estimation is a core technology in video compression and coding, and the primary method for removing temporal redundancy. Related technologies use a single search template, making it difficult to estimate motion and identify matching search points for image sequences with varying texture features and motion intensity.

[0003] It can be seen from this that the related art has the problem of poor effect in searching for matching points.

[0004] Currently, no effective solution has been proposed to the above-mentioned problems existing in the related technologies. Summary of the Invention

[0005] Embodiments of the present invention provide a matching point search method, device, storage medium, and electronic device to at least solve the problem of poor matching point search results in the related art.

[0006] According to one embodiment of the present invention, a matching point search method is provided, including: determining a first pixel point included in a reference frame image that matches a target pixel point included in a target frame image; when a first loss value between a motion vector corresponding to the first pixel point and the motion vector of the target pixel point is greater than a predetermined threshold, correcting the first pixel point to obtain a second pixel point, wherein the matching degree between the second pixel point and the target pixel point is higher than the matching degree between the first pixel point and the target pixel point; and searching for the best matching point included in the reference frame image that has the highest matching degree with the target pixel point with the second pixel point as the center.

[0007] According to another embodiment of the present invention, a matching point search device is provided, including: a determination module for determining a first pixel point included in a reference frame image that matches a target pixel point included in a target frame image; a correction module for correcting the first pixel point to obtain a second pixel point when a first loss value between a motion vector corresponding to the first pixel point and the motion vector of the target pixel point is greater than a predetermined threshold, wherein the matching degree between the second pixel point and the target pixel point is higher than the matching degree between the first pixel point and the target pixel point; and a search module for searching for the best matching point included in the reference frame image with the highest matching degree with the target pixel point, with the second pixel point as the center.

[0008] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0009] According to another embodiment of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any one of the above method embodiments.

[0010] Through the present invention, a first pixel point is determined to match a target pixel point included in a reference frame image with a target pixel point included in a target frame image. When a first loss value between a motion vector corresponding to the first pixel point and a motion vector of the target pixel point is greater than a predetermined threshold, the first pixel point is corrected to obtain a second pixel point, and a search is performed with the second pixel point as the center for the best matching point included in the reference frame image that has the highest degree of matching with the target pixel point. Because the first pixel point can be corrected before determining the best matching point, and the search is performed with the corrected pixel point as the center, the problem of poor matching point search results in the related art can be solved, thereby achieving the effect of improving the accuracy of the searched matching points. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a hardware structure block diagram of a mobile terminal for a matching point search method according to an embodiment of the present invention;

[0012] Figure 2 is a flowchart of a matching point search method according to an embodiment of the present invention;

[0013] Figure 3 is a schematic diagram of median prediction according to an exemplary embodiment of the present invention. Figure 1 ;

[0014] Figure 4 is a schematic diagram of median prediction according to an exemplary embodiment of the present invention. Figure 2 ;

[0015] Figure 5 is a schematic diagram of prediction of a corresponding block of a previous frame according to an exemplary embodiment of the present invention;

[0016] Figure 6 is a schematic diagram of adjacent reference frames according to an exemplary embodiment of the present invention;

[0017] Figure 7 is a flow chart of a matching point search method according to a specific embodiment of the present invention;

[0018] Figure 8 is a schematic diagram of a matching point search method according to a specific embodiment of the present invention;

[0019] Figure 9 4 is a structural block diagram of a matching point search device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with embodiments.

[0021] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0022] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG is a hardware structure block diagram of a mobile terminal for a matching point search method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown) a processor 102 (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA and other processing devices) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0023] Memory 104 can be used to store computer programs, such as software programs and modules of application software, such as the computer program corresponding to the matching point search method in the embodiments of the present invention. Processor 102 executes the computer programs stored in memory 104 to execute various functional applications and data processing, thereby implementing the aforementioned methods. Memory 104 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 104 may further include memory remotely located relative to processor 102, and such remote memory may be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0024] The transmission device 106 is used to receive or send data via a network. A specific example of the aforementioned network may include a wireless network provided by the mobile terminal's communications provider. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0025] In this embodiment, a matching point search method is provided. Figure 2 FIG. 1 is a flow chart of a method for searching matching points according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0026] Step S202, determining a first pixel point included in the reference frame image that matches a target pixel point included in the target frame image;

[0027] Step S204: if a first loss value between the motion vector corresponding to the first pixel and the motion vector of the target pixel is greater than a predetermined threshold, correct the first pixel to obtain a second pixel, wherein a matching degree between the second pixel and the target pixel is higher than a matching degree between the first pixel and the target pixel;

[0028] Step S206 : searching for the best matching point in the reference frame image with the second pixel point as the center, which has the highest matching degree with the target pixel point.

[0029] In the above embodiment, the reference frame image can be an image captured before or after the target frame image. The target pixel point can be a sub-block included in a macroblock of the target frame image. When the search begins, the first pixel point can be the predicted initial search point. In subsequent searches, the first pixel point can be the current best matching point determined by the previous search. In video coding, a coded image is typically divided into several macroblocks. A macroblock can include a luminance pixel block and two additional chrominance pixel blocks. The luminance block is a 16x16 pixel block, while the size of the two chrominance pixel blocks can be determined based on the image sampling format. For example, for a YUV420 sampled image, the chrominance block is an 8x8 pixel block. In each image, several macroblocks are arranged into slices. The video coding algorithm encodes each macroblock individually, organizing it into a continuous video stream. The macroblock can be a chrominance pixel block. Macroblocks can be further divided into various types of sub-blocks to obtain the target pixel point.

[0030] In the above embodiment, when predicting the initial search point, the loss value of the motion vector of the median prediction, the upper block prediction, the previous frame corresponding block prediction, the adjacent reference frame prediction and the current block motion vector can be calculated, and the loss value of the origin vector and the current block motion vector can be added, and the point corresponding to the motion vector with the smallest loss value is determined as the current best matching point.

[0031] In the above embodiment, median prediction is to predict the motion vector of the current macroblock by calculating the average value of the motion vectors of adjacent macroblocks in the panoramic video frame. Figure 1 Please refer to the attached Figure 3 ,like Figure 3 As shown, the median of the motion vectors of the left, upper and upper right blocks of the current block can be used as the predicted motion vector of the current block. and are the motion vectors of the left, top, and top right edge blocks respectively, then the median predicted motion vector of the current block can be expressed as Median forecast Figure 2 Please refer to the attached Figure 4 ,like Figure 4 As shown, the current macroblock E, the left macroblock A, the upper macroblock B, the upper right macroblock C, and the upper left macroblock D. If the number of macroblocks to the left of E is greater than 1, then the top block is selected as A, and the leftmost block above E is selected as B. Calculation is performed. Median prediction requires judging the position of the block to determine the specific motion vector parameters required for prediction. If the current macroblock is at the left edge of the image and the motion vector of macroblock A is missing, (0,0) is used instead. If the current macroblock is at the right edge of the image and the motion vector of macroblock C is missing, D is used instead. If the current macroblock is at the top edge of the image and the motion vectors of macroblocks B, C, and D are missing, the motion vector of macroblock A can be used directly instead; if the current macroblock is at the bottom edge of the image, the motion vectors of macroblocks A, B, and C can be used directly for median prediction.

[0032] The prediction process of the current macroblock E may follow the following criteria:

[0033] 1) Except for macroblocks with a size of 16*8 and 8*16, the motion vector MV of other macroblocks or sub-macroblocks is the median of the adjacent A, B, and C.

[0034] 2) If the current macroblock size is 16*8, the MVs of the upper and lower 16*8 sub-macroblocks are predicted by macroblock B and macroblock A respectively.

[0035] 3) If the current macroblock size is 8*16, the MVs of the 8*16 sub-macroblocks on the left and right sides are predicted by macroblock A and macroblock C respectively.

[0036] 4) For the SKIP macroblock, the 16*16 MV can be predicted according to 1).

[0037] In the above embodiment, the upper layer block prediction formula can be found in Where, is the motion vector of the upper block of the current block. Macroblocks and sub-macroblocks 16*16, 16*8, 8*16, 8*8, 8*4, 4*8, and 4*4 can be respectively recorded as modes 1 to 7. During the motion estimation process, in order to achieve the best coding effect, various combinations of macroblocks and sub-macroblocks can be traversed. You can search 4 to 7 first, then search modes 1 to 3, or search modes 1 to 3 first, then search modes 4 to 7. Mode 5 or Mode 6 is the upper block of Mode 7. Mode 4 is the upper block of Mode 5 and Mode 6, Mode 2 or Mode 3 is the upper block of Mode 4; Mode 1 is the upper block of Mode 2 and Mode 3. The motion vector of the upper block can be used as the predicted motion vector of the motion vector of the lower block.

[0038] In the above embodiment, the prediction formula for the previous frame corresponding block can be found in in, is the motion vector of the block with the same coordinates as the current block in the previous frame. This prediction mode is based on the principle of object motion continuity. In the previous panoramic video frame, the motion vector of the macroblock at the same position as the current panoramic video frame is used as the predicted motion vector of the macroblock in the current panoramic video frame. Figure 5 .

[0039] In the above embodiment, objects in the panoramic video sequence have strong motion continuity and correlation. Therefore, the motion vector of the macroblock in the current panoramic video frame and the motion vector of the macroblock at the corresponding position in different reference frames also have a certain correlation. Figure 6 ,like Figure 6 As shown, let the time of the frame where the current block is located be t. When searching for the best matching block of the current block in the reference frame t', the motion vector of the current block in the reference frame t'+1 can be used to predict the motion vector of the current block in the reference frame t'. The motion vector prediction formula of the adjacent reference frame prediction can be expressed as Where t represents the acquisition time of the current frame image, t' and t'-1 are the acquisition times of the reference frame images, is the motion vector in the reference frame frame t' with the same coordinates as the current block. t-t' represents the representational distance between frame t and frame t', t-(t'-1) represents the representational distance between frame t and frame t'-1, and t-t' / t-(t'-1) is like the ratio of similar triangles.

[0040] In the above embodiment, the origin prediction may be that all continuous video sequences have a motion characteristic of tending toward the origin, and therefore the motion vector of the point (0,0) may be used as a candidate prediction vector.

[0041] In the above embodiment, the first pixel point may be the current best matching point determined in each search. After determining the first pixel point, the first loss value between the motion vector corresponding to the first pixel point and the motion vector of the target pixel point may be determined. If the first loss value is less than or equal to a predetermined threshold, the first pixel point may be determined as the best matching point. If the first loss value is greater than the predetermined threshold, the first pixel point may be corrected to obtain a second pixel point, and the search range may be determined with the second pixel point as the center, and the best matching point may be determined in the reference frame image. The first loss value may be the absolute error and SAD. That is, the SAD value between the motion vector corresponding to the first pixel point and the motion vector of the target pixel point. The predetermined threshold may be a predetermined threshold.

[0042] Optionally, the execution entity of the above steps can be a background processor, or other devices with similar processing capabilities, or a machine that at least integrates a data processing device, where the data processing device can include terminals such as computers and mobile phones, but is not limited to this.

[0043] Through the present invention, a first pixel point is determined to match a target pixel point included in a reference frame image with a target pixel point included in a target frame image. When a first loss value between a motion vector corresponding to the first pixel point and a motion vector of the target pixel point is greater than a predetermined threshold, the first pixel point is corrected to obtain a second pixel point, and a search is performed with the second pixel point as the center for the best matching point included in the reference frame image that has the highest degree of matching with the target pixel point. Because the first pixel point can be corrected before determining the best matching point, and the search is performed with the corrected pixel point as the center, the problem of poor matching point search results in the related art can be solved, thereby achieving the effect of improving the accuracy of the searched matching points.

[0044] In an exemplary embodiment, correcting the first pixel point to obtain the second pixel point includes: determining the difference between the first loss value and a predetermined calibration loss value, wherein the calibration loss value is the loss value corresponding to a calibration point, and the calibration point is the most recently determined matching point with the highest degree of match with the target pixel point; correcting the first pixel point based on the difference to obtain the second pixel point. In this embodiment, when correcting the first pixel point, the difference between the first loss value and the predetermined calibration loss value can be determined, and the first similarity can be corrected according to the gradient relationship corresponding to the difference. The calibration loss value can be the most recently determined matching point with the highest degree of match with the target similarity. Initially, the calibration point can be the initial search point, and in the subsequent search process, the calibration point can be the most recently determined current best matching point.

[0045] In an exemplary embodiment, correcting the first pixel point based on the difference to obtain the second pixel point includes: when the difference is less than or equal to a first threshold, determining a third pixel point on the perpendicular bisector of the line connecting the first pixel point and the calibration point; searching with each point included in the third pixel point as the center to obtain multiple first pixel point sets; when the first pixel point is included in the target pixel point set, determining a fourth pixel point corresponding to the target pixel point set, wherein the target pixel point set is a pixel point set included in multiple first pixel point sets, and the target pixel point set is a matching point set obtained by searching with the fourth pixel point as the center; and determining the fourth pixel point as the second pixel point. In this embodiment, the first threshold value can be a predetermined value. When the difference is less than or equal to the first threshold value, that is, when the gradient of the difference between the SAD value of the first pixel point and the target pixel point and the SAD value of the calibration point and the target pixel point is extremely small, the point on the perpendicular bisector obtained by connecting the first pixel point and the calibration point can be taken as the new current best matching point (i.e., the second pixel point). At the same time, the next search method must include the old current best matching point. If it cannot be included, the new best matching point is discarded. That is, when the first pixel point is not a pixel point included in the first pixel point set, the first pixel point is determined to be the second pixel point. Among them, the first threshold value can be 0.1, 0.2, etc. Of course, the first threshold value can also be set to other values, such as 0.05, 0.3, etc., and the present invention is not limited to this.

[0046] In an exemplary embodiment, correcting the first pixel point based on the difference to obtain the second pixel point includes: determining a target distance between the first pixel point and the calibration point when the difference is greater than a first threshold and less than or equal to a second threshold, wherein the first threshold is less than the second threshold; determining a fifth pixel point included in an extension line of a line connecting the first pixel point and the calibration point, whose distance from the calibration point is the target distance; determining a second set of pixel points obtained by searching with the fifth pixel point as the center; and determining the fifth pixel point as the second pixel point when the second set of pixel points includes the first pixel point. In this embodiment, when the difference is greater than the first threshold and less than or equal to the second threshold, that is, the gradient decrease of the difference between the SAD value of the current best matching point (i.e., the first pixel point) and the calibration SAD value is small, a point equidistant from the extension line between the current best matching point and the calibration SAD value matching point can be selected as the new current best matching point, and the next search method must include the old current best matching point. If it cannot be included, the new best matching point is discarded. The second threshold value may be 0.5, 0.6, etc. Of course, the second threshold value may also be set to other values, such as 0.4, 0.7, 0.9, etc., which is not limited in the present invention.

[0047] In an exemplary embodiment, correcting the first pixel point based on the difference to obtain the second pixel point includes: when the difference value is greater than a second threshold, determining that the reference frame image includes a sixth pixel point that is closest to the first pixel point; determining a third pixel point set obtained by searching with the sixth pixel point as the center; and when the third pixel point set includes the first pixel point, determining the sixth pixel point as the second pixel point. In this embodiment, if the SAD value of the current best matching point has a large gradient decrease from the calibrated SAD value, a point close to the current best matching point can be taken as the new current best matching point, and the next search method must include the old current best matching point. If it cannot be included, the new best matching point is discarded.

[0048] In an exemplary embodiment, after obtaining the second pixel point, the method further includes: updating the calibration point to the second pixel point. In this embodiment, each time the first pixel point is corrected to obtain the second pixel point, the calibration point can be updated to the second pixel point.

[0049] In one exemplary embodiment, searching for the best matching point in the reference frame image with the second pixel as the center that has the highest degree of match with the target pixel includes: determining a fourth set of pixels obtained by searching a first pattern centered on the second pixel according to a first search method; determining a second loss value between the motion vector of each pixel in the fourth set of pixels and the motion vector of the target pixel; determining a seventh pixel corresponding to the minimum loss value included in the second loss values; and, if the seventh pixel is the second pixel, determining the best matching point based on the seventh pixel. In this embodiment, after determining the second pixel, the best matching point in the reference frame image can be searched with the second pixel as the center. The first image can be determined with the second pixel as the center and searched according to the first search method. The first pattern can be a hexagon, an octagon, or the like. That is, the fourth set of pixels can be obtained by searching a hexagon, an octagon, or the like with the second pixel as the center. The second loss value between the motion vector of each pixel in the fourth set of pixels and the motion vector of the target pixel is determined. Determine the seventh pixel point corresponding to the minimum loss value included in the second loss value, and when the seventh pixel point is the midpoint of the first figure, that is, when the seventh pixel point is the second pixel point, determine the best matching point based on the seventh pixel point.

[0050] In the above embodiment, the best matching point is corrected based on the marked SAD value. After obtaining the second pixel point, the calibration loss value, such as the SAD value, can be updated. An extended hexagon search is performed with the second pixel point as the current center, skipping the searched points. If the search result is the hexagon center, the extended hexagon search is terminated, and the current best matching point, i.e., the seventh pixel point, is determined. The best matching point is determined based on the seventh pixel point.

[0051] In an exemplary embodiment, after determining the seventh pixel point corresponding to the minimum loss value included in the second loss value, the method further includes: if the seventh pixel point is not the second pixel point, determining the side length of the figure to be searched each time; searching according to the first search method based on the side length with the second pixel point as the center to obtain a fifth pixel point set; determining a third loss value between the motion vector corresponding to each pixel point included in the fifth pixel point set and the motion vector of the target pixel point; determining the eighth pixel point corresponding to the minimum loss value included in the third loss value; and if the minimum loss value included in the third loss value is less than the predetermined threshold, searching according to the fourth search method with the eighth pixel point as the center to determine the best matching point. In this embodiment, when the seventh pixel point is not the second pixel point, the first figure can be expanded to search according to the first search method to obtain the fifth pixel point set. Determine a third loss value between the motion vector corresponding to each pixel included in the fifth pixel set and the target motion vector, and locate the eighth pixel corresponding to the minimum loss value included in the third loss value. If the minimum loss value included in the third loss value is less than a predetermined threshold, perform a search using a fourth search method with the eighth pixel as the center to determine the best matching point. The fourth search method may be a rectangular search, such as a 5*5 matrix.

[0052] In the above embodiment, when the minimum loss value included in the third loss value is greater than the predetermined threshold, the first graph can be further expanded and searched in the first search mode until a pixel point with a minimum loss value less than the predetermined threshold is found.

[0053] In the above embodiment, a hexagonal search may be performed first. If the currently found best matching point is not the center of the hexagon, the side length of the hexagon may be expanded, and the hexagonal search may be performed again. The loss value between the motion vector of each searched pixel and the motion vector of the target pixel is determined, and the minimum loss value is determined. If the minimum loss value is less than a predetermined threshold, a search may be performed using a fourth search method with the pixel corresponding to the minimum loss value as the center. If the minimum loss value is greater than the predetermined threshold, the search side length is expanded again, and a hexagonal search may be performed.

[0054] In one exemplary embodiment, determining the best matching point based on the seventh pixel includes: correcting the seventh pixel to obtain a ninth pixel; performing a search using a second search method with the ninth pixel as the center point to obtain a sixth pixel set; determining a fourth loss value between the motion vector corresponding to each pixel in the sixth pixel set and the motion vector of the target pixel; and, if the minimum loss value included in the fourth loss value is greater than the predetermined threshold, performing multiple searches using the first search method on a second image centered around the tenth pixel corresponding to the minimum loss value included in the fourth loss value, until the second pixel determined using the first search method is the first center of the search range corresponding to the first search method; and determining the best matching point based on the first center. In this embodiment, if the seventh pixel found using the first search method is the center of the first image, the seventh pixel can be corrected to obtain a ninth pixel. A second search method is then performed with the ninth pixel as the center. The second search method can be an asymmetric cross search. Determining a current best matching point, wherein the current best matching point is the tenth pixel corresponding to the minimum loss value included in the fourth loss value. When the minimum loss value is greater than a predetermined threshold, the search may be performed multiple times in the first search manner with the tenth pixel point as the center, until the currently found best matching point is the first center of the search range.

[0055] In the above embodiment, after obtaining the seventh pixel point, the seventh pixel point can be determined as the current best matching point. The current best matching point can be further corrected to obtain the ninth pixel point. An asymmetric cross search is performed with the ninth pixel point as the current center, skipping the previously searched pixels to determine the current best matching point. An early termination strategy is then used to determine whether the error of the current best matching point is within an acceptable range. If so, that is, the minimum loss value included in the fourth loss value is greater than a predetermined threshold, a hexagonal search is performed repeatedly with the current best matching point as the current center, skipping the previously searched points, until the center of the hexagon is the best matching point. The second graphic is an image with the same shape as the first graphic but different side lengths.

[0056] In an exemplary embodiment, after determining the fourth loss value between the motion vector corresponding to each pixel point included in the sixth pixel point set and the motion vector of the target pixel point, the method further includes: when the minimum loss value included in the fourth loss value is less than or equal to the predetermined threshold, searching according to the fourth search method with the tenth pixel point as the center to determine the best matching point. In this embodiment, when the minimum loss value included in the fourth loss value is less than or equal to the predetermined threshold, a 5×5 rectangular full search can be performed with the current best matching point as the current search center to determine the final best matching point and end the search.

[0057] In one exemplary embodiment, determining the best matching point based on the first center includes: correcting the first center to obtain a second center; if a fifth loss value between the motion vector corresponding to the second center and the motion vector of the target pixel exceeds a predetermined threshold, performing multiple searches according to a third search method until the second pixel determined according to the third search method is the third center of a search range corresponding to the third search method; and determining the third center as the best matching point. In this embodiment, after obtaining the first center, the first center may be corrected to obtain a second center; if a fifth loss value between the motion vector corresponding to the second center and the motion vector of the target pixel exceeds a predetermined threshold, performing a search according to a third search method, which may be a diamond search method, until the second pixel determined according to the third search method is the third center of a search range corresponding to the third search method. The search range corresponding to the third search method is a diamond shape. The third center point is determined as the best matching point. In each search according to the third search method, the search center point is the current best matching point determined previously.

[0058] In an exemplary embodiment, the method further includes: when a fifth loss value between the motion vector corresponding to the second center and the motion vector of the target pixel is less than or equal to the predetermined threshold, searching according to a fourth search method with the second center as the center to determine the best matching point. In this embodiment, when the fifth loss value is less than or equal to the predetermined threshold, a search can be performed according to a fourth search method with the second center as the center, and the point with the smallest searched SAD value is determined as the best matching point. The fourth search method can be a rectangular search, such as a 5*5 matrix search.

[0059] In an exemplary embodiment, the method further includes: when the first loss value corresponding to the first pixel point is less than or equal to the predetermined threshold, searching according to the fourth search method with the first pixel point as the center to obtain a seventh pixel point set; determining a sixth loss value between the motion vector of each pixel point included in the seventh pixel point set and the motion vector of the target pixel point; determining the eleventh pixel point corresponding to the minimum loss value included in the sixth loss value; and determining the eleventh pixel point as the best matching point. In this embodiment, when the first loss value is less than or equal to the predetermined threshold, the search can be directly performed according to the fourth search method with the first pixel point as the center, and the pixel point with the smallest SAD can be determined as the best matching point.

[0060] In an exemplary embodiment, searched pixels in the reference image that have been searched are marked, and the searched pixels are skipped during the search process. In this embodiment, searched pixels that have been searched can be marked during each search process, and the searched pixels can be skipped during the search process.

[0061] The following describes the matching point search method in conjunction with a specific implementation method:

[0062] Figure 7 FIG. 1 is a flow chart of a matching point search method according to a specific embodiment of the present invention. Figure 7 As shown, the starting search point is predicted. The SAD values ​​of the motion vectors for the current block and the median prediction, the prediction for the upper block, the prediction for the corresponding block in the previous frame, and the prediction for the adjacent reference frame are calculated. The SAD value of the origin vector and the current block's motion vector are added to determine the optimal motion vector, which serves as the current best matching point. The SAD value is then marked. An early termination strategy is then used to determine whether the error of the current best matching point is within an acceptable range. If so, a full search of a 5×5 rectangle with the current best matching point as the search center is performed to determine the final best matching point and terminate the search. If not, the search proceeds to the next step.

[0063] Macro search. First, the best matching point is corrected based on the marked SAD value and the SAD value is updated. An extended hexagon search is performed with the current best matching point as the current center, skipping the searched points. If the search result is the hexagon center, the extended hexagon search is terminated to determine the current best matching point. Next, the best matching point is further corrected, and an asymmetric cross search is performed with the current best matching point as the current center, skipping the searched points, to determine the current best matching point. An early termination strategy is used to determine whether the error of the current best matching point is within an acceptable range. If so, a full 5×5 rectangle search is performed with the current best matching point as the current search center to determine the final best matching point and end the search. If not, proceed to the next step, local search.

[0064] Local search. The best matching point is modified based on the marked SAD value and the SAD value is updated. A hexagonal search is repeated with the current best matching point as the current center, skipping previously searched points until the center of the hexagon is the best matching point. An early termination strategy is used to determine whether the error of the current best matching point is within an acceptable range. If so, a full 5×5 rectangle search is performed with the current best matching point as the current search center to determine the final best matching point and terminate the search. If not, proceed to the next step.

[0065] Detailed search. Modify the best matching point based on the marked SAD value and update the SAD value. Use the current best matching point as the current center to repeatedly search using a small diamond, skipping the searched points until the center of the small diamond is the best matching point.

[0066] End the search. The current best matching point is the final matching point.

[0067] The early termination strategy involves terminating the search early if the loss value of the current best matching point is less than the optimal threshold SADbest (the optimal threshold is obtained through the full search algorithm), indicating that the predicted motion vector is very close to the current motion vector. This significantly reduces the number of search points and improves coding efficiency.

[0068] The best matching point strategy is modified by marking the SAD value: if the SAD value of the current best matching point and the calibration SAD value have a very small gradient decrease, the point on the perpendicular bisector between the current best matching point and the matching point with the calibration SAD value can be taken as the new current best matching point, and the next search method must include the old current best matching point. If it cannot be included, the new best matching point is discarded; if the SAD value of the current best matching point and the calibration SAD value have a small gradient decrease, the point equidistant from the extension line between the current best matching point and the matching point with the calibration SAD value can be taken as the new current best matching point, and the next search method must include the old current best matching point. If it cannot be included, the new best matching point is discarded; if the SAD value of the current best matching point and the calibration SAD value have a large gradient decrease, the point close to the current best matching point can be taken as the new current best matching point, and the next search method must include the old current best matching point. If it cannot be included, the new best matching point is discarded.

[0069] Figure 8 FIG. 1 is a schematic diagram of a matching point search method according to a specific embodiment of the present invention. Figure 8 As shown, the first step is to determine the current best matching point by predicting the starting search point, such as Figure 8 The second step is to use the expanded hexagon search. If the search result is not the hexagon center point, further expand the hexagon to search. If the second expansion meets the early termination strategy, the current best search point ( Figure 8 Perform a full search of the 5×5 rectangle (solid hexagon with solid edges in the upper left corner) to determine the final best matching point ( Figure 8 The search ends early if the search result is the center point of the hexagon ( Figure 8 Center point), perform asymmetric cross search and determine the current best matching point ( Figure 8 The third step is to use the hexagon to search repeatedly until the center point of the hexagon is the best matching point ( Figure 8 The fourth step is to use a small diamond to search repeatedly until the center point of the small diamond is the best matching point ( Figure 8 The solid diamond in the middle is also the final best matching point, ending the search.

[0070] In the aforementioned embodiments, the search is divided into macroscopic, local, and detailed searches without compromising search accuracy, thereby improving search efficiency and comprehensiveness. Based on the center deviation characteristics of motion vector differences, a new strategy for correcting the best matching point by marking the SAD value has been added. This corrects the matching point and allows for a more comprehensive search of the valid area near the matching point.

[0071] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0072] This embodiment also provides a matching point search device for implementing the above-mentioned embodiments and preferred implementations. Details already described will not be repeated here. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0073] Figure 9 is a structural block diagram of a matching point search device according to an embodiment of the present invention. Figure 9 As shown, the device includes:

[0074] A determination module 92 is configured to determine a first pixel point included in the reference frame image that matches a target pixel point included in the target frame image;

[0075] a correction module 94 configured to correct the first pixel to obtain a second pixel when a first loss value between a motion vector corresponding to the first pixel and a motion vector corresponding to the target pixel is greater than a predetermined threshold, wherein a matching degree between the second pixel and the target pixel is higher than a matching degree between the first pixel and the target pixel;

[0076] The search module 96 is configured to search for a best matching point in the reference frame image with the second pixel point as the center, which has the highest matching degree with the target pixel point.

[0077] In an exemplary embodiment, the correction module 94 can correct the first pixel point to obtain the second pixel point in the following manner: determine the difference between the first loss value and a predetermined calibration loss value, wherein the calibration loss value is the loss value corresponding to the calibration point, and the calibration point is the most recently determined matching point with the highest matching degree with the target pixel point; correct the first pixel point based on the difference to obtain the second pixel point.

[0078] In an exemplary embodiment, the correction module 94 can correct the first pixel point based on the difference to obtain the second pixel point in the following manner: when the difference is less than or equal to the first threshold, determine the third pixel point on the perpendicular bisector of the line connecting the first pixel point and the calibration point; search with each point included in the third pixel point as the center to obtain multiple first pixel point sets; when the target pixel point set includes the first pixel point, determine the fourth pixel point corresponding to the target pixel point set, wherein the target pixel point set is the pixel point set included in the multiple first pixel point sets, and the target pixel point set is the matching point set obtained by searching with the fourth pixel point as the center; determine the fourth pixel point as the second pixel point.

[0079] In an exemplary embodiment, the correction module 94 can correct the first pixel point based on the difference to obtain the second pixel point in the following manner: when the difference is greater than a first threshold and less than or equal to a second threshold, determine the target distance between the first pixel point and the calibration point, wherein the first threshold is less than the second threshold; determine the fifth pixel point included in the extension line of the line connecting the first pixel point and the calibration point, whose distance from the calibration point is the target distance; determine the second pixel point set obtained by searching with the fifth pixel point as the center; and when the second pixel point set includes the first pixel point, determine the fifth pixel point as the second pixel point.

[0080] In an exemplary embodiment, the correction module 94 can correct the first pixel point based on the difference to obtain the second pixel point in the following manner: when the difference value is greater than a second threshold value, determine that the reference frame image includes a sixth pixel point that is closest to the first pixel point; determine a third pixel point set obtained by searching with the sixth pixel point as the center; when the third pixel point set includes the first pixel point, determine the sixth pixel point as the second pixel point.

[0081] In an exemplary embodiment, the apparatus is further configured to, after obtaining the second pixel point, update the calibration point to the second pixel point.

[0082] In an exemplary embodiment, the search module 96 can search for the best matching point with the highest matching degree with the target pixel point included in the reference frame image with the second pixel point as the center in the following manner: determine a fourth pixel point set obtained by searching according to the first search method on the first figure with the second pixel point as the center; determine a second loss value between the motion vector of each pixel point included in the fourth pixel point set and the motion vector of the target pixel point; determine the seventh pixel point corresponding to the minimum loss value included in the second loss value; and when the seventh pixel point is the second pixel point, determine the best matching point based on the seventh pixel point.

[0083] In an exemplary embodiment, the search module 96 can be used to, after determining the seventh pixel point corresponding to the minimum loss value included in the second loss value: when the seventh pixel point is not the second pixel point, determine the side length of the figure searched each time; with the second pixel point as the center, search according to the first search method based on the side length to obtain a fifth pixel point set; determine the third loss value between the motion vector corresponding to each pixel point included in the fifth pixel point set and the motion vector of the target pixel point; determine the eighth pixel point corresponding to the minimum loss value included in the third loss value; when the minimum loss value included in the third loss value is less than the predetermined threshold, search according to the fourth search method with the eighth pixel point as the center to determine the best matching point.

[0084] In an exemplary embodiment, the search module 96 can determine the best matching point based on the seventh pixel point in the following manner: correct the seventh pixel point to obtain a ninth pixel point; search according to the second search method with the ninth pixel point as the midpoint to obtain a sixth pixel point set; determine the fourth loss value between the motion vector corresponding to each pixel point included in the sixth pixel point set and the motion vector of the target pixel point; when the minimum loss value included in the fourth loss value is greater than the predetermined threshold, perform multiple searches according to the first search method on a second figure centered on the tenth pixel point corresponding to the minimum loss value included in the fourth loss value, until the second pixel point determined based on the first search method is the first center of the search range corresponding to the first search method; determine the best matching point based on the first center.

[0085] In an exemplary embodiment, the search module 96 can be used to determine the fourth loss value between the motion vector corresponding to each pixel point included in the sixth pixel point set and the motion vector of the target pixel point, and when the minimum loss value included in the fourth loss value is less than or equal to the predetermined threshold, search according to the fourth search method with the tenth pixel point as the center to determine the best matching point.

[0086] In an exemplary embodiment, the search module 96 can determine the best matching point based on the first center in the following manner: correct the first center to obtain the second center; when the fifth loss value between the motion vector corresponding to the second center and the motion vector of the target pixel point is greater than the predetermined threshold, perform multiple searches according to the third search method until the second pixel point determined based on the third search method is the third center of the search range corresponding to the third search method; and determine the third center as the best matching point.

[0087] In an exemplary embodiment, the search module 96 can be used to search according to the fourth search method with the second center as the center to determine the best matching point when the fifth loss value between the motion vector corresponding to the second center and the motion vector of the target pixel point is less than or equal to the predetermined threshold.

[0088] In an exemplary embodiment, the device can be used to search according to the fourth search method with the first pixel point as the center to obtain a seventh pixel point set when the first loss value corresponding to the first pixel point is less than or equal to the predetermined threshold; determine the sixth loss value between the motion vector of each pixel point included in the seventh pixel point set and the motion vector of the target pixel point; determine the eleventh pixel point corresponding to the minimum loss value included in the sixth loss value; and determine the eleventh pixel point as the best matching point.

[0089] In an exemplary embodiment, the apparatus may be configured to mark searched pixels in the reference image; and skip the searched pixels during the search process.

[0090] It should be noted that the above modules can be implemented through software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0091] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0092] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0093] An embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0094] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0095] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail here.

[0096] Obviously, those skilled in the art will appreciate that the various modules or steps of the present invention described above can be implemented using a general-purpose computing device, can be centralized on a single computing device, or can be distributed across a network of multiple computing devices. They can be implemented using program code executable by the computing device, and thus, can be stored in a storage device and executed by the computing device. In some cases, the steps shown or described herein can be performed in a different order than that shown, or can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0097] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A matching point search method, characterized in that: include: Determining a first pixel point included in the reference frame image that matches a target pixel point included in the target frame image; When a first loss value between a motion vector corresponding to the first pixel and a motion vector corresponding to the target pixel is greater than a predetermined threshold, correcting the first pixel to obtain a second pixel, wherein a matching degree between the second pixel and the target pixel is higher than a matching degree between the first pixel and the target pixel; Searching for a best matching point in the reference frame image with the second pixel as the center, which has the highest matching degree with the target pixel; Correcting the first pixel point to obtain the second pixel point includes: determining the difference between the first loss value and a predetermined calibration loss value, wherein the calibration loss value is the loss value corresponding to the calibration point, and the calibration point is the most recently determined matching point with the highest matching degree with the target pixel point; correcting the first pixel point based on the difference to obtain the second pixel point.

2. The method according to claim 1, characterized in that Correcting the first pixel point based on the difference to obtain the second pixel point includes: When the difference is less than or equal to a first threshold, determining a third pixel point on a perpendicular bisector of a line connecting the first pixel point and the calibration point; Searching with each point included in the third pixel point as a center to obtain multiple first pixel point sets; In a case where the first pixel point is included in a target pixel point set, determining a fourth pixel point corresponding to the target pixel point set, wherein the target pixel point set is a pixel point set included in multiple first pixel point sets, and the target pixel point set is a matching point set obtained by searching with the fourth pixel point as the center; The fourth pixel point is determined as the second pixel point.

3. The method according to claim 1, characterized in that Correcting the first pixel point based on the difference to obtain the second pixel point includes: When the difference is greater than a first threshold and less than or equal to a second threshold, determining a target distance between the first pixel point and the calibration point, wherein the first threshold is less than the second threshold; Determine a fifth pixel point included in an extension line of a line connecting the first pixel point and the calibration point, the fifth pixel point being at the target distance from the calibration point; Determine a second pixel point set obtained by searching with the fifth pixel point as the center; In a case where the second pixel point set includes the first pixel point, the fifth pixel point is determined as the second pixel point.

4. The method according to claim 1, wherein Correcting the first pixel point based on the difference to obtain the second pixel point includes: When the difference is greater than a second threshold, determining that the reference frame image includes a sixth pixel point that is closest to the first pixel point; Determine a third pixel point set obtained by searching with the sixth pixel point as the center; In a case where the third pixel point set includes the first pixel point, the sixth pixel point is determined as the second pixel point.

5. The method according to any one of claims 1 to 4, characterized in that After obtaining the second pixel point, the method further includes: updating the calibration point to the second pixel point.

6. The method according to claim 1, characterized in that Searching the reference frame image for a best matching point with the highest matching degree with the target pixel point with the second pixel point as the center includes: determining a fourth pixel point set obtained by searching the first graphic centered at the second pixel point according to the first search method; Determining a second loss value between a motion vector of each pixel point included in the fourth pixel point set and a motion vector of the target pixel point; Determine a seventh pixel point corresponding to a minimum loss value included in the second loss values; When the seventh pixel is the second pixel, the best matching point is determined based on the seventh pixel.

7. The method according to claim 6, characterized in that After determining the seventh pixel corresponding to the minimum loss value included in the second loss value, the method further includes: When the seventh pixel point is not the second pixel point, determining the side length of the graph searched each time; Taking the second pixel point as the center, searching based on the side length according to the first search method to obtain a fifth pixel point set; Determining a third loss value between a motion vector corresponding to each pixel point included in the fifth pixel point set and a motion vector of the target pixel point; Determine an eighth pixel point corresponding to a minimum loss value included in the third loss values; When the minimum loss value included in the third loss value is less than the predetermined threshold, a search is performed with the eighth pixel point as the center according to the fourth search method to determine the best matching point.

8. The method according to claim 6, characterized in that Determining the best matching point based on the seventh pixel point includes: Correcting the seventh pixel to obtain a ninth pixel; Taking the ninth pixel point as the midpoint, searching according to the second search method to obtain a sixth pixel point set; Determine a fourth loss value between a motion vector corresponding to each pixel point included in the sixth pixel point set and a motion vector of the target pixel point; When the minimum loss value included in the fourth loss values ​​is greater than the predetermined threshold, performing multiple searches according to the first search method on the second graph centered at the tenth pixel corresponding to the minimum loss value included in the fourth loss values ​​until the second pixel determined based on the first search method is the first center of the search range corresponding to the first search method; The best matching point is determined based on the first center.

9. The method according to claim 8, characterized in that After determining a fourth loss value between the motion vector corresponding to each pixel point included in the sixth pixel point set and the motion vector of the target pixel point, the method further includes: When the minimum loss value included in the fourth loss value is less than or equal to the predetermined threshold, a search is performed with the tenth pixel point as the center according to the fourth search method to determine the best matching point.

10. The method according to claim 8, characterized in that Determining the best matching point based on the first center includes: Correcting the first center to obtain a second center; When a fifth loss value between the motion vector corresponding to the second center and the motion vector of the target pixel is greater than the predetermined threshold, performing multiple searches according to the third search method until the second pixel determined based on the third search method is the third center of the search range corresponding to the third search method; The third center is determined as the best matching point.

11. The method according to claim 10, characterized in that The method further comprises: When the fifth loss value between the motion vector corresponding to the second center and the motion vector of the target pixel point is less than or equal to the predetermined threshold, a search is performed with the second center as the center according to the fourth search method to determine the best matching point.

12. The method according to claim 1, characterized in that The method further comprises: When the first loss value corresponding to the first pixel point is less than or equal to the predetermined threshold, searching according to the fourth search method with the first pixel point as the center to obtain a seventh pixel point set; determining a sixth loss value between a motion vector of each pixel point included in the seventh pixel point set and a motion vector of the target pixel point; Determining an eleventh pixel point corresponding to a minimum loss value included in the sixth loss values; The eleventh pixel point is determined as the best matching point.

13. The method according to any one of claims 6 to 10, characterized in that The method further comprises: Marking searched pixels in the reference frame image; The searched pixel points are skipped during the search process.

14. A matching point search device, characterized in that: include: a determination module, configured to determine a first pixel point included in the reference frame image that matches a target pixel point included in the target frame image; a correction module, configured to correct the first pixel point to obtain a second pixel point when a first loss value between a motion vector corresponding to the first pixel point and a motion vector corresponding to the target pixel point is greater than a predetermined threshold, wherein a matching degree between the second pixel point and the target pixel point is higher than a matching degree between the first pixel point and the target pixel point; A search module is configured to search for a best matching point in the reference frame image with the second pixel as the center, the best matching point having the highest matching degree with the target pixel; The correction module corrects the first pixel point to obtain the second pixel point in the following manner: determining the difference between the first loss value and a predetermined calibration loss value, wherein the calibration loss value is the loss value corresponding to a calibration point, and the calibration point is the most recently determined matching point with the highest matching degree with the target pixel point; and correcting the first pixel point based on the difference to obtain the second pixel point.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 13 when executed by a processor.

16. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 13.

Images