Template matching method and system based on gradient direction and distance depth fusion

Abstract

This invention discloses a template matching method and system based on gradient direction and distance-depth fusion. In the search and matching stage, the method first performs pixel-to-pixel correspondence matching between the search image and the template image using the edge map and index map of the template image to establish pixel-to-pixel correspondence under deformation matching. Then, it calculates distance weights using the distance map of the template image and performs weighted calculation of gradient direction similarity. When deformation causes changes in the position of edge pixels, the distance weights are automatically adjusted, making the algorithm tend to favor pixels that are closer to the edge and have a more stable structure. It can automatically ignore unreliable matching points caused by deformation, occlusion, or noise. Even if the edge position is offset, the depth fusion of gradient direction and distance can effectively avoid local mismatches and ensure that even if the target undergoes elastic deformation, it can still maintain stability based on the global shape information of the template image, achieving a balance between high deformation robustness and high positioning accuracy.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a template matching method and system based on gradient direction and distance-depth fusion, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Template matching is one of the most fundamental and widely used techniques in image processing. Its core task is to locate the region in a source image (also known as a search image) that is most similar to a given template image. Existing template matching techniques can be mainly divided into gray-level correlation-based matching methods and feature-based matching methods. Among them, gray-level-based methods (such as the Normalized Cross-Correlation (NCC) algorithm) are sensitive to changes in illumination, have a high computational cost, and are difficult to handle nonlinear illumination changes and occlusion. Therefore, in industrial vision, feature-based matching methods have become the mainstream due to their stronger robustness to changes in illumination.

[0003] Currently, commonly used feature-based matching methods include edge gradient direction-based matching methods and distance transformation-based shape matching methods. Among them, edge gradient direction-based matching methods first extract edge points and their gradient directions in the template image and the search image through edge detection operators (such as the Sobel operator and the Canny operator). The matching metric (i.e., gradient direction score) is usually the dot product of gradient directions or cosine similarity. However, this matching metric is extremely sensitive to small shifts in edge position. When the target object has even a few pixels of elastic deformation, slight rotation, or manufacturing tolerance, the dot product value or cosine similarity of the gradient vector will drop sharply, leading to matching failure or a significant reduction in positioning accuracy, thus resulting in poor deformation robustness of this method. Therefore, to address the deformation robustness issue, a shape matching method based on distance transform is proposed. This method performs a distance transform on the search image, generating a distance map. Each pixel value in the distance map represents the distance from that point to the nearest edge point in the search image. During matching, edge points from the template image are superimposed on the distance map of the search image, and the average or sum of the corresponding distance values ​​(i.e., the distance transform score) is calculated. However, because the distance transform process loses key details such as edge direction and intensity, it lacks the ability to distinguish details, and edge noise may lead to mismatches, resulting in poor matching and localization accuracy. Although in recent years, some studies have proposed weighted fusion of gradient direction scores and distance transform scores to take into account the advantages of both, this direct parallel fusion method suffers from problems such as difficulty in weight selection and dilution of matching score discrimination. Furthermore, the dynamic weight adjustment mechanism has complex parameters, making it difficult to achieve a balance between deformation robustness and high localization accuracy. Summary of the Invention

[0004] This invention provides a template matching method and system based on gradient direction and distance-depth fusion, as well as an electronic device and a computer-readable storage medium, which can achieve a balance between high deformation robustness and high positioning accuracy.

[0005] According to one aspect of the present invention, a template matching method based on gradient direction and distance-depth fusion is provided, comprising the following: Input a template image, perform multi-scale feature extraction on the template image to generate a multi-feature template; wherein, the multi-feature template includes a distance map, gradient direction map, edge map and index map of the template image; Input a search image and generate a gradient direction map of the search image; The optimal search pose is determined by matching the gradient direction map of the search image with the template image. In the search matching process, for each candidate search pose, the pixel correspondence between the search image and the template image is first matched based on the edge map and index map of the template image. Then, the distance weight is calculated based on the distance map of the template image, and the gradient direction maps of the template image and the search image are combined to perform distance weighted calculation to obtain the gradient direction similarity between each pair of matched pixels. The matching score of each candidate search pose is calculated based on the gradient direction similarity of all matched pixel pairs. The candidate search pose with the highest matching score is taken as the optimal search pose. The candidate region in the search image that is most similar to the template image is determined based on the optimal search pose.

[0006] Furthermore, the matching score for each candidate search pose is calculated based on the following formula: ; in, Indicates the matching score. Indicates distance weight, Represents the distance of the template image to the edge pixels in the image. q The corresponding distance values ​​show that pixels farther from the edge have smaller distance weights. Edge pixels in the gradient direction map of the template image q gradient direction, Represents the pixels in the gradient direction map of the search image. gradient direction, edge pixels q and pixels For a pair of matched pixels, N This indicates the number of matched pixel pairs.

[0007] Furthermore, the distance weights are calculated based on the following formula: ; in, Represents distance value The corresponding distance weights, It represents the standard deviation.

[0008] Furthermore, after determining the optimal search pose, the following is also included: The optimal search pose is finely optimized to achieve fine-tuning of the optimal search pose.

[0009] Furthermore, the process of fine-tuning the optimal search pose includes the following: Deformation field optimization initialization is performed based on the optimal search pose; A deformation field model is constructed, and a multi-objective optimization function is constructed to solve the deformation field model to obtain the optimal deformation field model; Nonlinear optimization is performed based on the optimal deformation field model to obtain the fine-tuned optimal search pose.

[0010] Furthermore, the deformation field model is as follows: ; in, Represents the deformation field. Represents the pixels in the template image. The first sampled from the edge map of the template image represents the... i One control point, n Indicates the number of control points. Indicates the first i The thin-plate spline coefficients corresponding to each control point , representing the basis functions of thin plate splines. Represents the pixels in the template image p With the i Distance values ​​between control points Indicates the first j One B-spline basis function, m This represents the number of B-spline basis functions. Indicates the first j The coefficients of the B-spline basis functions.

[0011] Furthermore, the multi-objective optimization function is: ; in, This represents the distance map after the template image has been deformed. Represents the distance map of the search image. Indicates the matching score. This represents the deformation field regularization term. , , This represents the weighting coefficient.

[0012] In addition, the present invention also provides a template matching system based on gradient direction and distance-depth fusion, comprising: The template image preprocessing module is used to input a template image, perform multi-scale feature extraction on the template image, and generate a multi-feature template; wherein, the multi-feature template includes a distance map, gradient direction map, edge map, and index map of the template image; The search image preprocessing module is used to input the search image and generate the gradient direction map of the search image; The search matching module is used to perform search matching based on the gradient direction map of the search image and the template image to determine the optimal search pose. In the search matching process, for each candidate search pose, firstly, the pixel correspondence between the search image and the template image is matched based on the edge map and index map of the template image. Then, the distance weight is calculated based on the distance map of the template image, and the gradient direction maps of the template image and the search image are combined to perform distance weighted calculation to obtain the gradient direction similarity between each pair of matched pixels. Based on the gradient direction similarity of all matched pixel pairs, the matching score of each candidate search pose is calculated, and the candidate search pose with the highest matching score is taken as the optimal search pose. The candidate region determination module is used to determine the candidate region in the search image that is most similar to the template image based on the optimal search pose.

[0013] In addition, the present invention also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor executes the steps of the method described above by calling the computer program stored in the memory.

[0014] In addition, the present invention provides a computer-readable storage medium for storing a computer program for template matching based on gradient direction and distance-depth fusion, wherein the computer program executes the steps of the method described above when running on a computer.

[0015] The present invention has the following beneficial effects: The template matching method based on gradient direction and distance-depth fusion of the present invention first performs pixel correspondence matching between the search image and the template image through the edge map and index map of the template image in the search matching stage. This can accurately establish the pixel correspondence under deformation matching. Then, the distance weight is calculated using the distance map of the template image, and the gradient direction similarity is weighted. When deformation causes the position of edge pixels to change, the distance weight will be automatically adjusted, so that the algorithm naturally tends to pixels that are closer to the edge and have a more stable structure. It can automatically ignore unreliable matching points caused by deformation, occlusion or noise. Even if the edge position is offset, the depth fusion of gradient direction and distance can effectively avoid local mismatches and ensure that even if the target undergoes elastic deformation, the matching process can still maintain stability based on the global shape information of the template image, thus achieving a balance between high deformation robustness and high positioning accuracy.

[0016] In addition, the template matching system based on gradient direction and distance-depth fusion of the present invention also has the above-mentioned advantages.

[0017] In addition to the objectives, features, and advantages described above, the present invention has other objectives, features, and advantages. The invention will now be described in further detail with reference to the figures. Attached Figure Description

[0018] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart illustrating a preferred embodiment of the template matching method based on gradient direction and distance-depth fusion in this application. Figure 2 This is another flowchart illustrating the template matching method based on gradient direction and distance-depth fusion according to a preferred embodiment of this application; Figure 3 yes Figure 2 A schematic diagram of the sub-process of step S3a; Figure 4 This is a schematic diagram of the module structure of a template matching system based on gradient direction and distance-depth fusion according to another embodiment of this application. Detailed Implementation

[0019] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] Reference Figure 1 A preferred embodiment of this application provides a template matching method based on gradient direction and distance-depth fusion, including the following: Step S1: Input a template image, perform multi-scale feature extraction on the template image, and generate a multi-feature template; wherein, the multi-feature template includes a distance map, gradient direction map, edge map, and index map of the template image; Step S2: Input the search image and generate the gradient direction map of the search image; Step S3: Perform search matching based on the gradient direction map of the search image and the template image to determine the optimal search pose; wherein, in the search matching process, for each candidate search pose, firstly, perform pixel-to-pixel matching between the search image and the template image based on the edge map and index map of the template image, then calculate the distance weight based on the distance map of the template image, and perform distance weighted calculation by combining the gradient direction maps of the template image and the search image to obtain the gradient direction similarity between each pair of matched pixels, calculate the matching score of each candidate search pose based on the gradient direction similarity of all matched pixel pairs, and take the candidate search pose with the highest matching score as the optimal search pose; Step S4: Determine the candidate region in the search image that is most similar to the template image based on the optimal search pose.

[0021] It is understood that the template matching method based on gradient direction and distance-depth fusion in this embodiment first performs pixel correspondence matching between the search image and the template image through the edge map and index map of the template image during the search matching stage. This can accurately establish the pixel correspondence under deformation matching. Then, the distance weight is calculated using the distance map of the template image, and the gradient direction similarity is weighted. When deformation causes the position of edge pixels to change, the distance weight will be automatically adjusted, so that the algorithm naturally tends to pixels that are closer to the edge and have a more stable structure. It can automatically ignore unreliable matching points caused by deformation, occlusion or noise. Even if the edge position is offset, the depth fusion of gradient direction and distance can effectively avoid local mismatches and ensure that even if the target undergoes elastic deformation, the matching process can still maintain stability based on the global shape information of the template image, thus achieving a balance between high deformation robustness and high positioning accuracy.

[0022] In step S1, a template image containing the target object is input. The image format needs to be converted from a color image to a grayscale image, for example, using a weighted average method: grayscale = 0.299R + 0.587G + 0.114B. Of course, if the input template image itself is a grayscale image, no conversion is needed. Optionally, after converting to a grayscale image, Gaussian blur processing can be applied to the grayscale image to suppress noise. The Gaussian kernel size is usually set to 3×3 or 5×5, and the specific choice can be made according to the image noise level. For example, a 3×3 kernel can be selected for low-noise images, and a 5×5 kernel can be selected for high-noise images. Then, the Canny operator is used to process the template image to generate a binarized edge map, where the pixel value of edge pixels is 255, and the pixel value of non-edge pixels is 0. The edge map includes the position coordinates of each edge pixel (i.e., the edge point set). Typically, the coordinates are stored as a list for later processing. Next, a distance transformation is applied to the edge map to generate a distance map of the template image. When the image is small, a vectorized implementation of the Euclidean distance transformation can be used, with the formula: When the image is large-scale, an approximation algorithm (such as a fast marching method) is used to improve efficiency. At the same time, an index map is generated to record any pixel in the template image. p Corresponding nearest edge pixel q In practice, the coordinates can be stored using a two-dimensional array, where each element is a pair of coordinates (x, y). For example, if point... p A pixel that is itself an edge pixel has a distance of 0 to its nearest edge point, meaning the distance value is: The index map records the points. p The coordinates for non-edge pixels p =(10,20), if its nearest edge point is q =(12,18), then the coordinates recorded in the index map are (12,18). For example, assuming the edge point set contains points (5,5) and (10,10), for non-edge pixels... p =(8,8), calculate the distance , ,but The indexed image records the coordinates as (10, 10). Additionally, the Sobel operator is used to calculate the gradient of the template image, where the Sobel operator's convolution kernel can be set as follows: , The calculation process is as follows: First, perform convolution calculation on the template image: , Then, calculate the gradient magnitude. and gradient direction The gradient direction results are then converted to angles (-180° to 180°). Finally, the distance map, gradient direction map, edge map, and index map of the template image, along with necessary metadata (such as image sluggishness, center point, etc.), are stored together as a multi-feature template. Optionally, the multi-feature template may also include a gradient magnitude map.

[0023] In addition, in step S2, the search image is input and processed using the same feature extraction method and parameter configuration as in step S1 to obtain the distance map, gradient direction map, edge map, index map and necessary metadata of the search image. The gradient direction map is extracted for subsequent search matching. Of course, if necessary, at least one of the distance map, edge map and index map can be extracted to participate in search matching.

[0024] Furthermore, in step S3, during the search matching stage, the goal is to traverse different positions, scales, and rotation angles (i.e., search poses) in the search image to determine the optimal search pose, thereby finding the candidate region most similar to the template image. Specifically, during the search matching process, for each candidate search pose (translation (x,y), scale s, rotation angle θ), pixel-to-pixel matching between the search image and the template image is first performed based on the edge map and index map of the template image. For example, for any pixel in the template image... p Find the corresponding nearest edge point by querying the index graph. q Then point q Based on the transformation of the candidate search pose to the search image coordinate system, the transformation formula is as follows: , where the rotation matrix This allows for the accurate establishment of pixel correspondences under deformation matching. Then, the distance weights are calculated based on the following formula: ,in, Represents distance value The corresponding distance weights, The standard deviation is used to assign a smaller distance weight to pixels farther from the edge. This distance weight is then calculated by combining the gradient direction maps of the template image and the search image to obtain the gradient direction similarity between each pair of matched pixels. Based on the gradient direction similarity of all matched pixel pairs, a matching score for each candidate search pose is calculated. Specifically, the matching score for each candidate search pose is calculated using the following formula: ; in, Indicates the matching score. Indicates distance weight, Represents the distance of the template image to the edge pixels in the image. q The corresponding distance values ​​show that pixels farther from the edge have smaller distance weights. Edge pixels in the gradient direction map of the template image q gradient direction, Represents the pixels in the gradient direction map of the search image. The gradient direction can be obtained by interpolation through searching the gradient direction map of the image. The specific interpolation method can be selected according to actual needs. This invention uses bilinear interpolation to avoid discretization errors. Edge pixels in the template image q and search for pixels in the image For a pair of matched pixels, N Indicates the number of matched pixel pairs, calculated as a dot product. , This represents the gradient direction difference. After calculating the matching score for each candidate search pose, the candidate search pose with the highest matching score is taken as the optimal search pose. Alternatively, in other embodiments of the invention, linear or non-linear decay methods can be used to calculate the distance weight, such that pixels farther from the edge pixels receive a higher weight. p The smaller the corresponding distance weight.

[0025] It is understandable that edge pixels represent important structural features in the template image. When performing gradient direction similarity calculation, this invention only performs matching calculations on edge pixels to avoid traversing all pixels in the image, thus ensuring both accuracy and efficiency. Furthermore, through deep fusion of gradient direction and distance weights, the distance weight of points farther from the edge decays exponentially, emphasizing the importance of edge structure. At the same time, it takes into account gradient information near the edge, so that the algorithm naturally tends to pixels closer to the edge and with more stable structures. It can automatically ignore unreliable matching points caused by deformation, occlusion, or noise. Even if the edge position is offset, through the weighted calculation of all matching pixel pairs, it can effectively avoid local mismatches and ensure that even if the target undergoes elastic deformation, the matching process can still maintain stability based on the global shape information of the template image, achieving a balance between high deformation robustness and high positioning accuracy.

[0026] Optionally, to further improve search matching efficiency, a pyramid hierarchical search approach can be adopted to determine the optimal search pose more quickly. Specifically, Gaussian pyramids are constructed for both the template image and the search image to support multi-scale matching. The number of pyramid layers is set to 5, which ensures scale robustness without increasing computational cost. Layer 0 is the original resolution image, and layers 1 to 4 are generated through Gaussian blur and downsampling. Gaussian blur is applied to the previous layer image first, followed by downsampling using bilinear interpolation with a scaling factor of 0.5, halving the image size of each layer. For example, if the original size is 800×600, then layer 1 is 400×300, layer 2 is 200×150, and so on. Finally, the pyramid images are stored as an array, with metadata recording the size and scaling factor of each layer, and generating a multi-feature template for each layer image. When performing pyramid layer search, the search is first performed at the top level, which is a full-range coarse search on the low-resolution image. The parameter range can be set as follows: translation (x,y) covers the entire image range with a step size of 1 / 10 of the image width and height, scale s ranges from 0.5 to 2.0 with a step size of 0.1, and rotation angle θ ranges from -180° to 180° with a step size of 10°. When refining the lower layers, the top 10% of candidate search poses with the highest matching scores at the top level are used as initial values ​​and passed to the next layer. Furthermore, the step size decreases layer by layer. For example, the step size is 20 pixels for the third layer, 10 pixels for the second layer, and so on until the original resolution layer.

[0027] In addition, in step S4, after determining the optimal search pose, the candidate region in the search image that is most similar to the template image can be determined. The specific process is existing technology and will not be described in detail here.

[0028] Optional, such as Figure 2 As shown, after determining the optimal search pose, the following content is also included: Step S3a: Perform fine-tuning of the optimal search pose to achieve fine-tuning of the optimal search pose.

[0029] Among them, such as Figure 3 As shown, the process of fine-tuning the optimal search pose includes the following: Step S31a: Initialize deformation field optimization based on the optimal search pose; Step S32a: Construct a deformation field model and construct a multi-objective optimization function to solve the deformation field model to obtain the optimal deformation field model; Step S33a: Perform nonlinear optimization based on the optimal deformation field model to obtain the fine-tuned optimal search pose.

[0030] Specifically, the optimal search pose is first selected as the initial value for deformation field optimization, and the deformation field is then... Initialize to the identity transformation, i.e. and from the edge point set of the template image edge map Q Approximately 50 uniform sampling control points are used for deformation field modeling.

[0031] Then, thin plate splines and B-splines are used to construct a deformation field model, which can be expressed as: ; in, Represents the deformation field. Represents the pixel points (i.e., coordinates) in the template image. The first sampled from the edge map of the template image represents the... i One control point, n Indicates the number of control points. Indicates the first i The thin-plate spline coefficients corresponding to each control point , representing the basis functions of thin-plate splines, is responsible for accurately modeling the overall, smooth, and non-rigid deformation of the target object, and can handle global deformation with high precision. Represents the pixels in the template image p With the i Distance values ​​between control points Indicates the first j This invention preferably uses cubic B-spline basis functions, which can achieve accurate modeling of local deformations. The grid spacing is controlled to be 10-20 pixels. m This represents the number of B-spline basis functions, which depends on the control network that defines the B-splines on the template image. Indicates the first j The coefficients of the B-spline basis functions. Among them, and These are unknown variables to be optimized in the deformation field model. To find the optimal variables, it is necessary to construct an objective function that can measure the quality of the deformation field. This invention simultaneously considers deformation consistency, feature matching degree, and motion smoothness, and constructs a multi-objective function to minimize the objective function, which can be expressed as: ; in, This represents the distance map after the template image has been deformed. Represents the distance map of the search image. Indicates the matching score. The deformation field regularization term can be represented by a second derivative penalty. , , This represents the weighting coefficient, which can be set according to actual needs. For example, this invention sets... 0.5 0.3 The value is 0.2. The first term represents deformation consistency, which measures the optimization of the deformation field by comparing the difference between the distance field inside the template after deformation and the distance field of the search image. This ensures that the edge contour of the template image, after elastic deformation, coincides as closely as possible with the target edge contour in the search image. Deformation consistency provides global smooth geometric constraints, is insensitive to local occlusion and noise, and guides optimization to avoid local minima. The second term represents feature matching degree, which provides fine-grained local feature constraints, ensuring that the detailed direction of the edges is consistent in addition to overall shape alignment, thus achieving sub-pixel-level positioning accuracy. The third term represents motion smoothness, a regularization term that prevents unnatural or overly drastic distortions in the deformation field, ensuring physical plausibility. Next, the gradient descent method or the Gauss-Newton method is used to solve the deformation field model to obtain the optimal variable parameters. and Thus, the optimal deformation field model is obtained.

[0032] Finally, after the deformation field model is optimized, nonlinear optimization is performed based on the optimal deformation field model to further optimize the global pose, obtaining the fine-tuned optimal search pose (including position (x,y), scaling scale s, and rotation angle θ). The nonlinear optimization can employ the existing Levenberg-Marquardt algorithm for multi-scale optimization, optimizing layer by layer from low resolution to high resolution to avoid local optima. Furthermore, after optimization convergence, the final fusion score and optional deformation field parameters can be output.

[0033] It is understood that the optimal search pose obtained in step S3 is the best rigid body pose. Based on this, the present invention introduces a parameterized deformation field, evolving from rigid body pose to elastic deformation field optimization. Furthermore, thin plate splines and B-splines are used to construct the deformation field model, comprehensively considering high-precision modeling of global and local deformation. The two work together to achieve global optimal modeling. At the same time, when constructing the objective function, multiple objectives such as deformation consistency, feature matching degree, and motion smoothness are comprehensively considered, which can accurately solve for the optimal variables of the deformation field. Then, nonlinear optimization is performed based on the optimal deformation field model. Through the stepwise optimization strategy of first rigid, then elastic, and then overall fine-tuning, the optimization can avoid getting trapped in local optima, balancing efficiency and accuracy. It can also achieve robust matching for complex deformations, ensuring the physical legality and stability of the results. More importantly, it can achieve sub-pixel level positioning accuracy.

[0034] In addition, such as Figure 4As shown, another embodiment of the present invention also provides a template matching system based on gradient direction and distance-depth fusion, preferably employing the template matching method based on gradient direction and distance-depth fusion as described above, including: The template image preprocessing module is used to input a template image, perform multi-scale feature extraction on the template image, and generate a multi-feature template; wherein, the multi-feature template includes a distance map, gradient direction map, edge map, and index map of the template image; The search image preprocessing module is used to input the search image and generate the gradient direction map of the search image; The search matching module is used to perform search matching based on the gradient direction map of the search image and the template image to determine the optimal search pose. In the search matching process, for each candidate search pose, firstly, the pixel correspondence between the search image and the template image is matched based on the edge map and index map of the template image. Then, the distance weight is calculated based on the distance map of the template image, and the gradient direction maps of the template image and the search image are combined to perform distance weighted calculation to obtain the gradient direction similarity between each pair of matched pixels. Based on the gradient direction similarity of all matched pixel pairs, the matching score of each candidate search pose is calculated, and the candidate search pose with the highest matching score is taken as the optimal search pose. The candidate region determination module is used to determine the candidate region in the search image that is most similar to the template image based on the optimal search pose.

[0035] It is understood that the template matching system based on gradient direction and distance depth fusion in this embodiment first performs pixel correspondence matching between the search image and the template image through the edge map and index map of the template image during the search matching stage. This can accurately establish the pixel correspondence under deformation matching. Then, the distance weight is calculated using the distance map of the template image, and the gradient direction similarity is weighted. When deformation causes the position of edge pixels to change, the distance weight will be automatically adjusted, so that the algorithm naturally tends to pixels that are closer to the edge and have a more stable structure. It can automatically ignore unreliable matching points caused by deformation, occlusion or noise. Even if the edge position is offset, the depth fusion of gradient direction and distance can effectively avoid local mismatches and ensure that even if the target undergoes elastic deformation, the matching process can still maintain stability based on the global shape information of the template image, thus achieving a balance between high deformation robustness and high positioning accuracy.

[0036] In addition, the template matching system based on gradient direction and distance-depth fusion also includes: The fine-tuning module is used to fine-tune the optimal search pose, enabling adjustments to the optimal search pose.

[0037] It is understood that each module of this system embodiment corresponds to each step of the above method embodiment. Therefore, the specific working principle of each module will not be repeated here, and you can refer to the steps of the above method embodiment.

[0038] In addition, another embodiment of the present invention provides an electronic device including a processor and a memory, wherein the memory stores a computer program, and the processor executes the steps of the method described above by calling the computer program stored in the memory.

[0039] In addition, another embodiment of the present invention provides a computer-readable storage medium for storing a computer program for template matching based on gradient direction and distance-depth fusion, wherein the computer program executes the steps of the method described above when running on a computer.

[0040] Common computer-readable storage media include: floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, any other optical media, punch cards, paper tape, any other physical media with perforated patterns, random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), flash erasable programmable read-only memory (FLASH-EPROM), any other memory chips or cartridges, or any other media readable by a computer. Instructions may further be transmitted or received by a transmission medium. The term transmission medium can include any tangible or intangible medium used to store, encode, or carry instructions for execution by a machine, and includes digital or analog carrier communication signals or intangible media that facilitate communication of such instructions. Transmission media include coaxial cables, copper wires, and optical fibers, which contain conductors for transmitting a bus of computer data signals.

[0041] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0042] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0043] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0044] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0045] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0046] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

[0047] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A template matching method based on gradient direction and distance-depth fusion, characterized in that, Includes the following: Input a template image, perform multi-scale feature extraction on the template image to generate a multi-feature template; wherein, the multi-feature template includes a distance map, gradient direction map, edge map and index map of the template image, and the index map is used to record any pixel in the template image. p Corresponding nearest edge pixel q The coordinates; Input a search image and generate a gradient direction map of the search image; The optimal search pose is determined by matching the gradient direction map of the search image with the template image. In the search matching process, for each candidate search pose, the pixel correspondence between the search image and the template image is first matched based on the edge map and index map of the template image. Then, the distance weight is calculated based on the distance map of the template image, and the gradient direction maps of the template image and the search image are combined to perform distance weighted calculation to obtain the gradient direction similarity between each pair of matched pixels. The matching score of each candidate search pose is calculated based on the gradient direction similarity of all matched pixel pairs. The candidate search pose with the highest matching score is taken as the optimal search pose. The candidate region in the search image that is most similar to the template image is determined based on the optimal search pose.

2. The template matching method based on gradient direction and distance-depth fusion as described in claim 1, characterized in that, The matching score for each candidate search pose is calculated based on the following formula: ； in, Indicates the matching score. Indicates distance weight, Represents the distance of the template image to the edge pixels in the image. q The corresponding distance values ​​show that pixels farther from the edge have smaller distance weights. Edge pixels in the gradient direction map of the template image q gradient direction, Represents the pixels in the gradient direction map of the search image. gradient direction, edge pixels q and pixels For a pair of matched pixels, N This indicates the number of matched pixel pairs.

3. The template matching method based on gradient direction and distance-depth fusion as described in claim 2, characterized in that, The distance weights are calculated based on the following formula: ； in, Represents distance value The corresponding distance weights, It represents the standard deviation.

4. The template matching method based on gradient direction and distance-depth fusion as described in claim 1, characterized in that, After determining the optimal search pose, the following also applies: The optimal search pose is finely optimized to achieve fine-tuning of the optimal search pose.

5. The template matching method based on gradient direction and distance-depth fusion as described in claim 4, characterized in that, The process of fine-tuning the optimal search pose includes the following: Deformation field optimization initialization is performed based on the optimal search pose; A deformation field model is constructed, and a multi-objective optimization function is constructed to solve the deformation field model to obtain the optimal deformation field model; Nonlinear optimization is performed based on the optimal deformation field model to obtain the fine-tuned optimal search pose.

6. The template matching method based on gradient direction and distance-depth fusion as described in claim 5, characterized in that, The deformation field model is as follows: ； in, Represents the deformation field. Represents the pixels in the template image. The first sampled from the edge map of the template image i One control point, n Indicates the number of control points. Indicates the first i The thin-plate spline coefficients corresponding to each control point , representing the basis functions of thin plate splines. Represents the pixels in the template image p With the i Distance values ​​between control points Indicates the first j A B-spline basis function, m This represents the number of B-spline basis functions. Indicates the first j The coefficients of the B-spline basis functions.

7. The template matching method based on gradient direction and distance-depth fusion as described in claim 6, characterized in that, The multi-objective optimization function is: ； in, This represents the distance map after the template image has been deformed. Represents the distance map of the search image. Indicates the matching score. This represents the deformation field regularization term. , , This represents the weighting coefficient.

8. A template matching system based on gradient direction and distance-depth fusion, characterized in that, include: The template image preprocessing module is used to input a template image, perform multi-scale feature extraction on the template image, and generate a multi-feature template. The multi-feature template includes a distance map, gradient direction map, edge map, and index map of the template image. The index map is used to record any pixel in the template image. p Corresponding nearest edge pixel q The coordinates; The search image preprocessing module is used to input the search image and generate the gradient direction map of the search image; The search matching module is used to perform search matching based on the gradient direction map of the search image and the template image to determine the optimal search pose. In the search matching process, for each candidate search pose, firstly, the pixel correspondence between the search image and the template image is matched based on the edge map and index map of the template image. Then, the distance weight is calculated based on the distance map of the template image, and the gradient direction maps of the template image and the search image are combined to perform distance weighted calculation to obtain the gradient direction similarity between each pair of matched pixels. Based on the gradient direction similarity of all matched pixel pairs, the matching score of each candidate search pose is calculated, and the candidate search pose with the highest matching score is taken as the optimal search pose. The candidate region determination module is used to determine the candidate region in the search image that is most similar to the template image based on the optimal search pose.

9. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program, and the processor executes the steps of the method as described in any one of claims 1 to 7 by calling the computer program stored in the memory.

10. A computer-readable storage medium for storing a computer program for template matching based on gradient direction and distance-depth fusion, characterized in that, The computer program, when run on a computer, performs the steps of the method as described in any one of claims 1 to 7.

Images