Template matching method and device, electronic equipment and storage medium
By generating an image pyramid and matching it with a preset template image pyramid, and by using sub-pixel edge point optimization, the problems of high computational complexity and susceptibility to environmental interference in template matching methods are solved, achieving efficient and accurate matching results.
Patent Information
- Application Number
- CN202411037050.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-22
AI Technical Summary
Existing template matching methods have high computational complexity, resulting in slow matching speed and susceptibility to environmental interference, which affects matching accuracy.
By generating an image pyramid, the image to be matched is matched with a preset template image pyramid. Subpixel edge points are used to optimize the initial matching results, reducing computation and improving accuracy.
It improves the efficiency and accuracy of template matching, reduces the impact of environmental interference, and ensures the stability and accuracy of matching results.
Smart Images

Figure CN119006856A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a template matching method, device, electronic equipment and storage medium. Background Art
[0002] Template matching is a common and important image recognition and target detection technology in the field of computer vision and image processing. This technology is based on specific patterns or templates in the image, and locates and identifies the target by finding similar areas in the target image. Template matching has a wide range of uses in various applications, including target tracking, image retrieval, face recognition, medical image analysis, etc.
[0003] In traditional template matching methods, commonly used techniques include square difference matching, correlation matching, and normalized cross-correlation matching. These methods find the best matching position by calculating the similarity between the template and each position in the image, thereby achieving target detection. However, traditional template matching methods have some limitations. First, for large-size images and complex scenes, the computational complexity is high, resulting in slow matching speed. Secondly, due to the influence of factors such as illumination changes, occlusion, and noise, traditional methods are easily disturbed, resulting in reduced matching accuracy. Therefore, how to achieve a template matching method with high matching accuracy and not affected by the environment has become a problem that needs to be solved urgently. Summary of the invention
[0004] The present invention provides a template matching method, device, electronic device and storage medium to solve the problems in the prior art that the template matching is relatively complex, resulting in a relatively slow matching speed, and the template matching process is susceptible to environmental interference.
[0005] According to one aspect of the present invention, a template matching method is provided, wherein the method comprises:
[0006] Generate an image pyramid according to the image to be matched; wherein the target number of layers of the image pyramid is the same as the preset number of layers of the preset template image pyramid;
[0007] Matching the first pyramid image of each layer in the image pyramid with the second pyramid image of the corresponding layer in the preset template image pyramid in sequence to obtain an initial matching result;
[0008] Sub-pixel edge points of the preset template image pyramid are obtained, and the initial matching result is optimized based on the sub-pixel edge points to obtain a target matching result.
[0009] According to another aspect of the present invention, a template matching device is provided, wherein the device comprises:
[0010] A pyramid generation module, used to generate an image pyramid according to the image to be matched; wherein the target number of layers of the image pyramid is the same as the preset number of layers of the preset template image pyramid;
[0011] An initial matching module, used for sequentially matching the first pyramid image of each layer in the image pyramid with the second pyramid image of the corresponding layer in the preset template image pyramid to obtain an initial matching result;
[0012] The target matching module is used to obtain the sub-pixel edge points of the preset template image pyramid, and optimize the initial matching results based on the sub-pixel edge points to obtain the target matching results.
[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:
[0014] at least one processor; and
[0015] a memory communicatively connected to the at least one processor; wherein,
[0016] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the template matching method described in any embodiment of the present invention.
[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the template matching method described in any embodiment of the present invention when executed.
[0018] The technical solution of the embodiment of the present invention generates an image pyramid according to the image to be matched, and sequentially matches the first pyramid image of each layer in the image pyramid with the second pyramid image of the corresponding layer in the preset template image pyramid to obtain an initial matching result. The image pyramid matching method reduces the amount of calculation of template matching, thereby improving the matching efficiency. At the same time, by obtaining the sub-pixel edge points of the preset template image pyramid, the initial matching result is optimized based on the sub-pixel edge points to obtain the target matching result, and the sub-pixel edge points are used to refine and adjust the posture, so that the accuracy of the target matching result is higher, thereby ensuring the accuracy of template matching.
[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0021] Figure 1 is a flow chart of a template matching method provided according to Embodiment 1 of the present invention;
[0022] Figure 2 is a flow chart of another template matching method provided according to the second embodiment of the present invention;
[0023] Figure 3 is a flow chart of another template matching method provided according to Embodiment 3 of the present invention;
[0024] Figure 4 is a schematic diagram of a pyramid processing provided according to Embodiment 3 of the present invention;
[0025] Figure 5 is a schematic diagram of an edge image provided according to Embodiment 3 of the present invention;
[0026] Figure 6 is a schematic diagram of a target image and an image pyramid provided according to Embodiment 3 of the present invention;
[0027] Figure 7 is a schematic diagram of a top-level candidate matching result provided according to Embodiment 3 of the present invention;
[0028] Figure 8 is a schematic diagram of an initial matching result of a bottom layer provided according to Embodiment 3 of the present invention;
[0029] Fig. 9 is a schematic diagram of a target matching result provided according to Embodiment 3 of the present invention;
[0030] Fig.10 is a structural schematic diagram of a template matching device provided according to a fourth embodiment of the present invention;
[0031] Fig.11 It is a schematic diagram of the structure of an electronic device for implementing the template matching method according to an embodiment of the present invention. DETAILED DESCRIPTION
[0032] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0034] Embodiment 1
[0035] Figure 1 1 is a flow chart of a template matching method provided according to the first embodiment of the present invention. This embodiment is applicable to the case of identifying target matching results in an image through template matching. The method can be executed by a template matching device. The template matching device can be implemented in the form of hardware and / or software. The template matching device can be configured in an electronic device. Figure 1 As shown, the method includes:
[0036] S110 , generating an image pyramid according to the image to be matched; wherein the target number of layers of the image pyramid is the same as the preset number of layers of the preset template image pyramid.
[0037] Among them, the image to be matched can be understood as the initial image for which the matching result is to be determined. Exemplarily, the image to be matched can be a high-resolution image. Image pyramid is a kind of multi-scale expression of an image, and is an effective but conceptually simple structure for interpreting an image with multiple resolutions. The image pyramid of an image to be matched is a set of image resolutions that are gradually reduced in a pyramid shape (from bottom to top) and are derived from the same image to be matched. It is obtained by stepwise down sampling until a certain target number of layers is reached before sampling is stopped. The higher the level, the smaller the image size and the lower the resolution. The preset template image pyramid refers to a preset image pyramid generated according to the template image, and the target number of layers of the image pyramid is the same as the preset number of layers of the preset template image pyramid, so as to facilitate the matching of images at each layer. Exemplarily, the target number of layers may include but is not limited to 3 layers, 4 layers, and 5 layers.
[0038] In an embodiment, the image to be matched can be used as the first pyramid image of the bottom layer in the image pyramid, and the image to be matched can be downsampled to obtain the first pyramid image of the upper layer of the bottom layer, and then the first pyramid image of the upper layer can be downsampled as the new image to be matched until the first pyramid image of the target number of layers is obtained to obtain the image pyramid. In the actual operation process, the image to be matched can be preprocessed to obtain the target image, such as filtering and noise reduction, to reduce the noise of the image to be matched. Then, the even rows and even columns of the target image are deleted for downsampling to obtain the first pyramid image of the upper layer of the bottom layer, and the obtained first pyramid image is continuously downsampled until the number of pyramid layers reaches the target number of layers.
[0039] S120 , sequentially matching the first pyramid image of each layer in the image pyramid with the second pyramid image of the corresponding layer in the preset template image pyramid to obtain an initial matching result.
[0040] Among them, the first pyramid image refers to the downsampled image of each layer in the image pyramid; the second pyramid image refers to the downsampled image of each layer in the preset template image pyramid. The initial matching result can be understood as the result of matching the first pyramid image of each layer in the image pyramid with the second pyramid image of the corresponding layer in the preset template image pyramid layer by layer. The range of the initial matching result of the first pyramid image that performs matching first includes the matching range of the first pyramid image that performs matching later, that is, the initial matching result is the image matching result of the first pyramid image at the bottom layer in the image pyramid, that is, the initial matching result of the image to be matched. In the actual operation process, the initial matching result can be a candidate target matching result of the image to be matched.
[0041] In an embodiment, a first pyramid image at the top layer in an image pyramid may be template matched with a second pyramid image at the top layer in a preset template image pyramid to obtain an initial matching result in the first pyramid image, and then a corresponding position of the initial matching result is determined in the first pyramid image at a layer below the top layer in the image pyramid, and template matching is performed with a second pyramid image at a layer below the top layer in the preset template image pyramid according to the corresponding position to obtain a new initial matching result, until the first pyramid images of each layer are matched, and a candidate matching result of the first pyramid image at the bottom layer is obtained as the initial matching result. In practical applications, the rotation angle may be adjusted for the second pyramid image of each layer in the preset template image pyramid to obtain initial matching results at each rotation angle.
[0042] In the actual operation process, the edge point pixel coordinates of the second pyramid image corresponding to the top layer in the preset template image pyramid and the gradient values corresponding to the edge point pixel coordinates can be obtained, and the edge point pixel coordinates and the edge gradient values can be used as initial matching data. Generally speaking, the edge point pixel coordinates of the second pyramid image corresponding to the top layer in the preset template image pyramid and the gradient values corresponding to the edge point pixel coordinates can be pre-extracted after constructing the preset template image pyramid. At the same time, the angle rotation step length, the preset initial rotation angle, and the preset rotation range of the second pyramid image can be determined, and the rotation angle of the second pyramid image at the top layer is determined according to the preset initial rotation angle, the preset rotation range, and the angle rotation step length. Among them, the angle rotation step length can be determined based on the number of layers to which the second pyramid image belongs, the preset image height, and the preset image width. Exemplarily, the angle rotation step length of each layer can be Among them, A L is the angle rotation step of each layer, w is the width of the second pyramid image, h is the height of the second pyramid image, and L is the number of layers to which the second pyramid image belongs. According to the rotation angle, the initial matching data and the preset scaling factor, the transformed edge point pixel coordinates corresponding to each rotation angle and the gradient value corresponding to the transformed edge point pixel coordinates can be determined, and the transformed edge point pixel coordinates and the gradient value corresponding to the transformed edge point pixel coordinates are used as edge transformation data. According to each edge transformation data and the first pyramid image of the top layer in the image pyramid, the matched transformed edge data and the matching score can be obtained, and the transformed edge data corresponding to the matching score greater than the preset score threshold can be used as the candidate matching result. Exemplarily, the matching score of each transformed edge data can be calculated by a normalized product correlation algorithm, a mean absolute difference algorithm, an absolute error sum algorithm, an error square sum algorithm, and an average error square sum algorithm, etc., which are not limited to this.
[0043] S130, obtaining sub-pixel edge points of a preset template image pyramid, and optimizing an initial matching result based on the sub-pixel edge points to obtain a target matching result.
[0044] Among them, the sub-pixel edge point refers to the edge position in the image whose accuracy reaches the sub-pixel level. In the actual operation process, the sub-pixel accurate edge extraction algorithm based on the facet model can be used to determine the sub-pixel edge points of the preset template image pyramid. Generally speaking, the sub-pixel edge points of the preset template image pyramid can be predetermined after the preset template image pyramid is constructed. The target matching result can be understood as the final template matching result of the image to be matched. Exemplarily, the target matching result may include information such as the image contour and image positioning position in the image to be matched.
[0045] In an embodiment, sub-pixel edge points of a preset template image pyramid may be extracted, and sub-pixel edge points adjacent to the transformed edge data in each initial matching result may be determined respectively. The sum of squared distances between the transformed edge data belonging to the same rotation angle and the sub-pixel edge points adjacent to each transformed edge data may be determined based on a preset least squares method, and an initial matching result corresponding to the minimum sum of squared distances may be determined, and the initial matching result may be used as a target matching result.
[0046] In the embodiment of the present invention, an image pyramid is generated according to an image to be matched, and the first pyramid image of each layer in the image pyramid is matched with the second pyramid image of the corresponding layer in the preset template image pyramid in sequence to obtain an initial matching result. The calculation amount of template matching is reduced through the image pyramid matching method, thereby improving the matching efficiency. At the same time, by obtaining the sub-pixel edge points of the preset template image pyramid, the initial matching result is optimized based on the sub-pixel edge points to obtain the target matching result, and the sub-pixel edge points are used to refine and adjust the posture, so that the accuracy of the target matching result is higher, and the accuracy of the template matching is guaranteed.
[0047] In one embodiment, the process of creating a preset template image pyramid includes:
[0048] Acquire an initial template image, filter and reduce noise on the initial template image to obtain a template image, and use the template image as a second pyramid image at the bottom layer of a preset template image pyramid;
[0049] Down-sampling the template image to obtain a second pyramid image of the upper layer of the bottom layer;
[0050] Down-sampling the second pyramid image of the previous layer as a new template image until a second pyramid image of a preset number of layers is obtained, thereby obtaining a preset template image pyramid;
[0051] Determine the gradient value of each pixel in the second pyramid image based on a preset horizontal gradient matrix and a preset vertical gradient matrix, and aggregate the gradient values to obtain a gradient image of the second pyramid image;
[0052] Determine the edge image of the gradient image by using a preset edge detection rule, determine the pixel point coordinates corresponding to each edge image as the edge point pixel coordinates, and determine the gradient value corresponding to the edge point pixel coordinates as the edge gradient value;
[0053] The sub-pixel edge points of the template image are determined based on a preset sub-pixel edge extraction algorithm, and the edge point pixel coordinates, edge gradient values and sub-pixel edge points are used as edge information of a preset template image pyramid.
[0054] The initial template image can be understood as an image used to construct a preset template image pyramid, which is an initial template for image template matching. The template image can be understood as an initial template image for noise reduction or. The preset horizontal gradient matrix can be understood as a preset Sobel gradient matrix in the horizontal direction; the preset vertical gradient matrix can be understood as a preset Sobel gradient matrix in the vertical direction; illustratively, the preset horizontal gradient matrix Kx can be The preset vertical gradient matrix Ky can be
[0055] The image area can be the same as the number of elements of the preset horizontal gradient matrix and the preset horizontal gradient matrix. For example, when the preset horizontal gradient matrix and the preset horizontal gradient matrix are 3×3 matrices, the image area can be a 3×3 image area. The gradient value is a vector used to describe the rate of change of a function at a certain point. In image processing, the image can be regarded as a two-dimensional function, whose input is the coordinates of the pixel and the output is the grayscale value of the pixel. At this time, the gradient value reflects the degree and direction of change of the grayscale value of the point in the horizontal and vertical directions. The gradient value can be obtained by convolving the pixel values of each image area based on the preset horizontal gradient matrix and the preset vertical gradient matrix to obtain the horizontal gradient value and the vertical gradient value, and then determining the square root of the sum of the squares of the horizontal gradient value and the vertical gradient value. The gradient image is an image obtained by calculating the gradient values of each pixel in the image in the horizontal and vertical directions.
[0056] The preset edge detection rule refers to a preset rule for detecting edge images of gradient images. Exemplarily, the preset edge detection rule may include but is not limited to the Canny edge detection algorithm. Image edge points refer to pixel points where the grayscale value in the image changes sharply, and edge gradient values refer to the gradient values corresponding to the pixel coordinates of the edge points. The preset sub-pixel edge extraction algorithm may be a preset algorithm for extracting sub-pixel edge points. Exemplarily, the preset sub-pixel edge extraction algorithm may be a sub-pixel precise edge extraction algorithm based on a facet model.
[0057] In an embodiment, an initial template image may be obtained, the initial template image may be filtered and denoised to obtain a template image, and the template image may be used as a second pyramid image of a bottom layer in a preset template image pyramid, and then the template image may be downsampled to obtain a second pyramid image of a layer above the bottom layer, and then the first pyramid image of the layer above the bottom layer may be downsampled as a new template image until a second pyramid image of a preset number of layers is obtained, thereby obtaining an image pyramid. In an actual operation process, downsampling may be performed by deleting even rows and even columns of the template image to obtain a second pyramid image of a layer above the bottom layer, and the obtained second pyramid image may be continuously downsampled until the number of pyramid layers reaches a preset number of layers.
[0058] Determine 8 neighboring pixels adjacent to each pixel in the second pyramid image respectively, perform convolution operation on these 9 pixels with a preset horizontal gradient matrix to obtain a horizontal gradient value, perform convolution operation on these 9 pixels with a preset vertical gradient matrix to obtain a vertical gradient value, and determine the square root of the sum of the squares of the horizontal gradient value and the vertical gradient value as the gradient value of the pixel. Exemplarily, the gradient value can be calculated as follows: Among them, G(x,y) represents the gradient value at the pixel coordinate (x,y); Gx(x,y) represents the gradient value along the horizontal x direction at the pixel coordinate (x,y); Gy(x,y) represents the gradient value along the vertical y direction at the pixel coordinate (x,y). After obtaining the gradient value of each pixel, aggregate the gradient values to obtain the gradient image of the second pyramid image.
[0059] Then, the edge image of the gradient image is determined by the preset edge detection rule. Specifically, the preset edge detection rule may include the Canny edge detection algorithm. The threshold for Canny edge detection may be calculated according to the gradient image G: a cumulative histogram is generated by the gradient value of the gradient image G, the horizontal coordinate of the histogram is the gradient value, and the vertical coordinate is the cumulative value of the number of pixels corresponding to the gradient value. Then, the gradient size corresponding to the cumulative value of 0.9 times and 0.3 times the total number of pixels is used as the high threshold and low threshold of Canny edge detection. After obtaining the threshold, the edge image of the gradient image is obtained using the Canny detection algorithm. The pixel point coordinates corresponding to each edge image are determined as the edge point pixel coordinates, and the gradient value corresponding to the edge point pixel coordinates is determined as the edge gradient value. Edge gradient value (dx, dy); wherein, dx = Gx (x, y); dy = Gy (x, y), Gx (x, y) represents the gradient value along the horizontal x direction at the pixel point coordinate (x, y); Gy (x, y) represents the gradient value along the vertical y direction at the pixel point coordinate (x, y). In one embodiment, the pixel coordinates (x, y) of the edge point may be converted to relative coordinates (tx, ty) relative to the center point of the template in the following manner:
[0060] tx=xw / 2;
[0061] ty=yh / 2;
[0062] Wherein, w is the width of the second pyramid image, and h is the height of the second pyramid image.
[0063] The sub-pixel edge points of the template image are determined based on a preset sub-pixel edge extraction algorithm, and the edge point pixel coordinates, edge gradient values and sub-pixel edge points are used as edge information of a preset template image pyramid.
[0064] In one embodiment, determining sub-pixel edge points of a template image based on a preset sub-pixel edge extraction algorithm includes:
[0065] A surface is generated by fitting the pixel coordinates of edge points in the gradient image corresponding to the bottom second pyramid image through a preset sub-pixel edge extraction algorithm;
[0066] Gradient adjustment information of edge point pixel coordinates is determined based on the curved surface, and sub-pixel coordinates of the template image are determined based on the gradient adjustment information and the edge point pixel coordinates, and the pixel points corresponding to the sub-pixel coordinates are used as sub-pixel edge points of the preset template image pyramid.
[0067] The gradient adjustment information may be understood as information for adjusting the pixel coordinates of edge points.
[0068] In an embodiment, the gradient image corresponding to the second pyramid image at the bottom layer is the template image. After determining the pixel coordinates of the edge points in the gradient image of the template image, a surface can be generated by fitting the pixel coordinates of the edge points using a preset sub-pixel edge extraction algorithm. In the actual operation process, it is assumed that the preset sub-pixel edge extraction algorithm is a sub-pixel precise edge extraction algorithm based on a facet model. Assuming that (x, y) is the pixel coordinate of an edge point in the pixel edge, the facet model fits the template gradient image G at the coordinates into a three-dimensional surface.
[0069] z=k1y+k2x+k3y 2 +k4xy+k5x 2 ;
[0070] Among them, x, y are the pixel coordinates of the edge point, z is the pixel gradient, and the coefficients k1, k2, k3, k4 and k5 can be obtained by convolving the pixel coordinates of the edge point by the preset convolution matrix:
[0071] Among them, the preset convolution matrix can be respectively
[0072] The coefficients k1, k2, k3, k4 and k5 can be obtained by taking the dot product of the above five preset convolution matrices and the 3*3 area centered at the (x, y) pixel position in the gradient image G, thus completing the surface fitting and obtaining the surface function.
[0073] After obtaining the fitted surface function, the gradient direction corresponding to the pixel coordinates of the edge point can be determined as:
[0074]
[0075]
[0076] Then the sub-pixel coordinates corresponding to the pixel coordinates of the edge point can be calculated by the following formula:
[0077] sx=x+ρsinα;
[0078] sy=x+ρcosα;
[0079] in:
[0080]
[0081] ρ and the gradient direction can be used as gradient adjustment information, the sub-pixel coordinates of the template image can be determined based on the gradient adjustment information and the edge point pixel coordinates, and the pixel points corresponding to the sub-pixel coordinates in the template image can be used as the sub-pixel edge points of the preset template image pyramid.
[0082] In one embodiment, valid sub-pixel edge point conditions may be preset. For example, when the following conditions are met, the pixel point corresponding to the calculated coordinates is confirmed to be a valid sub-pixel edge point:
[0083] |ρsinα|<0.5;
[0084] |ρcosα|<0.5;
[0085] The pixel point corresponding to the valid sub-pixel coordinate in each sub-pixel coordinate in the template image is determined as the sub-pixel edge point.
[0086] Embodiment 2
[0087] Figure 2 1 is a flow chart of another template matching method provided according to the second embodiment of the present invention. This embodiment is further optimized and expanded based on the above embodiment and can be combined with various optional technical solutions in the above embodiment. Figure 2 As shown, the method includes:
[0088] S201 , obtaining an image to be matched, filtering and denoising the image to be matched to obtain a target image, and using the target image as a first pyramid image at the bottom layer of an image pyramid.
[0089] In an embodiment, after obtaining the image to be matched, the image to be matched is filtered and denoised to obtain a target image. In actual operation, the filtering and denoising methods may include Gaussian filtering, mean filtering, median filtering, and bilateral filtering. After obtaining the target image, the target image may be used as the first pyramid image at the bottom layer of the image pyramid.
[0090] S202 , down-sample the target image to obtain a first pyramid image of a layer above the bottom layer.
[0091] In an embodiment, down-sampling may be performed by deleting even-numbered rows and even-numbered columns of the target image to obtain a first pyramid image of a layer above the bottom layer.
[0092] S203 , down-sampling the first pyramid image of the previous layer as a new target image until the first pyramid image of the target number of layers is obtained, thereby obtaining an image pyramid.
[0093] In an embodiment, the first pyramid image of the previous layer may be used as a new target image, and the target image may be downsampled according to the same steps until the first pyramid image of the target layer is obtained, thereby obtaining an image pyramid.
[0094] S204, extracting edge point pixel coordinates of the second pyramid image corresponding to the top layer in the preset template image pyramid and edge gradient values corresponding to the edge point pixel coordinates, and using the edge point pixel coordinates and the edge gradient values as initial matching data.
[0095] In an embodiment, the edge point pixel coordinates of the second pyramid image and the edge gradient values corresponding to the edge point pixel coordinates may be predetermined, and the edge point pixel coordinates of the second pyramid image corresponding to the top layer in the predetermined preset template image pyramid and the edge gradient values corresponding to the edge point pixel coordinates may be extracted, and the edge point pixel coordinates and the edge gradient values may be used as initial matching data for matching the first pyramid image at the top layer.
[0096] S205: Determine the layer number, preset image height and preset image width of the second pyramid image of each layer in the preset template image pyramid, and determine the angle rotation step corresponding to each layer based on the layer number, preset image height and preset image width.
[0097] In an embodiment, the layer number, preset image height and preset image width of each layer of the second pyramid image in the preset template image pyramid can be extracted, and the angle rotation step corresponding to each layer can be determined by a preset algorithm. Exemplarily, the angle rotation step can be calculated as follows: Among them, A Lis the angle rotation step of each layer, w is the width of the second pyramid image, h is the height of the second pyramid image, and L is the number of layers to which the second pyramid image belongs.
[0098] S206 : Determine a rotation angle of the second pyramid image at the top layer according to a preset initial rotation angle of the second pyramid image at the top layer, a preset rotation range, and an angle rotation step length of the top layer.
[0099] In an embodiment, after determining a preset initial rotation angle, a preset rotation range, and an angle rotation step length of the second pyramid image of the top layer, the preset initial rotation angle may be increased according to the angle rotation step length of the top layer until the preset rotation range is reached, and an angle obtained by each increase is used as the rotation angle of the second pyramid image of the top layer.
[0100] S207 : Determine the transformed edge data corresponding to each rotation angle based on the rotation angle, the initial matching data and the preset scaling factor.
[0101] The transformed edge data includes the transformed edge point pixel coordinates and the edge gradient values corresponding to the transformed edge point pixel coordinates.
[0102] In an embodiment, for the initial matching data of the second pyramid image, due to the different rotation angles, the initial matching data may be subjected to rotation scaling transformation to obtain the transformed edge data corresponding to each rotation angle. Since the image sizes of the second pyramid image and the first pyramid image of the corresponding layer may be different, a preset scaling factor may be pre-set and the image size of the second pyramid image of the corresponding layer may be the same as that of the first pyramid image. In the actual operation process, for the rotation angle angle, the preset scaling factor scale, the transformed edge data after transformation may be:
[0103] tx′=a·tx+b·ty;
[0104] ty′=-b·tx+a·ty;
[0105] dx′=a·tx+b·ty;
[0106]
[0107] Among them, (tx, ty) is the edge point pixel coordinates (x, y) converted to the relative coordinates relative to the template center point; the edge gradient value (dx, dy) is the edge gradient value corresponding to the edge point pixel coordinates (x, y). The transformed edge point pixel coordinates are (tx′, ty′); the edge gradient value corresponding to the transformed edge point pixel coordinates is (dx′, dy′).
[0108] At the same time, a = scale·cos(angle); b = scale·sin(angle); c = a 2 +b 2 .
[0109] By calculating the initial matching data corresponding to each rotation angle according to the above calculation, the transformed edge data corresponding to each rotation angle can be determined.
[0110] S208 , matching the edge transformation data at each rotation angle with the first pyramid image at the top layer of the image pyramid respectively to determine a candidate matching result.
[0111] In one embodiment, matching the edge transformation data at each rotation angle with the first pyramid image at the top layer of the image pyramid to determine the candidate matching results includes:
[0112] Match each edge transformation data with the first pyramid image of the top layer in the image pyramid, and obtain a matching score of each pixel point in the first pyramid image of the top layer;
[0113] The pixel points corresponding to the matching scores greater than the preset score threshold are taken as candidate matching results.
[0114] The matching score can be considered as the similarity between the edge transformation data and the corresponding pixel point in the first pyramid image. The candidate matching result can be understood as the edge transformation data corresponding to the matching score greater than the preset score threshold.
[0115] In an embodiment, the edge transformation data may be matched with the first pyramid image at the top layer in the image pyramid, and the matching score between the edge transformation data and the corresponding pixel in the first pyramid image may be calculated. In an actual operation, the horizontal gradient value and the vertical gradient value of each pixel in the first pyramid image may be determined, and the matching score of each pixel may be determined by matching the edge transformation data based on the horizontal gradient value and the vertical gradient value. Exemplarily, taking the image coordinates of the pixel in the first pyramid image as (c, r) as an example, the matching score calculation formula at this position is:
[0116]
[0117] Among them, s is the matching score, n is the number of edge points corresponding to the edge transformation data, (dx i ,dy i ) is the edge gradient value of the i-th edge point, is the horizontal gradient value at the pixel position (c+tx, r+ty) in the first pyramid image; is the vertical gradient value at the pixel position (c + tx, r + ty) of the first pyramid image. Traverse to obtain the matching scores at all pixel positions, compare them with a preset score threshold, and use the pixels corresponding to the matching scores greater than the preset score threshold as the matching positions, and determine the matching positions as the candidate matching results. Determine all candidate matching results for all combinations of rotation angles and preset scaling factors. In one embodiment, the candidate matching results may include the rotation angle, the preset scaling factor, and the pixel. In one embodiment, a matrix equal in size to the first pyramid image at the top layer of the image pyramid may be created to save the matching scores of the corresponding pixels.
[0118] In one embodiment, the candidate matching results may be determined in a multi-threaded manner. For the first pyramid image at the top layer, the proportion of the number of potential matching edge transformation data to the total number of pixels is very small, and the computational amount of calculating the matching scores of all pixels according to the matching score confirmation formula is relatively large. The condition for belonging to the candidate matching results may be set as the matching score of the pixel being greater than the preset score threshold scoreThreshold.
[0119]
[0120] Among the number of edge points corresponding to all accumulated edge transformation data, the value of each edge point is between [-1, 1]. Assume that sj is the matching score after accumulating j edge points:
[0121]
[0122] If the values of the remaining edge points are all 1: s(n - j) = 1 - j / n;
[0123] At this time, if sj < scoreThreshold - 1 + j / n, it can be considered that the matching score of this pixel cannot reach the preset score threshold, and it is possible to end the calculation of most positions in advance and reduce the computational amount.
[0124] S209. Extract the edge point pixel coordinates and the edge gradient values of the edge point pixel coordinates of the second pyramid image corresponding to the next layer of the top layer, and determine the initial matching data of the second pyramid image of the next layer based on the candidate matching results and the edge point pixel coordinates and the edge gradient values of the next layer.
[0125] In an embodiment, the edge point pixel coordinates and edge gradient values of the edge point pixel coordinates of the second pyramid image corresponding to the next layer of the predetermined top layer can be extracted, and the edge point pixel coordinates and edge gradient values of the edge point pixel coordinates of the second pyramid image are determined as the initial matching data of the second pyramid image of the next layer. In the actual operation process, when downsampling is performed based on deleting even rows and even columns of the target image, if the edge point pixel coordinates in the candidate matching result of the top layer are (x0, y0), the edge point pixel coordinates of the initial matching data of the second pyramid image of the next layer can be determined as (2×x0, 2×y0), until the first pyramid images of each layer are matched, and the candidate matching results of the first pyramid image of the bottom layer are obtained as the initial matching results.
[0126] S210, continue to perform the operation of determining the transformed edge data based on the initial matching data and matching with the first pyramid image of the next layer to determine the candidate matching result, until the first pyramid images of each layer are matched, and obtain the candidate matching result of the first pyramid image of the bottom layer as the initial matching result.
[0127] In an embodiment, after obtaining the candidate matching results of the top layer, each candidate matching result can be tracked to the matching of the second pyramid image of the next layer until the matching score is less than the preset score threshold or matches to the bottom layer of the preset template image pyramid. In the actual operation process, in the matching process of the second pyramid image except the top layer, the rotation angle can be the following three {angle0-A(i), angle0, angle0+A(i)}, A(i) is the angle rotation step of the current layer, and angle0 is the rotation angle corresponding to the candidate matching result, so as to reduce the calculation amount of the matching. In one embodiment, all the candidate matching results obtained from the top layer can be evenly divided into multiple threads for tracking to reduce the running time.
[0128] S211, extracting sub-pixel edge points of a preset template image pyramid.
[0129] S212, determining a pyramid sub-pixel edge point corresponding to the transformed edge data in each initial matching result of the image pyramid, and respectively determining sub-pixel edge points adjacent to each pyramid sub-pixel edge point as matching sub-pixel edge points.
[0130] In an embodiment, the pyramid sub-pixel edge point corresponding to the transformed edge data in each initial matching result of the image pyramid may be determined in the same manner as the sub-pixel edge point of the preset template image pyramid, and the sub-pixel edge point closest to each pyramid sub-pixel edge point may be determined as the matching sub-pixel edge point.
[0131] S213, determining the sum of squared distances between the pyramid sub-pixel edge points and the matching sub-pixel edge points corresponding to the same rotation angle based on a preset least squares method, and taking the initial matching result corresponding to the minimum sum of squared distances as the target matching result.
[0132] In the embodiment, for the first pyramid image at the bottom layer of the image pyramid, there are multiple initial matching results, and the target matching result closest to the template image can be determined by a preset least square method. In the actual operation process, the method of determining the target matching result is as follows:
[0133]
[0134] Among them, (sx, sy) are the sub-pixel coordinates of the matching sub-pixel edge point of the edge point in the preset template image pyramid, (sc(a), sr(a)) are the sub-pixel coordinates of the pyramid sub-pixel edge point of the edge point corresponding to the transformed edge data under the rotation angle a, and the initial matching result corresponding to the minimum sum of squared distances is taken as the target matching result.
[0135] The embodiment of the present invention downsamples the target image to obtain a first pyramid image of the upper layer of the bottom layer, thereby realizing the construction of an image pyramid; matching is performed in a pyramid manner to improve the processing speed of template matching; the matching degree of the to-be-matched image can be improved by determining initial matching results corresponding to a plurality of rotation angles, so that the template matching is not easily affected by illumination, noise and partial occlusion, and the matching stability is better; the sub-pixel edge points of the pyramid of a preset template image are extracted, the pyramid sub-pixel edge points corresponding to the transformed edge data in each initial matching result of the image pyramid are determined, and the sub-pixel edge points adjacent to each pyramid sub-pixel edge point are respectively determined as matching sub-pixel edge points; the sum of squares of distances between the pyramid sub-pixel edge points corresponding to the same rotation angle and the matching sub-pixel edge points is determined based on a preset least square method, and the initial matching result corresponding to the minimum sum of squares of distances is used as the target matching result, thereby improving the matching accuracy of the template matching.
[0136] Embodiment 3
[0137] Figure 3 1 is a flow chart of another template matching method provided according to Embodiment 3 of the present invention. This embodiment is based on the above embodiment and takes a preset template image pyramid and edge information of the preset template image pyramid saved as a model file as an example to further illustrate a template matching method. Figure 3 As shown, the method includes:
[0138] Step 1: Get the initial template image.
[0139] Step 2: Perform Gaussian filtering on the initial template image to obtain a template image.
[0140] Step 3: Perform pyramid processing on the template image to obtain a preset template image pyramid with a preset number of layers.
[0141] In one embodiment, Figure 4 FIG. 1 is a schematic diagram of a pyramid processing method provided according to Embodiment 3 of the present invention. Figure 4 As shown, a template image of 59×96 is used as an example, and the preset number of layers is 4. After processing, the area of the second pyramid image at the top layer of the preset template image pyramid is 1 / 64 of the area of the bottom layer, i.e., the original template image.
[0142] Step 4: extract edge point information of each layer of the preset template image pyramid and sub-pixel edge points of the bottom layer, wherein the edge point information includes edge point pixel coordinates and edge gradient values corresponding to the edge point pixel coordinates.
[0143] In one embodiment, Figure 5 FIG. 1 is a schematic diagram of an edge image provided according to Embodiment 3 of the present invention. The edge image can be generated in the second pyramid image of each layer according to the edge point information. Figure 5 As shown, the left figure is the second pyramid image of each layer, and the right figure is the edge image of each layer.
[0144] Step 5: Save the preset template image pyramid and edge information of the preset template image pyramid as a model file.
[0145] Step 6: Get the image to be matched.
[0146] Step 7: Perform Gaussian filtering on the image to be matched to obtain the target image.
[0147] Step 8: Perform pyramid processing on the target image to obtain an image pyramid based on a preset number of layers of a preset template image pyramid.
[0148] In one embodiment, Figure 6 FIG. 1 is a schematic diagram of a target image and an image pyramid provided according to Embodiment 3 of the present invention. Figure 6 As shown, the target image is a 4448×4448 image, and the number of layers of the image pyramid is 4.
[0149] Step 9: Match the edge point information of the top layer of the preset template image pyramid with the top layer of the image pyramid to obtain a candidate matching result.
[0150] In one embodiment, Figure 7 FIG. 1 is a schematic diagram of a top-level candidate matching result provided according to Embodiment 3 of the present invention. Figure 7 As shown, the dot position indicates that the matching score of the position is greater than the preset score threshold and is a candidate matching result.
[0151] Step 10: Track and match the candidate matching results corresponding to the corresponding layers of the image pyramid based on the edge point information of other layers of the preset template image pyramid to obtain an initial matching result.
[0152] In one embodiment, Figure 8 FIG. 1 is a schematic diagram of an initial matching result of a bottom layer provided according to Embodiment 3 of the present invention. Figure 8 As shown, the left figure shows the initial matching result, and the right figure is an enlarged view of a certain area.
[0153] Step 11: Optimize the initial matching result by presetting the sub-pixel edge points of the bottom layer of the template image pyramid to obtain the target matching result. The target matching result includes similar matching coordinates, matching rotation angles, and a preset scaling factor.
[0154] In one embodiment, Fig. 9 FIG. 1 is a schematic diagram of a target matching result provided according to Embodiment 3 of the present invention. Fig. 9 As shown, the middle white circle area represents the matching center, and the surrounding white pixels represent the target image pixels corresponding to the template edge.
[0155] The embodiment of the present invention uses the edge information of the preset template image pyramid for matching, and uses processing methods such as the pyramid and matching result refinement to make the template matching more accurate and faster, and less susceptible to the influence of lighting, noise and partial occlusion, thereby making the matching more stable.
[0156] Embodiment 4
[0157] Fig.10 is a schematic diagram of the structure of a template matching device provided according to Embodiment 4 of the present invention. Fig.10 As shown, the device includes: a pyramid generation module 101, an initial matching module 102 and a target matching module 103.
[0158] The pyramid generation module 101 is used to generate an image pyramid according to the image to be matched; wherein the target number of layers of the image pyramid is the same as the preset number of layers of the preset template image pyramid;
[0159] An initial matching module 102 is used to sequentially match the first pyramid image of each layer in the image pyramid with the second pyramid image of the corresponding layer in the preset template image pyramid to obtain an initial matching result;
[0160] The target matching module 103 is used to obtain sub-pixel edge points of a preset template image pyramid, and optimize the initial matching result based on the sub-pixel edge points to obtain a target matching result.
[0161] In the embodiment of the present invention, an image pyramid is generated according to an image to be matched by a pyramid generation module, and an initial matching module sequentially matches a first pyramid image of each layer in the image pyramid with a second pyramid image of a corresponding layer in a preset template image pyramid to obtain an initial matching result. The image pyramid matching method is used to reduce the amount of calculation of template matching, thereby improving the matching efficiency. At the same time, a sub-pixel edge point of a preset template image pyramid is obtained by a target matching module, and an initial matching result is optimized based on the sub-pixel edge point to obtain a target matching result. The sub-pixel edge point is used to refine and adjust the posture, so that the accuracy of the target matching result is higher, thereby ensuring the accuracy of template matching.
[0162] In one embodiment, the template matching device further includes:
[0163] A template image determination module is used to obtain an initial template image, filter and reduce noise on the initial template image to obtain a template image, and use the template image as a second pyramid image at the bottom layer of a preset template image pyramid;
[0164] A downsampling module, used for downsampling the template image to obtain a second pyramid image of the upper layer of the bottom layer;
[0165] A template pyramid determination module is used to downsample the second pyramid image of the previous layer as a new template image until a second pyramid image of a preset number of layers is obtained, thereby obtaining a preset template image pyramid;
[0166] A gradient image construction module, which determines the gradient value of each pixel in the second pyramid image based on a preset horizontal gradient matrix and a preset vertical gradient matrix, and aggregates the gradient values to obtain a gradient image of the second pyramid image;
[0167] A gradient value determination module is used to determine the edge image of the gradient image by using a preset edge detection rule, determine the pixel point coordinates corresponding to each edge image as the edge point pixel coordinates, and determine the gradient value corresponding to the edge point pixel coordinates as the edge gradient value;
[0168] The edge information determination module is used to determine the sub-pixel edge points of the template image based on a preset sub-pixel edge extraction algorithm, and use the edge point pixel coordinates, edge gradient values and sub-pixel edge points as the edge information of the preset template image pyramid.
[0169] In one embodiment, the edge information determination module includes:
[0170] A curved surface generating unit, configured to generate a curved surface by fitting the pixel coordinates of edge points in the gradient image corresponding to the bottom layer second pyramid image through a preset sub-pixel edge extraction algorithm;
[0171] The edge point determination unit is used to determine the gradient adjustment information of the edge point pixel coordinates based on the curved surface, and determine the sub-pixel coordinates of the template image based on the gradient adjustment information and the edge point pixel coordinates, and use the pixel point corresponding to the sub-pixel coordinate as the sub-pixel edge point of the preset template image pyramid.
[0172] In one embodiment, the pyramid generation module 101 includes:
[0173] An image acquisition unit, used to acquire the image to be matched, filter and reduce noise on the image to be matched to obtain a target image, and use the target image as the first pyramid image at the bottom layer of the image pyramid;
[0174] A downsampling unit, used for downsampling the target image to obtain a first pyramid image of an upper layer of the bottom layer;
[0175] The pyramid processing unit is used to downsample the first pyramid image of the previous layer as a new target image until the first pyramid image of the target number of layers is obtained, thereby obtaining an image pyramid.
[0176] In one embodiment, the initial matching module 102 includes:
[0177] An initial data determination unit is used to extract edge point pixel coordinates of a second pyramid image corresponding to a top layer in a preset template image pyramid and edge gradient values corresponding to the edge point pixel coordinates, and use the edge point pixel coordinates and the edge gradient values as initial matching data;
[0178] a step length determining unit, used to determine the number of layers, the preset image height and the preset image width of the second pyramid image of each layer in the preset template image pyramid, and determine the angle rotation step length corresponding to each layer based on the number of layers, the preset image height and the preset image width;
[0179] a rotation angle determining unit, configured to determine a rotation angle of the second pyramid image at the top layer according to a preset initial rotation angle of the second pyramid image at the top layer, a preset rotation range, and an angle rotation step length of the top layer;
[0180] A gradient value determination unit, configured to determine the transformed edge data corresponding to each rotation angle based on the rotation angle, the initial matching data and the preset scaling factor; wherein the transformed edge data includes the transformed edge point pixel coordinates and the edge gradient value corresponding to the transformed edge point pixel coordinates;
[0181] A candidate result determination unit, configured to match the edge transformation data at each rotation angle with the first pyramid image at the top layer of the image pyramid to determine a candidate matching result;
[0182] a data determination unit, configured to extract edge point pixel coordinates and edge gradient values of the edge point pixel coordinates of the second pyramid image corresponding to the next layer of the top layer, and determine initial matching data of the second pyramid image of the next layer based on the candidate matching result and the edge point pixel coordinates and edge gradient values of the next layer;
[0183] The candidate result determination unit is used to continue to perform the operation of determining the transformed edge data based on the initial matching data and matching with the first pyramid image of the next layer to determine the candidate matching result, until the first pyramid images of each layer are matched, and the candidate matching result of the first pyramid image of the bottom layer is obtained as the initial matching result.
[0184] In one embodiment, the candidate result determination unit is specifically configured to:
[0185] Match each edge transformation data with the first pyramid image of the top layer in the image pyramid, and obtain a matching score of each pixel point in the first pyramid image of the top layer;
[0186] The pixel points corresponding to the matching scores greater than the preset score threshold are taken as candidate matching results.
[0187] In one embodiment, the target matching module 103 includes:
[0188] An edge point extraction unit, used for extracting sub-pixel edge points of a preset template image pyramid;
[0189] A matching edge point determination unit is used to determine the pyramid sub-pixel edge point corresponding to the transformed edge data in each initial matching result of the image pyramid, and respectively determine the sub-pixel edge points adjacent to each pyramid sub-pixel edge point as the matching sub-pixel edge point;
[0190] The target matching unit is used to determine the sum of square distances between the pyramid sub-pixel edge points and the matching sub-pixel edge points corresponding to the same rotation angle based on a preset least squares method, and take the initial matching result with the minimum sum of square distances as the target matching result.
[0191] The template matching device provided in the embodiment of the present invention can execute the template matching method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0192] Embodiment 5
[0193] Fig.111 is a schematic diagram of the structure of an electronic device for implementing the template matching method of an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.
[0194] like Fig.11 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0195] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0196] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the template matching method.
[0197] In some embodiments, the template matching method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the template matching method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the template matching method in any other appropriate manner (e.g., by means of firmware).
[0198] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0199] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0200] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0201] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0202] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0203] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0204] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.
[0205] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A template matching method, characterized in that: include: Generate an image pyramid according to the image to be matched; wherein the target number of layers of the image pyramid is the same as the preset number of layers of the preset template image pyramid; Matching the first pyramid image of each layer in the image pyramid with the second pyramid image of the corresponding layer in the preset template image pyramid in sequence to obtain an initial matching result; Sub-pixel edge points of the preset template image pyramid are obtained, and the initial matching result is optimized based on the sub-pixel edge points to obtain a target matching result.
2. The method according to claim 1, characterized in that The process of creating the preset template image pyramid includes: Acquire an initial template image, filter and reduce noise of the initial template image to obtain a template image, and use the template image as a second pyramid image at the bottom layer of a preset template image pyramid; Down-sampling the template image to obtain a second pyramid image of an upper layer of the bottom layer; Down-sampling the second pyramid image of the previous layer as a new template image until a second pyramid image of a preset number of layers is obtained, thereby obtaining a preset template image pyramid; Determining a gradient value of each pixel in the second pyramid image based on a preset horizontal gradient matrix and a preset vertical gradient matrix, and aggregating each of the gradient values to obtain a gradient image of the second pyramid image; Determine the edge image of the gradient image by using a preset edge detection rule, determine the pixel point coordinates corresponding to each edge image as the edge point pixel coordinates, and determine the gradient value corresponding to the edge point pixel coordinates as the edge gradient value; The sub-pixel edge points of the template image are determined based on a preset sub-pixel edge extraction algorithm, and the edge point pixel coordinates, the edge gradient value and the sub-pixel edge point are used as edge information of the preset template image pyramid.
3. The method according to claim 2, characterized in that The determining the sub-pixel edge points of the template image based on a preset sub-pixel edge extraction algorithm comprises: A surface is generated by fitting the pixel coordinates of edge points in the gradient image corresponding to the bottom second pyramid image through a preset sub-pixel edge extraction algorithm; Gradient adjustment information of the edge point pixel coordinates is determined based on the curved surface, and sub-pixel coordinates of the template image are determined based on the gradient adjustment information and the edge point pixel coordinates, and the pixel points corresponding to the sub-pixel coordinates are used as sub-pixel edge points of a preset template image pyramid.
4. The method according to claim 1, characterized in that: The step of generating an image pyramid according to the image to be matched comprises: Acquire an image to be matched, filter and reduce noise on the image to be matched to obtain a target image, and use the target image as the first pyramid image at the bottom layer of the image pyramid; Down-sampling the target image to obtain a first pyramid image of an upper layer of the bottom layer; The first pyramid image of the previous layer is taken as a new target image for downsampling until the first pyramid image of the target layer is obtained, thereby obtaining an image pyramid.
5. The method according to claim 1, characterized in that The step of sequentially matching the first pyramid image of each layer in the image pyramid with the second pyramid image of the corresponding layer in the preset template image pyramid to obtain an initial matching result includes: Extracting edge point pixel coordinates of a second pyramid image corresponding to a top layer in the preset template image pyramid and edge gradient values corresponding to the edge point pixel coordinates, and using the edge point pixel coordinates and the edge gradient values as initial matching data; Determine the layer number, preset image height and preset image width of the second pyramid image of each layer in the preset template image pyramid, and determine the angle rotation step corresponding to each layer based on the layer number, the preset image height and the preset image width; Determine a rotation angle of the second pyramid image of the top layer according to a preset initial rotation angle of the second pyramid image of the top layer, a preset rotation range, and an angle rotation step length of the top layer; Determine the transformed edge data corresponding to each of the rotation angles based on the rotation angle, the initial matching data and a preset scaling factor; wherein the transformed edge data includes the transformed edge point pixel coordinates and the edge gradient value corresponding to the transformed edge point pixel coordinates; Matching the edge transformation data at each rotation angle with the first pyramid image at the top layer of the image pyramid respectively to determine candidate matching results; Extracting edge point pixel coordinates of the second pyramid image corresponding to the next layer of the top layer and edge gradient values of the edge point pixel coordinates, and determining initial matching data of the second pyramid image of the next layer based on the candidate matching results and the edge point pixel coordinates and edge gradient values of the next layer; The operation of determining the transformed edge data based on the initial matching data and matching with the first pyramid image of the next layer to determine the candidate matching result is continued until the first pyramid images of each layer are matched, and the candidate matching result of the first pyramid image of the bottom layer is obtained as the initial matching result.
6. The method according to claim 5, characterized in that The edge transformation data according to each rotation angle is matched with the first pyramid image of the top layer in the image pyramid respectively to determine the candidate matching result, including: Matching each of the edge transformation data with the first pyramid image of the top layer in the image pyramid, respectively, to obtain a matching score of each pixel point in the first pyramid image of the top layer; The pixel points corresponding to the matching scores that are greater than the preset score threshold are taken as candidate matching results.
7. The method according to claim 1, characterized in that The obtaining of sub-pixel edge points of the preset template image pyramid and optimizing the initial matching result based on the sub-pixel edge points to obtain a target matching result includes: Extract sub-pixel edge points of a preset template image pyramid; Determine a pyramid sub-pixel edge point corresponding to the transformed edge data in each initial matching result of the image pyramid, and respectively determine sub-pixel edge points adjacent to each pyramid sub-pixel edge point as matching sub-pixel edge points; The sum of squared distances between the pyramid sub-pixel edge points and the matching sub-pixel edge points corresponding to the same rotation angle is determined based on a preset least squares method, and an initial matching result corresponding to the minimum sum of squared distances is taken as a target matching result.
8. A template matching device, characterized in that: include: A pyramid generation module, used to generate an image pyramid according to the image to be matched; wherein the target number of layers of the image pyramid is the same as the preset number of layers of the preset template image pyramid; An initial matching module, used for sequentially matching the first pyramid image of each layer in the image pyramid with the second pyramid image of the corresponding layer in the preset template image pyramid to obtain an initial matching result; The target matching module is used to obtain the sub-pixel edge points of the preset template image pyramid, and optimize the initial matching results based on the sub-pixel edge points to obtain the target matching results.
9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the template matching method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the template matching method according to any one of claims 1 to 7 when executed.
Citation Information
Cited By
Image matching method and device based on multilayer inspection mechanism and computer equipment
CN119762822A
Image Matching Method, Device and Computer Equipment Based on Multi-Layer Verification Mechanism
CN119762822B
Semiconductor bare chip high-speed sorting and positioning method based on CCD image
CN119831998A
