Template matching method, template matching device, and storage medium

Abstract

The present invention provides a template matching method, comprising: obtaining a template image; generating at least one model image by applying transformation parameters to the template image; extracting edge points from each of the at least one model image; for each model image, filtering the extracted edge points by a first filter based on the pixel gradient magnitude to obtain a first pixel subset corresponding to each model image; filtering the first pixel subset by a second filter based on the pixel gradient direction and the pixel distance to obtain a second pixel subset corresponding to each model image; obtaining a first matching degree of each model image by calculating the similarity between each pixel in the second pixel subset and the corresponding pixel in the image region to be detected; and determining whether each model image in the at least one model image matches the image region to be detected based on the first matching degree.

Description

Technical Field

[0001] Generally, the present invention relates to the field of image processing, and more particularly to a template matching method, a template matching device, and a computer-readable storage medium. Background Art

[0002] In the field of image processing, template matching is usually used to detect parameters such as the position and pose of a target image in an actual scene, and has a certain robustness to scenes such as occlusion, clutter, contrast inversion, and non-linear illumination conditions. Therefore, target recognition and positioning based on template matching can be widely applied to practical applications such as automated inspection and defect detection. For example, by matching the pixel points between a template image and an image to be detected (such as an image containing defects), numerical values such as translation, scaling, and rotation between the images can be found, so as to obtain the parameters of the target image using the parameters of the known template image. Therefore, how to select the pixel points for matching in the template image is crucial for template matching.

[0003] However, in the existing template matching methods, due to improper selection of the pixel points for matching in the template image, problems such as low matching efficiency caused by too many selected pixel points or low matching accuracy caused by missing effective pixel points may occur. Therefore, an improved template matching method needs to be proposed to obtain a trade-off between efficiency and accuracy through a reasonable pixel selection strategy and matching method. Summary of the Invention

[0004] In view of this, according to one aspect of the present invention, there is provided a template matching method, including: obtaining a template image; generating at least one model image by applying transformation parameters to the template image; extracting edge points from each of the at least one model image; for each model image, filtering the extracted edge points through a first filter based on the pixel gradient magnitude to obtain a first pixel subset corresponding to each model image; filtering the first pixel subset through a second filter based on the pixel gradient direction and pixel distance to obtain a second pixel subset corresponding to each model image; calculating the similarity between each pixel in the second pixel subset and the corresponding pixel in the image region to be detected to obtain a first matching degree of each model image; and based on the first matching degree, determining whether each of the at least one model image matches the image region to be detected.

[0005] In addition, according to an embodiment of the present invention, determining whether each model image in at least one model image matches a region of an image to be detected includes: comparing a first matching degree with a threshold value; wherein, if the first matching degree is not less than the threshold value, determining that a specific model image associated with the first matching degree not less than the threshold value positively matches the region of the image to be detected; and if the first matching degree is less than the threshold value, determining that a specific model image associated with the first matching degree less than the threshold value does not match the region of the image to be detected.

[0006] In addition, according to an embodiment of the present invention, after determining that a specific model image positively matches a region of an image to be detected, it further includes: extracting edge points from the matched region of the image to be detected, and performing a first filtering and a second filtering on the edge points of the matched region of the image to be detected to obtain a third subset of pixels, calculating a similarity between each pixel in the third subset of pixels and the corresponding pixel in the matched specific model image to obtain a second matching degree of the specific model image; and determining whether the specific model image matches the region of the image to be detected based on both the first matching degree and the second matching degree.

[0007] In addition, according to an embodiment of the present invention, the template matching method further includes: determining whether the specific model image matches the region of the image to be detected based on the minimum of both the first matching degree and the second matching degree.

[0008] In addition, according to an embodiment of the present invention, the template matching method further includes: calculating the sharpness of each pixel in a second subset of pixels, where the sharpness is defined as a value associated with the angle formed by a pixel and its adjacent pixels; assigning a weight value to each pixel in the second subset of pixels based on the calculated sharpness of each pixel; and calculating the first matching degree by weighting with the weight value.

[0009] In addition, according to an embodiment of the present invention, the template matching method further includes: calculating the sharpness of each pixel in a third subset of pixels, assigning a weight value to each pixel in the third subset of pixels based on the calculated sharpness of each pixel; and calculating the second matching degree by weighting with the weight value.

[0010] In addition, according to an embodiment of the present invention, the similarity between pixels is calculated by: calculating the difference between the gradient direction of a pixel and the gradient direction of the corresponding pixel; and using the difference between the gradient directions to characterize the similarity between pixels.

[0011] According to another aspect of the present invention, there is provided a template matching device, comprising: a generating unit configured to obtain a template image and generate at least one model image by performing pose transformation on the template image; an extracting unit configured to extract edge points of each of the at least one model image; a filtering unit configured to, for each model image, filter a first pixel subset through a second filtering based on pixel gradient direction and pixel distance to obtain a second pixel subset corresponding to each model image, and filter the first pixel subset through the second filtering based on pixel gradient direction and pixel distance to obtain a second pixel subset; and a matching unit configured to calculate a first matching degree between each pixel in the second pixel subset and a corresponding pixel in the image region to be detected, and based on the calculated first matching degree, determine whether each of the at least one model image matches the image region to be detected.

[0012] According to still another aspect of the present invention, there is provided a template matching device, comprising a processor and a memory, in which computer program instructions are stored, wherein when the computer program instructions are run by the processor, the processor is caused to perform the following steps: obtain a template image; generate at least one model image by applying transformation parameters to the template image; extract edge points from each of the at least one model image; for each model image, filter the extracted edge points through a first filtering based on pixel gradient magnitude to obtain a first pixel subset corresponding to each model image; filter the first pixel subset through a second filtering based on pixel gradient direction and pixel distance to obtain a second pixel subset corresponding to each model image; obtain a first matching degree of each model image by calculating a similarity between each pixel in the second pixel subset and a corresponding pixel in the image region to be detected; and based on the first matching degree, determine whether each of the at least one model image matches the image region to be detected.

[0013] According to another aspect of the present invention, there is provided a computer-readable storage medium having computer program instructions stored thereon, wherein when the computer program instructions are executed by a processor, the following steps are implemented: obtaining a template image; generating at least one model image by applying transformation parameters to the template image; extracting edge points from each of the at least one model image; for each model image, filtering the extracted edge points by a first filter based on the pixel gradient magnitude to obtain a first pixel subset corresponding to each model image; filtering the first pixel subset by a second filter based on the pixel gradient direction and the pixel distance to obtain a second pixel subset corresponding to each model image; calculating the similarity between each pixel in the second pixel subset and the corresponding pixel in the image region to be detected to obtain a first matching degree for each model image; and based on the first matching degree, determining whether each model image in the at least one model image matches the image region to be detected.

[0014] According to the above template matching method, template matching device and storage medium of the present invention, through a reasonable matching pixel selection strategy, without increasing additional computational complexity, pixels with strong description ability are selected for template matching, so that high-efficiency and high-accuracy template matching performance can be obtained. Moreover, a sharpness-based weighting model is also used to further enhance the description ability of the matching model. In addition, two-way matching measurement is also used to further reduce the probability of false matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] By describing the embodiments of the present invention in detail in conjunction with the following drawings, the above and other objects, features, and advantages of the present invention will become clearer. In the drawings, the same reference numerals generally represent the same components or steps, where:

[0016] Figure 1 is a schematic diagram showing an example of the template matching method;

[0017] Figure 2 (a) to (c) in are schematic diagrams showing an example of extracting model points;

[0018] Figure 3 is a flowchart showing the template matching method according to an embodiment of the present invention;

[0019] Figure 4 is a schematic diagram showing the definition of sharpness according to an embodiment of the present invention;

[0020] Figure 5 is a block diagram showing an example of the template matching device according to an embodiment of the present invention;

[0021] Figure 6 is a block diagram showing another example of the template matching device according to an embodiment of the present invention.

[0022] It should be understood that these drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. Detailed implementation manners

[0023] Embodiments of an image processing method, a template matching device, and a computer-readable recording medium according to the present invention will be described below with reference to the above drawings. It should be understood that based on the embodiments described in the present invention, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention. Moreover, the embodiments described herein are only a part of the embodiments of the present invention, rather than all the embodiments of the present invention. These embodiments are merely illustrative and exemplary, and thus should not be construed as limiting the scope of the present invention. In addition, in order to make the specification more clear and concise, detailed descriptions of functions and structures well-known in the art will be omitted, and repeated explanations of steps and elements will also be omitted.

[0024] First, in combination with Figure 1 to describe an example of the template matching method. As Figure 1 shown, the figure on the left can be used as an example of the template image, and the image on the right is the image to be detected. In this image to be detected, there are multiple images of building blocks that can be used as detection targets, and the building blocks in the image are presented in different poses in the image. Therefore, in order to detect the building blocks in a specific pose from the image to be detected, the following template matching method can be adopted.

[0025] For example, if the laterally placed building blocks are used as the detection target, considering that the shape presented by the laterally placed building blocks in the image to be detected is a rectangle, the rectangular graph in the left example can be used as the template image for matching. It can be understood that if building blocks in other postures are used as the detection target, other suitable shapes can be selected as the template image. In addition, considering that the building blocks in the image to be detected present different angles or sizes relative to the horizontal direction, a transformation parameter or a set of transformation parameters also needs to be applied to the template image to perform operations such as rotation transformation and / or scaling on it, and then the template image with different rotation angles and / or sizes (which can also be referred to as "model" or "model image" in the present invention) traverses the entire image to be detected or a specified part of the image to be detected. For example, it can traverse the entire image or a part of the area from left to right and from top to bottom, and perform a similarity evaluation at each position, so as to match the images of each building block in the image to be detected. For the matching method, the present invention can be implemented by using appropriate existing or future methods in the field such as feature point matching, NCC algorithm, etc., and calculate the similarity for the corresponding parameters according to the adopted matching method to measure the matching degree of the model image, which will not be elaborated here.

[0026] After the above traversal process, as Figure 1 shown, a total of four matching results are obtained, which are shown as four solid boxes in the image to be detected on the right. However, it can be seen that one of the matching results is incorrect, that is, a flat-laid building block will be recognized as a laterally placed building block, which will have an adverse impact on application scenarios such as automated inspection. The reason for this incorrect matching may be that the traditional template matching method is insensitive to new structures in the target image. For example, although the area covered by the rectangular template can match a part of the target image, the area of the target image outside the rectangular template actually does not match the template, resulting in the above false detection. Therefore, an improved matching method needs to be proposed to reduce the false matching rate.

[0027] In addition, since the traditional method of selecting model points (i.e., the pixel points used for template matching in the model) is prone to losing detailed information. For example, only extracting model points according to distance may not be able to well describe the internal details of the model, and may also lead to insensitivity to the internal details of the model image during the template matching process, resulting in false detection. The processing of selecting model points will be described below in combination with Figure 2 an illustration.

[0028] Figure 2 An example of extracting model points is shown. Before performing template matching, it is necessary to first extract / select the pixel points for matching from the model image to perform template matching with the corresponding pixels in the image to be detected. To ensure the accuracy of the matching, the extracted model points preferably should have a strong description ability for the model. AsFigure 2 As shown, the template image (or model image) in the figure consists of two circles. Conventional model selection strategies usually select model points based on a fixed distance of pixels, which may result in the loss of model points at positions close to the edge. As shown in Figure 2 (a) of [Figure], since the distance between the edges of the inner circle and the outer circle is close, no model points are selected at the edge of the inner circle close to the outer circle, resulting in only two model points remaining on the edge of the inner circle. However, it will be very difficult to describe the inner circle as a circle based on only these two model points, which will be disadvantageous for subsequent template matching.

[0029] On the other hand, to ensure the efficiency of matching, the number of extracted model points should not be too large. Therefore, an improved model point selection strategy needs to be proposed to obtain a balance between the efficiency and accuracy of matching.

[0030] Therefore, at least to solve one of the above problems, the present invention proposes an improved template matching method, which will be described in detail below in conjunction with Figure 3 for detailed description.

[0031] Figure 3 is a flowchart showing the template matching method according to an embodiment of the present invention. As shown in Figure 3 shown, the process 300 of the proposed template matching method may include the following steps:

[0032] At S301, start the process and obtain a template image.

[0033] According to an embodiment of the present invention, the way to obtain a template image may include but is not limited to the following: directly obtaining by photographing or recording an object with a device having an image capture function. Alternatively, it can also be obtained by receiving from other devices (such as a server or a storage device) via wired or wireless means.

[0034] In addition, to improve the performance of template matching, the template image preferably includes at least features that can describe the target object, such as the contour of the target object or a part thereof, the general shape, color, etc. For example, as described in the embodiment of Figure 1 , when the target object is a laterally placed building block, a rectangular template image will be used. It can be understood that when the target object is a flat building block, a square template image or a template image with a similar pattern inside can be used. That is, depending on the specific application scenario, a suitable template image can be obtained for subsequent template matching. Thereafter, the process proceeds to step S302.

[0035] At S302, generate at least one model image by applying transformation parameters to the template image.

[0036] According to an embodiment of the present invention, the template image can be transformed in terms of size, rotation angle, position, etc. That is, the transformation parameters can include at least one of the following: scaling parameters, rotation parameters, etc., and depending on whether the application scenario is 2D or 3D, the rotation can correspondingly be planar or 3D. In addition, considering that the image to be detected itself may have distortions (such as wide-angle distortion, etc.) resulting in partial image distortion, therefore, the transformation parameters can also include deformation parameters or similar parameters. It can be understood that the transformation parameters can also be set to appropriate values without transforming the template image, that is, directly using the untransformed template image for template matching. By transforming the template image to generate at least one transformed template image (i.e., the model image), it is possible to match the target object in different poses at various positions in the image to be detected. Thereafter, the process proceeds to step S303.

[0037] At S303, edge points are extracted from each of the at least one model image.

[0038] Specifically, the edge points of each model image generated in step S302 will be extracted first for subsequent matching with the corresponding pixels in the image to be detected. For the extraction of edge points of an image, there are many known methods in the art. For example, the edge points of the model image can be extracted by clustering. More specifically, the edge points of the image can be obtained by clustering according to distance, such as using the Canny operator, K-means clustering, or other appropriate existing or future methods in the art, which will not be elaborated here.

[0039] The edge points extracted from each model image usually already have a certain ability to describe the model because the pixels located at the edges of the image often contain more features than those at other positions. Therefore, through the edge point extraction process at S303, most of the pixels with weak description ability in the image can be significantly removed, and only the edge points are retained for subsequent template matching, thereby improving the efficiency and accuracy of the matching. Thereafter, the process proceeds to step S303.

[0040] At S304, for each model image, the extracted edge points are filtered through a first filter based on the pixel gradient magnitude to obtain a first pixel subset corresponding to each model image.

[0041] In the art, the pixel gradient represents the degree of change (i.e., the pixel gradient magnitude) and the direction of change (i.e., the pixel gradient direction) of the gray value of the pixels in the image. The pixel gradient magnitude and pixel gradient direction of the image can be calculated by conventional gradient operators such as the Sobel operator, Krisch operator, etc., which will not be elaborated here.

[0042] Specifically, the edge points extracted at step S303 are subjected to a first filtering, where the first filtering is a filtering of pixel points based on the pixel gradient magnitude, that is, the edge points are filtered based on the gradient magnitude of the edge points. After the first filtering of the edge points of the model image, the edge points with pixel gradient magnitudes within a reasonable range are retained, and a first pixel subset corresponding to the model image is obtained as a candidate for model points. For example, considering that pixel points with little change in pixel gray values often do not contain much valid information, in order to extract edge points with pixel gradient magnitudes within a reasonable range, by setting an appropriate pixel gradient magnitude threshold, the edge points below the threshold can be removed from the candidates for model points. Alternatively, a range interval of pixel gradient magnitudes can be set, and only the edge points whose pixel gradient magnitude values fall within this interval are extracted. Thereafter, the process proceeds to step S305.

[0043] At S305, the first pixel subset is filtered through a second filtering based on the pixel gradient direction and pixel distance to obtain a second pixel subset corresponding to each model image.

[0044] Specifically, the first pixel subset filtered at step S304 is subjected to a second filtering, that is, a second-round filtering process is performed on the edge points of the model image. The second filtering is a filtering of pixel points based on both the pixel gradient direction and pixel distance, and finally the second pixel subset corresponding to the model image can be used as model points for subsequent matching with the corresponding pixels in the image to be detected.

[0045] Return reference Figure 2 , and further illustrate the first filtering and second filtering processes performed at steps S303 and S304 respectively.

[0046] As described above, the commonly used method in the art is to only filter multiple model points that are close at a fixed distance to remove redundant model points or noise, such as the above example described in (a) of Figure 2 , but this method will also cause the problem of losing key model points and being unable to completely describe the model.

[0047] According to the embodiments of the present invention, the edge points of the model image are first subjected to a first filtering based on the pixel gradient magnitude (S304), and the part of the edge points with gradient intensities within a reasonable range will be retained, and the situation of losing key model points due to only selecting model points based on a fixed distance will not occur, as shown in (b) of Figure 2 , after the first filtering, enough edge points for describing the model are also retained on the inner circle edge.

[0048] On this basis, a second filtering (S305) based on the pixel gradient direction and pixel distance is further performed on the edge points after the first filtering. Specifically, for the curved part of the model edge, the change direction of the pixel gray value often changes greatly. Therefore, the edge points with a large difference in gradient direction change will have a stronger ability to describe the graphic contour. Therefore, retaining these edge points as model points will significantly improve the accuracy of template matching. On the contrary, the edge points with a small difference in gradient direction change contribute less to describing the model, and these edge points will be removed through the second filtering, thereby improving the efficiency while ensuring the matching accuracy. As shown in Figure 2 (c) of, after the second filtering, the edge points with similar pixel gradient directions in the inner circle are filtered; on the other hand, for the straight part of the edge, redundant edge points can be removed based on the pixel distance.

[0049] It can be seen that according to the embodiments of the present invention, first, a preliminary filtering is performed on the extracted model points through the first filtering based on the pixel gradient magnitude, and while filtering, it is ensured that the key model points will not be lost. On this basis, a second filtering is further performed based on both the pixel gradient direction and distance, and only the pixel points with strong description ability in the corresponding situation are retained as model points, which improves the existing model point selection strategy and significantly improves the efficiency and accuracy of the template matching method. Thereafter, the process proceeds to step S306.

[0050] At S306, the first matching degree of each model image is obtained by calculating the similarity between each pixel in the second pixel subset and the corresponding pixel in the image region to be detected.

[0051] Specifically, after generating the model image and selecting the model points for matching through the above steps, each model image will be matched with the image to be detected. As shown in the example described in Figure 1 The model image will traverse each position of the entire image to be detected. During the matching process, the similarity between the model points of each model image and the pixels (i.e., corresponding pixels) in the image region to be detected at the corresponding position will be calculated. The calculated similarity between the pixels can be used to measure the matching degree between the corresponding model image and the image region to be detected. For example, the sum of the calculated similarities between the pixels is used as the matching degree of the model image.

[0052] It can be understood that there can be various ways to represent the similarity between pixels. Preferably, according to an embodiment of the present invention, the similarity between pixels can be calculated in the following manner: calculating the difference between the gradient direction of a pixel and the gradient direction of the corresponding pixel; and using the difference between the gradient directions to represent the similarity between pixels. For example, the cosine value of the difference between the gradient direction of a model point and the gradient direction of the corresponding pixel can be calculated as the similarity, that is, the similarity can be calculated by the following formula (1):

[0053]

[0054] wherein, represents the gradient direction at the model point r in the model image and represents the gradient direction at the corresponding pixel t in the image to be detected . It can be seen from this formula that the greater the difference between the gradient directions of the pixels, the smaller the similarity; conversely, the smaller the difference between the gradient directions of the pixels, the greater the similarity. Similarly, it can be understood that the above example of using the cosine value of the gradient direction angle to represent the similarity between pixels is only a preferred method. In addition, those skilled in the art can expect various ways to use the difference between the gradient directions to represent the similarity between pixels, which will not be elaborated here.

[0055] In addition, according to an embodiment of the present invention, weights can also be assigned to the model points based on specific parameters of the pixels, and the weighted similarity can be calculated as the matching degree of the model image. For example, considering that there may be a large number of matching model points on the flat edges of two non-matching images, which cannot effectively distinguish the target, while the edge points at the corner points or inflection points of the model are actually more recognizable. Therefore, when calculating the similarity, greater weights need to be assigned to these edge points located at the corner points.

[0056] Preferably, the similarity can be weighted based on the sharpness of the model points. Here, first, the definition of sharpness is described in combination with Figure 4 . As Figure 4 shown, for any pixel point t, two adjacent pixel points t - 1 and t + 1 can be obtained based on the above clustering method. Then, the sharpness of the pixel point t can be defined according to the following formula (2):

[0057]

[0058] Among them, α represents the included angle (in degrees) formed by the line segment between pixel points t - 1 and t and the line segment between pixel points t and t + 1. From this formula, it can be seen that the smaller the sharpness of a specific pixel point, the smoother the position where the point is located. Therefore, through the above definition of sharpness, the smoothness of the line segment where a specific pixel point is located can be described.

[0059] On this basis, the sharpness of pixels in the model image can be calculated. Preferably, the sharpness of each pixel in the extracted model points can be calculated only, and corresponding weights can be assigned to each model point based on the calculated sharpness of each pixel. For example, it is set that the weight of the model point is proportional to the sharpness, and then the matching degree of the model is calculated by weighted calculation, that is, the matching degree can be calculated by the following formula (3):

[0060]

[0061] where w i represents the weight assigned to each model point.

[0062] In this way, the description ability of model points for model details will be further improved, highlighting the role of corner points in the edge during the matching process, thereby improving the efficiency and accuracy of matching.

[0063] Returning to process 300, it will proceed to step S307.

[0064] At S307, based on the first matching degree, it is determined whether each model image in at least one model image matches the image region to be detected.

[0065] Specifically, as described above, the similarity calculated in step S306 can be used to measure the matching degree between the corresponding model image and the image region to be detected. On this basis, for example, the first matching degree can be compared with a threshold value; among them, if the first matching degree is not less than the threshold value, it is determined that the specific model image associated with the first matching degree not less than the threshold value matches the image region to be detected; and if the first matching degree is less than the threshold value, it is determined that the specific model image associated with the first matching degree less than the threshold value does not match the image region to be detected.

[0066] In addition, according to an embodiment of the present invention, after determining that a specific model image matches the image region to be detected (i.e., forward matching), reverse matching is also performed, which specifically includes: first, edge points are extracted from the matching image region to be detected, and the edge points of the matching image region to be detected are subjected to the first filtering and the second filtering as described above to obtain a third pixel subset, and the second matching degree of the specific model image is obtained by calculating the similarity between each pixel in the third pixel subset and the corresponding pixel in the matching specific model image.

[0067] Among them, in the process of reverse matching, the extraction of edge points and pixel gradients can be carried out in a similar way to forward matching, which will not be elaborated here. And to improve the efficiency of the algorithm, considering that the pixel gradients of the current specific model image have been calculated in the forward matching, they can be directly used in the reverse matching process to avoid repeated calculations. In addition, considering that the specific model image to be matched has undergone transformations such as size and rotation through the application of transformation parameters in the forward matching process, in the reverse matching process, there is no need to transform the region of the image to be detected to be matched to generate the model image, and it can be directly matched with the specific model image.

[0068] After obtaining the second matching degree of the specific model image through reverse matching, based on both the first matching degree and the second matching degree obtained through forward matching, it is determined whether the specific model image matches the region of the image to be detected. For example, the sum or weighted sum of the first matching degree and the second matching degree can be used as the matching degree of the specific model image. Alternatively, the minimum value of the first matching degree and the second matching degree can also be taken as the matching degree of the specific model image to determine whether the specific model image matches the region of the image to be detected, that is, it is determined that the two match only when the specific model image can match both forward and backward with the region of the image to be detected. Through the above two-way matching method, the problem that Figure 1 the template matching in the described example is not sensitive to the new structure in the target image can be solved. Therefore, the existing matching method is improved, and the false detection rate of template matching is further reduced.

[0069] In addition, according to an embodiment of the present invention, after determining whether the specific model image matches the region of the image to be detected in step S307, the matching judgment can also be performed on other model images in at least one model image, so as to determine whether each model image matches the region of the image to be detected. In this way, after performing template matching on each model image and the region of the image to be detected, the matching degree of each model image will be obtained. Therefore, one or more specific model images with the desired matching degree can be selected as the matching result. For example, specific model images with a matching degree higher than a specific threshold or within a specific interval, or several specific model images with the highest matching degree can be selected as the matching result.

[0070] In addition, according to an embodiment of the present invention, after obtaining the matching model image, the obtained matching model image can be further processed as expected according to the specific application scenario. For example, since the parameters of the template image and each model image (such as size, pose, position, etc.) can be known in the process of obtaining the model image, the corresponding parameters of the region of the image to be detected that matches can be obtained based on the parameters of the model image to identify and detect the target object in the image to be detected.

[0071] As can be seen, through the above embodiments of the present invention, the existing matching method is improved. The template matching method proposed by the present invention can not only significantly improve the accuracy of template matching through a reasonable model point selection strategy and matching method, but also improve the overall efficiency of the algorithm, so as to have better performance in specific applications.

[0072] Next, reference will be made to Figure 5 to describe the template matching device according to an embodiment of the present invention.

[0073] Figure 5 FIG. shows a block diagram of a template matching device 500 according to an embodiment of the present invention. As Figure 5 shown, the template matching device 500 includes: a generating unit 501, an extracting unit 502, a filtering unit 503, and a matching unit 504. It should be understood that the shown structure is merely exemplary and not restrictive, and in addition to these units, the template matching device 1100 may further include other components. Since these other components are less relevant to the content of the embodiments of the present invention or are well-known in the art, the illustration and specific description of these components will be omitted.

[0074] In addition, since the specific details of the following processing performed by the template matching device 500 according to an embodiment of the present invention are substantially the same as the details described above with reference to Figures 1 to 5 for the sake of brevity, the description of the same processing will be omitted herein. The following will introduce each unit or component in the template matching device 500 one by one.

[0075] The generating unit 501 is configured to obtain a template image and generate at least one model image by performing pose transformation on the template image. The specific processing performed by the generating unit 501 is consistent with the corresponding content of steps S301 and S302 described above.

[0076] Specifically, the generating unit 501 may include an image capture module such as a camera, a camera, or a module for receiving images to obtain a template image, where such a module may also have the function of performing pose transformation on the template image, or the generating unit 501 further includes another module having this function.

[0077] In addition, the generating unit 501 can be physically separated from other units in the template matching device 500, and the generating unit 501 transmits the image to other units or modules in the template matching device 500 via wired or wireless means. Alternatively, the generating unit 501 can be physically located in the same position as other modules or components in the template matching device 500, even inside the same housing, and other units or modules in the template matching device 500 receive the image transmitted by the generating unit 501 via an internal bus.

[0078] The extraction unit 502 is configured to extract edge points of each model image in at least one model image. The specific processing performed by the extraction unit 502 is consistent with the corresponding content of step S303 described above.

[0079] Specifically, the extraction unit 502 will first extract edge points of each generated model image for subsequent matching with corresponding pixels in the image to be detected. For the extraction of edge points of an image, as described above, the extraction of edge points can be achieved by appropriate existing or future methods in the art, which will not be elaborated here.

[0080] The filtering unit 503 is configured to filter the first pixel subset for each model image through a second filtering based on pixel gradient direction and pixel distance to obtain a second pixel subset corresponding to each model image, and filter the first pixel subset through a second filtering based on pixel gradient direction and pixel distance to obtain a second pixel subset. The specific processing performed by the filtering unit 503 is consistent with the corresponding content of steps S304 and S305 described above.

[0081] Specifically, the filtering unit 503 will perform a first filtering on the edge points extracted from the model image, where the first filtering is a filtering of pixel points based on pixel gradient magnitude. After performing the first filtering on the edge points of the model image, the edge points with pixel gradient magnitude within a reasonable range are retained, and a first pixel subset corresponding to the model image is obtained as a candidate for model points. Moreover, the filtering unit 503 also performs a second filtering on the first pixel subset after the first filtering, that is, a second-round filtering process on the edge points of the model image. Among them, the second filtering is a filtering of pixel points based on both pixel gradient direction and pixel distance, and finally a second pixel subset corresponding to the model image can be used as model points for subsequent matching with corresponding pixels in the image to be detected. As described above, the acquisition of pixel gradients can be achieved by appropriate existing or future methods in the art, which will not be elaborated here.

[0082] The matching unit 504 is configured to calculate a first matching degree between each pixel in the second pixel subset and the corresponding pixel in the image region to be detected, and based on the calculated first matching degree, determine whether each model image in at least one model image matches the image region to be detected. The specific processing performed by the matching unit 504 is consistent with the corresponding content of steps S306 and S307 described above.

[0083] Specifically, the matching unit 504 will match each model image with the image to be detected. The matching process can be similar to the example described in combination with Figure 1 As described above, the similarity between pixels calculated during the matching process can be used to measure the matching degree between the corresponding model image and the image region to be detected. As mentioned above, the similarity between pixels can have various representation methods, and weights can also be assigned to model points based on specific parameters of the pixels, and the weighted similarity is calculated as the matching degree of the model image, thereby improving the efficiency and accuracy of the matching. On this basis, the matching unit 504 can compare the first matching degree with a threshold; wherein, if the first matching degree is not less than the threshold, it is determined that the specific model image associated with the first matching degree not less than the threshold matches the image region to be detected; and if the first matching degree is less than the threshold, it is determined that the specific model image associated with the first matching degree less than the threshold does not match the image region to be detected.

[0084] In addition, according to an embodiment of the present invention, after the matching unit 504 determines that a specific model image matches the image region to be detected (i.e., forward matching), reverse matching is also performed, specifically including: first extracting edge points from the matched image region to be detected, and performing the first filtering and the second filtering on the edge points of the matched image region to be detected as described above to obtain a third pixel subset, and calculating the similarity between each pixel in the third pixel subset and the corresponding pixel in the matched specific model image to obtain a second matching degree of the specific model image. The processing of the reverse matching is consistent with the embodiment described above and will not be elaborated here.

[0085] In addition, according to an embodiment of the present invention, after the matching unit 504 determines whether a specific model image matches the image region to be detected, it can also perform the matching judgment on other model images in at least one model image, so as to determine whether each model image matches the image region to be detected, and select one or more specific model images with an expected matching degree as the matching result, as described above.

[0086] In addition, according to an embodiment of the present invention, after the matching unit 504 obtains the matched model image, an appropriate unit or module in the template matching device can use the obtained matched model image to further perform expected processing according to the specific application scenario, as described above.

[0087] It should be understood that the processing performed by each of the above units or modules in the template matching device is not limited to the above examples. For example, one or more of these units or modules may have the functions of other units or modules and perform the processing of other units or modules, or these units or modules may be integrated into one component.

[0088] Next, reference will be made to Figure 6 to describe the template matching device according to an embodiment of the present invention.

[0089] Figure 6 FIG. shows a block diagram of a template matching device 600 according to an embodiment of the present invention. As Figure 6 shown, the template matching device 600 includes: a processor 601 and a memory 602. Among them, the template matching device 600 may be a device such as a computer or a server. It should be understood that the shown structure is only exemplary and not restrictive, and in addition to these units, the template matching device 600 may further include other components. However, since these components are not related to the content of the embodiments of the present invention, their illustrations and descriptions are omitted herein.

[0090] In addition, since the specific details of the processing performed by the template matching device 600 according to the embodiment of the present invention are substantially the same as the details described above with reference to Figures 1 to 5 Therefore, for the sake of brevity, part of the description of the same details is omitted herein. The following will introduce each module or component in the template matching device 600 one by one.

[0091] The processor 601 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may utilize the computer program instructions stored in the memory 602 to execute the desired functions. Among them, when the computer program instructions are run by the processor 601, the processor performs the following steps: obtaining a template image; generating at least one model image by applying transformation parameters to the template image; extracting edge points from each of the at least one model images; for each model image, filtering the extracted edge points through a first filter based on the pixel gradient magnitude to obtain a first pixel subset corresponding to each model image; filtering the first pixel subset through a second filter based on the pixel gradient direction and pixel distance to obtain a second pixel subset corresponding to each model image; calculating the similarity between each pixel in the second pixel subset and the corresponding pixel in the image region to be detected to obtain a first matching degree of each model image; and based on the first matching degree, determining whether each of the at least one model images matches the image region to be detected.

[0092] The above steps performed by the processor 601 are combined with the aboveFigures 1 to 5 is consistent with the corresponding content of steps S301 to S307 described above.

[0093] The memory 602 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. One or more computer program instructions may be stored on the computer-readable storage media, so that the processor 601 may run the program instructions to implement the functions of the image processing device of the embodiments of the present invention described above and / or other desired functions, and / or may execute the image processing method according to the embodiments of the present invention. Various application programs and various data may also be stored in the computer-readable storage media.

[0094] Next, a computer-readable storage medium according to an embodiment of the present invention will be described. The present invention also provides a computer-readable storage medium, on which computer program instructions are stored, wherein when the computer program instructions are executed by a processor, the following steps are implemented: obtaining a template image; generating at least one model image by applying transformation parameters to the template image; extracting edge points from each of the at least one model image; for each model image, filtering the extracted edge points through a first filter based on the pixel gradient magnitude to obtain a first pixel subset corresponding to each model image; filtering the first pixel subset through a second filter based on the pixel gradient direction and pixel distance to obtain a second pixel subset corresponding to each model image; obtaining a first matching degree of each model image by calculating the similarity between each pixel in the second pixel subset and the corresponding pixel in the image region to be detected; and based on the first matching degree, determining whether each of the at least one model image matches the image region to be detected.

[0095] The above steps are combined with the above Figures 1 to 5 is consistent with the corresponding content of steps S301 to S307 described above.

[0096] In addition, it should be understood that each component or module in the above image processing device may be implemented by hardware, may be implemented by software, and may also be implemented by a combination of hardware and software.

[0097] The above embodiments are merely exemplary and not restrictive, and those skilled in the art can combine and combine some steps and devices from the above separately described embodiments according to the concept of the present invention to achieve the effects of the present invention. Such combined embodiments are also included in the present invention, and such combinations are not described one by one here. Note that the advantages, advantages, effects, etc. mentioned in the present invention are merely examples and not restrictive, and it cannot be considered that these advantages, advantages, effects, etc. are essential for each embodiment of the present invention. In addition, the specific details of the above invention are only for the purpose of illustration and easy understanding, and are not restrictive. The above details do not limit the present invention to necessarily adopt the above specific details to implement.

[0098] The block diagrams of the modules, devices, equipment, and systems involved in the present invention are only illustrative examples and do not wish to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these modules, devices, equipment, and systems can be connected, arranged, and configured in any manner. In addition, words such as "including", "comprising", "having", etc. are open-ended terms, meaning "including but not limited to", and can be used interchangeably with them. The word "or" and "and" used here refer to the word "and / or", and can be used interchangeably with it, unless the context clearly indicates otherwise. The word "such as" used here refers to the phrase "such as but not limited to", and can be used interchangeably with it.

[0099] The step flowcharts in the present invention and the above method descriptions are only illustrative examples and do not wish to require or imply that the steps of each embodiment must be carried out in the order given. As those skilled in the art will recognize, the steps in the above embodiments can be carried out in any order. Words such as "subsequently", "then", "next", etc. do not wish to limit the order of the steps; these words are only used to guide the reader through the description of these methods. In addition, for example, any reference to a singular element using the articles "a", "an", or "the", "said" is not to be construed as limiting that element to the singular.

[0100] In addition, the steps and devices in each of the embodiments herein are not limited to being implemented in a certain embodiment. In fact, according to the concept of the present invention, relevant partial steps and partial devices in each of the embodiments herein can be combined to conceive new embodiments, and these new embodiments are also included within the scope of the present invention. And, the methods and functions disclosed herein include one or more actions for implementing the said methods. The methods and / or actions can be interchanged with each other without departing from the scope of the claims. In other words, unless a specific order of actions is specified, the order and / or use of specific actions can be modified without departing from the scope of the claims.

[0101] Each operation of the methods described above can be performed by any suitable means capable of performing the corresponding functions. The means can include various hardware and / or software components and / or modules, including but not limited to circuits, application specific integrated circuits (ASICs) or processors. The various illustrated logic blocks, modules and circuits can be implemented or performed using a general purpose processor, a digital signal processor (DSP), an ASIC, a field programmable gate array signal (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components or any combination thereof designed to perform the functions described herein. The general purpose processor can be a microprocessor, but in the alternative, the processor can be any commercially available processor, controller, microcontroller or state machine. The processor can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core or any other such configuration.

[0102] The steps of the methods or algorithms described in connection with the present invention can be directly embedded in hardware, in a software module executed by a processor, or in a combination of the two. The software modules can exist in any form of tangible storage medium. Some examples of storage media that can be used include random access memory (RAM), read only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, etc. The storage medium can be coupled to the processor so that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral with the processor. The software modules can be a single instruction or many instructions, and can be distributed over several different code segments, different programs, and across multiple storage media.

[0103] Accordingly, a computer program product can perform the operations given herein. For example, such a computer program product can be a computer-readable tangible medium having instructions tangibly stored (and / or encoded) thereon that are executable by one or more processors to perform the operations described herein. The computer program product can include packaging materials. Software or instructions can also be transmitted through a transmission medium. For example, software can be transmitted from a website, server, or other remote source using a transmission medium such as coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technologies such as infrared, radio, or microwave.

[0104] In addition, modules and / or other suitable means for performing the methods and techniques described herein can be downloaded and / or otherwise obtained by the user terminal and / or the base station as appropriate. For example, such devices can be coupled to a server to facilitate the transfer of means for performing the methods described herein. Alternatively, the various methods described herein can be provided via a storage component (e.g., RAM, ROM, a physical storage medium such as a CD or floppy disk) so that the user terminal and / or the base station can obtain the various methods when coupled to the device or provided with the storage component. In addition, any other suitable techniques for providing the methods and techniques described herein to the device can be utilized.

[0105] Other examples and implementations are within the scope and spirit of the present invention and the appended claims. For example, due to the nature of software, the functions described above can be implemented using software executed by a processor, hardware, firmware, hardwiring, or any combination thereof. The features implementing the functions can also be physically located in various positions, including being distributed so that portions of the functions are implemented at different physical locations. Also, as used herein, including in the claims, the "or" used in the listing of items beginning with "at least one" indicates a disjunctive listing so that, for example, the listing of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Further, the phrase "exemplary" does not mean that the examples described are preferred or better than other examples.

[0106] Various changes, substitutions, and alterations to the techniques described herein can be made without departing from the teachings of the technology defined by the appended claims. In addition, the scope of the claims of the present invention is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and acts described above. Current or later-developed processes, machines, manufactures, compositions of events, means, methods, or acts that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Accordingly, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or acts within their scope.

[0107] The above description of the aspects of the invention is provided to enable any person skilled in the art to make or use the invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the invention. Therefore, the invention should not be limited to the aspects shown herein, but rather should be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0108] The foregoing description has been presented for purposes of illustration and description. Furthermore, this description should not be construed as limiting the embodiments of the invention to the form disclosed herein. Although several example aspects and embodiments have been discussed above, those skilled in the art will recognize some variations, modifications, alterations, additions, and subcombinations thereof.

Claims

1. A template matching method, comprising: obtaining a template image; generating at least one model image by applying transformation parameters to the template image; extracting edge points from each of the at least one model image; for each of the model images, filtering the extracted edge points through a first filter based on pixel gradient magnitude to obtain a first pixel subset corresponding to each model image, wherein the first filter includes removing edge points with a pixel gradient magnitude lower than a first threshold; filtering the first pixel subset through a second filter based on pixel gradient direction and pixel distance to obtain a second pixel subset corresponding to each model image, wherein the second filter includes removing edge points with a pixel gradient direction difference lower than a second threshold and a pixel distance lower than a third threshold; obtaining a first matching degree of each of the model images by calculating the similarity between each pixel in the second pixel subset and the corresponding pixel in the image region to be detected; and judging whether each of the at least one model image matches the image region to be detected based on the first matching degree.

2. The method according to claim 1, wherein judging whether each of the at least one model image matches the image region to be detected includes: comparing the first matching degree with a threshold; wherein if the first matching degree is not less than the threshold, determining that a specific model image associated with the first matching degree not less than the threshold positively matches the image region to be detected; and if the first matching degree is less than the threshold, determining that a specific model image associated with the first matching degree less than the threshold does not match the image region to be detected.

3. The method according to claim 2, wherein after determining that the specific model image positively matches the image region to be detected, further comprising: extracting edge points from the matching image region to be detected, and performing the first filter and the second filter on the edge points of the matching image region to be detected to obtain a third pixel subset, obtaining a second matching degree of the specific model image by calculating the similarity between each pixel in the third pixel subset and the corresponding pixel in the matching specific model image; and determining whether the specific model image matches the image region to be detected based on both the first matching degree and the second matching degree.

4. The method according to claim 3, further comprising: determining whether the specific model image matches the image region to be detected based on the minimum of both the first matching degree and the second matching degree.

5. The method according to claim 1 or 2, further comprising: calculating the sharpness of each pixel in the second pixel subset, the sharpness being defined as a value associated with the angle formed by the pixel and its adjacent pixels; assigning a weight value to each pixel in the second pixel subset based on the calculated sharpness of each pixel; and weighting and calculating the first matching degree through the weight value.

6. The method according to claim 3, further comprising: Calculate the sharpness of each pixel in the third pixel subset, where the sharpness is defined as a value associated with the angle formed by the pixel and its adjacent pixels; Based on the calculated sharpness of each pixel, assign a weight value to each pixel in the third pixel subset; And Calculate the second matching degree by weighting with the weight value.

7. The method according to any one of claims 1 to 4, wherein the similarity between pixels is calculated by the following method: Calculate the difference between the gradient direction of a pixel and the gradient direction of the corresponding pixel; and Use the difference between the gradient directions to characterize the similarity between the pixels.

8. A template matching device, comprising: a generating unit configured to obtain a template image and generate at least one model image by performing a pose transformation on the template image; an extracting unit configured to extract edge points of each model image in the at least one model image; a filtering unit configured to, for each of the model images, filter the edge points through a first filter based on the pixel gradient magnitude to obtain a first pixel subset corresponding to each model image, and filter the first pixel subset through a second filter based on the pixel gradient direction and the pixel distance to obtain a second pixel subset, wherein the first filter includes removing edge points with a pixel gradient magnitude lower than a first threshold, and the second filter includes removing edge points with a pixel gradient direction difference less than a second threshold and a pixel distance lower than a third threshold; And a matching unit configured to calculate a first matching degree between each pixel in the second pixel subset and the corresponding pixel in the image region to be detected, and based on the calculated first matching degree, determine whether each model image in the at least one model image matches the image region to be detected.

9. A template matching device, comprising: a processor; and a memory in which computer program instructions are stored, wherein when the computer program instructions are run by the processor, the processor is caused to perform the following steps: Obtain a template image; Generate at least one model image by applying transformation parameters to the template image; Extract edge points from each model image in the at least one model image; For each of the model images, filter the extracted edge points through a first filter based on the pixel gradient magnitude to obtain a first pixel subset corresponding to each model image, wherein the first filter includes removing edge points with a pixel gradient magnitude lower than a first threshold; Filter the first pixel subset through a second filter based on the pixel gradient direction and the pixel distance to obtain a second pixel subset corresponding to each model image, wherein the second filter includes removing edge points with a pixel gradient direction difference less than a second threshold and a pixel distance lower than a third threshold; Obtain the first matching degree of each model image by calculating the similarity between each pixel in the second pixel subset and the corresponding pixel in the image region to be detected; and Based on the first matching degree, determine whether each model image in the at least one model image matches the image region to be detected.

10. A computer-readable storage medium having computer program instructions stored thereon, wherein, when the computer program instructions are executed by a processor, the following steps are implemented: obtain a template image; generate at least one model image by applying transformation parameters to the template image; extract edge points from each of the at least one model image; for each of the model images, filter the extracted edge points through a first filtering based on the pixel gradient magnitude to obtain a first pixel subset corresponding to each model image, wherein the first filtering includes removing edge points with a pixel gradient magnitude lower than a first threshold; filter the first pixel subset through a second filtering based on the pixel gradient direction and the pixel distance to obtain a second pixel subset corresponding to each model image, wherein the second filtering includes removing edge points with a pixel gradient direction difference less than a second threshold and a pixel distance lower than a third threshold; obtain a first matching degree of each of the model images by calculating the similarity between each pixel in the second pixel subset and the corresponding pixel in the image region to be detected; and based on the first matching degree, determine whether each of the at least one model image matches the image region to be detected.

Images