Template matching method based on local-overall view angle

Through the template matching method of local-holistic viewing angle, combined with shape context and metric descriptor, the problem of sensitivity to light changes and slow matching speed in pollen particle detection in the prior art is solved, and the matching accuracy and speed are improved.

CN119006860APending Publication Date: 2024-11-22BEIJING UNIV OF TECH
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202410984986.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-22
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing template matching methods have problems such as sensitivity to light changes and slow matching speed in pollen particle detection. Grayscale-based methods are difficult to deal with objects of similar colors, while shape-based methods are not sensitive to subtle differences.

Method used

The template matching method based on the local-total viewing angle is adopted. The local and overall similarity of the template and the target object is calculated by the shape context descriptor and the shape metric descriptor, and the local and overall similarity of the outline are extracted. Features to improve matching accuracy.

Benefits of technology

Improve matching accuracy and boundary integrity, enhance sensitivity to subtle differences in objects, reduce the impact of light changes, and improve matching speed.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006860A_ABST
    Figure CN119006860A_ABST
Patent Text Reader

Abstract

The invention discloses a template matching method based on a local-overall view angle, and belongs to the field of computer vision. According to the method, local and overall similarities of a template and a target object are calculated through a shape context descriptor and a shape measurement descriptor respectively, and template matching is completed by combining information of local and overall view angles. According to the method, firstly, a template library needs to be constructed, then template matching is carried out, and the matching process comprises the following steps of contour extraction, contour feature description and feature matching. Firstly, a target contour is extracted from an original picture, and then local and overall features of the contour are represented by two feature descriptors (a shape context feature and a shape measurement feature). And finally, matching the target contour with the pollen template in the constructed template library according to the feature descriptor to form a corresponding relationship between the target contour and the template, and screening a target contour region with a strong corresponding relationship as a matching result. According to the method, the matching accuracy is improved through two feature descriptors, details and an overall angle.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention discloses a template matching method based on local-global perspective, belonging to the field of computer vision. Background Art

[0002] The present invention constructs a universal template library for specific objects and matches it with the extracted object contours in the image to achieve the purpose of extracting the target object. In practical scenarios, it can be used to match pollen grains in microscopic images, count pollen grains in microscopic images and predict concentrations, which helps pollen allergy patients prevent allergies.

[0003] Existing template matching can be divided into two types according to the matching features: grayscale-based methods and shape-based methods. Grayscale-based template matching methods mainly focus on the grayscale value features of the image, and perform statistics on the grayscale values ​​of the template image and the image to be detected, and calculate the similarity to infer their matching degree. In previous studies, Barbara et al. [1] proposed the normalized correlation method, which treats the entire image as a vector and calculates the correlation coefficient between the template and the image to determine whether they match. Wang et al. [2] It combines the sequential similarity detection algorithm and the image block-based matching algorithm to achieve better performance. Grayscale-based template matching methods generally do not require complex preprocessing of the target image to achieve high matching accuracy, but they are more sensitive to grayscale changes caused by the environment, and the matching speed is slow, which has certain limitations. Shape-based template matching methods focus on the contour features of the image. This method usually first describes the contours of the template image and the image to be detected, and then calculates the similarity between the feature descriptors to achieve template matching. In previous studies, Hu et al. [3] First, the geometric invariant moment is applied to the two-dimensional image feature description, which can describe the invariant morphological features and high-order statistical features of the shape, but only one of them has translation, rotation and scaling invariance, so it is not suitable for describing complex shapes. [4] A mean field based chamfer template matching algorithm is proposed, where each template is represented as a field model, and matching the template with the input image is expressed as an estimate of the posterior maximum in the field model. [5] A shape descriptor for contour segments is proposed for target detection in complex backgrounds. The descriptor extracts the distance, angle and curvature information of contour segments and uses the correlation coefficient between descriptors as a similarity measure.

[0004] All of the above methods can segment pollen regions to a certain extent, but they have the following limitations: (1) Grayscale-based methods focus on the grayscale value features of the image. In actual scenes, the color of objects is easily affected by lighting, and there are cases where the colors of target objects are similar, so the matching between the template and the target object cannot be guaranteed. (2) Shape-based template matching methods focus more on the overall shape of the target object and are insensitive to subtle differences in the object.

[0005] To overcome these challenges, this patent proposes a template matching method based on a local-global perspective. Based on the contour of the object, two feature descriptors are used to calculate the local and overall similarities between the template and the target object respectively, thereby improving the matching accuracy through details and overall angles.

[0006] References:

[0007] [1]Barbara Zitová, Flusser J.Image registration methods:a survey[J].Image and Vision Computing, 2003, 21(11):977-1000.

[0008] [2]Wang Z,Wang B,Zhou Z,et al.A Novel SSDA-Based Block MatchingAlgorithm for Image Stabilization[C].International Conference on IntelligentHuman-machine Systems&Cybernetics.

[0009] IEEE Computer Society, 2015.

[0010] [3]Hu M.Visual pattern recognition by moment invariants[J].

[0011] Information Theory Ire Transactions on,1962,8(2):179-187.

[0012] [4]Nguyen D TA Novel Chamfer Template Matching Method UsingVariational Mean Field[C].2014IEEE Conference on Computer Vision and PatternRecognition(CVPR).IEEE Computer Society, 2014:2425-2432.

[0013] [5] Hou Chunping, Zhang Qianwen, Wang Xiaoyan, et al. Target detection algorithm in complex background based on contour matching[J]. Journal of Harbin Institute of Technology, 2020, 52(05): 121-128. Summary of the invention

[0014] The present invention proposes a template matching method based on local-global perspective, which calculates the local and global similarities of the template and the target object respectively through the shape context descriptor and the shape metric descriptor, and completes the template matching by combining the information of the local and global perspectives. The present invention first needs to build a template library, and then perform template matching. The matching process includes the following steps: contour extraction, contour feature description, and feature matching. First, the target contour is extracted from the original image, and then the local and global features of the contour are represented by two feature descriptors (shape context feature and shape metric feature). Finally, according to the feature descriptor, the target contour is matched with the pollen template in the template library we constructed, forming a corresponding relationship between the target contour and the template, and the target contour area with a strong corresponding relationship is screened as the matching result.

[0015] The specific scheme of the present invention is as shown in the attached Figure 1 As shown, the specific steps are as follows.

[0016] Step 1: Build a template library

[0017] Build a set of target object template libraries and extract the area containing the complete object contour features as the template image. To ensure the accuracy of the template, it is necessary to manually locate the target object in the image, then use the Canny operator to extract the outer edge lines of the target object, and after manual correction, form a contour template library.

[0018] Step 2: Contour extraction

[0019] First, the image is processed by migration mean filtering and Gaussian filtering to weaken the influence of internal texture and smooth the image edge; after the image is converted into a grayscale image, the Canny operator is used to extract all the edge lines in the image; finally, the outermost closed lines are retained as the contour extraction result.

[0020] Step 3: Shape context feature extraction

[0021] The shape context feature descriptor is used to count the context information of each point in the contour point set. The shape context features of the target contour are obtained through operations such as contour point sampling and shape context feature descriptor extraction.

[0022] Step 4: Shape metric feature extraction

[0023] In order to measure the overall shape of the contour, the present invention proposes a shape measurement feature, which includes four feature dimensions, namely roundness ratio, circular variance, rectangularity and concavity. We extract the above four-dimensional features of the target contour to obtain its shape measurement feature.

[0024] Step 5: Feature Matching

[0025] Two feature descriptors are calculated for the template contour and the target contour respectively. The extracted target contour is matched with the template image according to the feature distance. The contour area with high matching degree is selected. After contour filtering and K-NN classification, the matching result is obtained.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] 1. The present invention adopts a template matching method based on a local-global perspective, which fully considers the local features and overall features of the object, improves the matching accuracy from the perspective of details and the whole, and ensures the integrity of the boundary of the matching area.

[0028] 2. The position-boundary information guided image object segmentation method adopted by the present invention fully considers the characteristics of multiple dimensions in terms of the overall perspective of the object contour, maps the similarity between objects to the similarity between objects and regular shapes, and completes matching by comparing the similarities. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is the overall model diagram of the method proposed in the present invention. DETAILED DESCRIPTION

[0030] The following is a detailed description of the implementation examples of the present invention in conjunction with the accompanying drawings:

[0031] The present invention is a template matching method based on local-global perspective. The local and global similarities of the template and the target object are calculated respectively by shape context descriptors and shape metric descriptors, and the template matching is completed by combining the information of local and global perspectives. The present invention first needs to build a template library, and then perform template matching. The matching process includes the following steps: contour extraction, contour feature description, and feature matching. First, the target contour is extracted from the original image, and then the local and global features of the contour are represented by two feature descriptors (shape context features and shape metric features). Finally, according to the feature descriptors, the target contour is matched with the pollen template in the template library we constructed, forming a corresponding relationship between the target contour and the template, and the target contour area with a strong corresponding relationship is screened as the matching result.

[0032] Specifically, the method comprises the following steps:

[0033] Step 1: Build a template library

[0034] Construct a set of target object template libraries and extract the area containing the complete object contour features as the template image. To ensure the accuracy of the template, it is necessary to manually locate the target object in the image, then use the Canny operator to extract the outer edge lines of the target object, and after manual correction, form a contour template library with M categories and N numbers in each category, that is, the total number of templates is M×N

[0035] Step 2: Contour extraction

[0036] First, the image is processed with migration mean filtering and Gaussian filtering to weaken the influence of internal texture and smooth the image edge; then the image is converted into a grayscale image and all edge lines in the image are extracted using the Canny operator; finally, an edge filtering step is performed, which retains the outermost closed lines as the sample contour result.

[0037] Step 3: Shape context feature extraction

[0038] The present invention uses shape context feature descriptors to count the local context information of each point in a contour point set.

[0039] Step 3.2.1 Contour point sampling:

[0040] For n contour points of a single contour, the present invention counts the distance between each point and the remaining n-1 points to obtain an n×n relative distance matrix. Each time, any point in the shortest distance point pair is eliminated and the distance relationship between it and other points is eliminated. This step is repeated until only m contour points (m<=n, m is set to 100 in the present invention) are left in the point pair to complete the sampling.

[0041] Step 3.2.2 Calculate the shape context feature descriptor: Among the m contour sampling points, for one of the points p i , construct the polar logarithmic coordinate system centered on it. In order to ensure that the descriptor is rotationally invariant, we define p i The direction of the line connecting the contour center point g is the positive direction of the coordinate system X axis. The center point g is calculated as shown in formula (1):

[0042]

[0043] Where m represents the total number of contour sampling points, x i ,y i Respectively represent the horizontal and vertical coordinates of the i-th point.

[0044] The coordinate system is divided into 12 angle regions and 5 distance regions (bins), that is, a total of 60 polar coordinate regions (denoted as K) are generated, and the number of sample points falling into different regions is counted. Finally, normalization is performed to obtain the shape context descriptor F SC As shown in formula (2):

[0045]

[0046] The formula #{U:V} represents the number of V under the condition U. p¢ represents the remaining boundary sample points, and bin(k) represents p i Each contour sampling point generates a shape context histogram Represents the value corresponding to the kth polar coordinate area in the histogram, and the m histograms obtained from m sampling points in the image are defined as the shape upper and lower features of the image.

[0047] Step 4: Shape metric feature extraction

[0048] In order to measure the overall shape of the contour, the present invention proposes a shape measurement feature, which includes four feature dimensions, namely roundness ratio, circular variance, rectangularity and concavity. We extract the above four-dimensional features of the target contour and calculate its shape measurement feature.

[0049] Step 4.1 Calculate the roundness ratio: The roundness ratio can measure the degree of circularity of the contour, which is defined as the ratio between the area enclosed by the contour and the area of ​​a circle with the same circumference, denoted as F cr , as shown in formula (3).

[0050]

[0051] Where A represents the area, P represents the perimeter, and A boundary , A circle , P boundaryThey are the area enclosed by the target contour, the area of ​​the circle, and the circumference of the target contour respectively.

[0052] Step 4.2 Calculate the circular variance: The circular variance can measure the degree of circularity of the contour, which is defined as the ratio between the standard deviation of the distance set between the contour points and the centroid and the mean value, denoted as F cv , as shown in formula (4)(5).

[0053] r i =p i -g(4)

[0054]

[0055] The formula ‖‖ represents the radial distance, g represents the center of gravity (calculated by formula (1)), p i represents the i-th contour point, r i Indicates the radial distance between the i-th point and the centroid, s r represents the standard deviation of radial distance, μ r Represents the average value of radial distance.

[0056] Step 4.3 Calculate the rectangularity: The rectangularity can measure the density of the shape inside the circumscribed rectangle, that is, the similarity between the contour and the rectangle, which is defined as the ratio between the area enclosed by the contour and the area of ​​its smallest circumscribed rectangle, denoted by F re , as shown in formula (6).

[0057]

[0058] Among them A box Represents the area of ​​the minimum enclosing rectangle.

[0059] Step 4.4 Calculate concavity: Concavity measures the degree of concavity of a contour and is defined as the ratio between the perimeter of the convex hull of the contour and the perimeter of the contour, denoted as F co , as shown in formula (7).

[0060]

[0061] Where P hull Represents the convex hull perimeter.

[0062] Step 4.5 Calculate the shape metric feature descriptor: Finally, the present invention calculates the above four eigenvalues ​​for the sample contour points and integrates them to obtain the shape metric feature descriptor F SM , as shown in formula (8).

[0063] F SM =[F cr ,F cv ,F re ,F co ](8)

[0064] Step 5: Feature Matching

[0065] Two feature descriptors are calculated for the template contour and the target contour respectively. The extracted target contour is matched with the template image according to the feature distance. The contour area with high matching degree is selected. After contour filtering and K-NN classification, the matching result is obtained.

[0066] Step 5.1 Calculate the distance of the shape context feature: The shape context feature descriptor can be used to calculate the distance of each point p on the first contour. i , find the "best" matching point q on the second contour j The characteristic distance between all matching points is used as the characteristic distance measure between contours.

[0067] For a point p on the target contour i and a point q on the template contour j , using the chi-square statistic C(p i ,q j ), to represent the feature distance between the two points. The feature distance between each two points constitutes an m×m feature distance matrix C, as shown in formula (9). We use F to represent the feature of the target contour and T to represent the feature of the template contour.

[0068]

[0069] and They represent the feature values ​​of the kth polar coordinate region of the ith target contour point and the jth template contour point respectively, and K represents the total number of polar coordinate regions.

[0070] Shape context feature distance metric D SC It represents the sum of the feature distances of the best matching points on the target contour and the template contour, and is obtained by symmetrically adding the feature distance matrices, as shown in formula (10).

[0071]

[0072] represents the minimum value of V under the condition of U, m represents the number of sample points, p i and q j They represent the i-th target contour sample point and the j-th template contour sample point respectively.

[0073] Step 5.2: Calculate the distance of the shape metric feature: For the shape metric feature of the contour, the present invention uses the Euclidean distance between the target contour and the template contour feature to represent the shape metric feature distance D SM , as expressed in formula (11).

[0074]

[0075] where F type and T type They represent the type features of the target contour and template contour respectively. Type can be ce, cv, re, or co.

[0076] Step 5.3 Calculate the joint distance representation: We define the shape distance D between the target contour and the template contour as the shape context distance D SC Distance D from shape metric feature SM The weighted sum of is shown in formula (12):

[0077] D=w×D SC +(1-w)×D SM (12)

[0078] Wherein, w is a weighted parameter corresponding to the two distances, which is defined as 0.8 in the present invention.

[0079] Step 5.4 K-NN classification: For each target contour extracted from the image, M×N distance features can be generated from M×N templates. We use the idea of ​​K-NN classification to count the proportion of each category in the N templates that are closest to the sample contour (that is, the smallest distance feature), and take the category with the largest proportion as the category of this contour.

Claims

1. A template matching method based on local-global perspective, characterized in that: Step 1: Build a template library Construct a set of target object template libraries, extract the area containing the complete object contour features as the template image; locate the target object in the image, and then use the Canny operator to extract the outer edge lines of the target object to form a contour template library with M categories and N numbers in each category, that is, the total number of templates is M×N; Step 2: Contour extraction First, the image is processed by migration mean filtering and Gaussian filtering to weaken the influence of internal texture and smooth the image edge; then the image is converted into a grayscale image and all edge lines in the image are extracted using the Canny operator; finally, an edge filtering step is performed, which retains the outermost closed lines as the sample contour result; Step 3: Shape context feature extraction The shape context feature descriptor is used to count the local context information of each point in the contour point set; Step 3.2.1 Contour point sampling: For the n contour points of a single contour, the distance between each point and the remaining n-1 points is counted to obtain an n×n relative distance matrix; each time, any point in the shortest distance point pair is removed and the distance relationship between it and other points is eliminated, and this step is repeated until only m contour points are left in the point pair to complete the sampling; m<=n; Step 3.2.2 Calculate the shape context feature descriptor: Among the m contour sampling points, for one of the points p i , construct the polar logarithmic coordinate system centered on it, and define p to ensure that the descriptor is rotationally invariant. i The direction of the line connecting the contour center point g is the positive direction of the coordinate system X axis; the center point g is calculated as shown in formula (1): Where m represents the total number of contour sampling points, x i ,y i Respectively represent the horizontal and vertical coordinates of the i-th point; The coordinate system is divided into 12 angle regions and 5 distance region bins, that is, a total of 60 polar coordinate regions are generated, denoted as K; the number of sample points falling into different regions is counted, and finally normalized to obtain the shape context descriptor F SC ; As shown in formula (2): The formula #{U:V} represents the number of V under the condition U; p¢ represents the remaining boundary sample points, and bin(k) represents p i The k-th polar coordinate area of ​​the point; each contour sampling point generates a shape context histogram Represents the value corresponding to the kth polar coordinate area in the histogram. The m histograms obtained from m sampling points in the image are defined as the upper and lower shape features of the image. Step 4: Shape metric feature extraction A shape metric feature is proposed, which includes four feature dimensions, namely roundness ratio, circular variance, rectangularity and concavity. The shape metric feature is calculated by extracting the features of the above four dimensions from the target contour. Step 4.1 Calculate the circularity ratio: The circularity ratio measures the degree of circularity of the contour and is defined as the ratio between the area enclosed by the contour and the area of ​​a circle with the same circumference, denoted as F cr , as shown in formula (3); Where A represents the area, P represents the perimeter, and A boundary , A circle , P boundary They are the area enclosed by the target contour, the area of ​​the circle, and the circumference of the target contour respectively; Step 4.2 Calculate the circular variance: The circular variance measures the degree of circularity of the contour, and is defined as the ratio between the standard deviation of the set of distances between the contour points and the centroid and the mean value, denoted as F cv , as shown in formula (4)(5); r i =||p i -g|| (4) The formula ‖‖ represents the radial distance, g represents the center of gravity (calculated by formula (1)), p i represents the i-th contour point, r i Indicates the radial distance between the i-th point and the centroid, s r represents the standard deviation of radial distance, μ r represents the average value of radial distance; Step 4.3 Calculate rectangularity: The rectangularity measures the density of the shape inside the circumscribed rectangle, that is, the similarity between the contour and the rectangle, and is defined as the ratio between the area enclosed by the contour and the area of ​​its smallest circumscribed rectangle, denoted by F re , as shown in formula (6); Among them A box Represents the area of ​​the minimum enclosing rectangle; Step 4.4 Calculate concavity: Concavity measures the degree of concavity of a contour and is defined as the ratio between the perimeter of the convex hull of the contour and the perimeter of the contour, denoted as F co , as shown in formula (7); Where P hull represents the perimeter of the convex hull; Step 4.5 Calculate the shape metric feature descriptor: Calculate the above four eigenvalues ​​for the sample contour points and integrate them to obtain the shape metric feature descriptor F SM , as shown in formula (8); F SM =[F cr ,F cv ,F re ,F co ] (8) Step 5: Feature Matching Two feature descriptors are calculated for the template contour and the target contour respectively. The extracted target contour is matched with the template image according to the feature distance. The contour area with high matching degree is selected. After contour filtering and K-NN classification, the matching result is obtained. Step 5.1 Calculate the distance of the shape context feature: For each point p on the first contour, use the shape context feature descriptor i , find the "best" matching point q on the second contour j ; The characteristic distance between all matching points is used as the characteristic distance measure between contours; For a point p on the target contour i and a point q on the template contour j , using the chi-square statistic C(p i ,q j ), to represent the characteristic distance between the two points; the characteristic distance between each two points constitutes an m×m characteristic distance matrix C, as shown in formula (9); F represents the characteristics of the target contour, and T represents the characteristics of the template contour; and They represent the feature values ​​of the k-th polar coordinate region of the i-th target contour point and the j-th template contour point, respectively, and K represents the total number of polar coordinate regions; Shape context feature distance metric D SC represents the sum of the feature distances of the best matching points on the target contour and the template contour, which is obtained by symmetrically adding the feature distance matrix, as shown in formula (10); represents the minimum value of V under the condition of U, m represents the number of sample points, p i and q j Respectively represent the i-th target contour sample point and the j-th template contour sample point; Step 5.2 Calculate the distance of shape metric features: For the shape metric features of the contour, the Euclidean distance between the target contour and the template contour feature is used to represent the shape metric feature distance D SM , as expressed in formula (11); Among them, F type and T type Represents the type features of the target contour and template contour respectively, and type can be ce, cv, re or co; Step 5.3 Calculate the joint distance representation: The shape distance D between the target contour and the template contour is defined as the shape context distance D SC Distance D from shape metric feature SM The weighted sum of is shown in formula (12): D=w×D SC +(1-w)×D SM (12) In the formula, w is the weighting parameter corresponding to the two distances, which is defined as 0.8; Step 5.4 K-NN classification: For each target contour extracted from the image, M×N distance features are generated from M×N templates. Using the idea of ​​K-NN classification, the proportion of each category in the N templates that are closest to the sample contour, that is, the smallest distance feature, is counted, and the category with the largest proportion is taken as the category of this contour.

Citation Information

Patent Citations

  • Shape matching method based on measurement information

    CN103093461A

  • Target recognition method based on image contour characteristic

    CN107103323A

  • Image object segmentation method based on position-boundary information guidance

    CN116883710A

  • Contour shape recognition method

    US20230047131A1