Corner point detection method and device

Abstract

The invention discloses a corner detection method and device, and the method comprises the following steps: obtaining a to-be-detected picture, and initializing a corner set; performing edge detection on the to-be-detected picture to obtain an edge graph, and extracting a curve from the edge graph; each curve is processed as follows: gaps in the curve are filled, a plurality of T junctions in the curve are found, and all the found T junctions are added to an angular set; the curve cj is smoothed, angular points in the curve are recognized, the recognized angles are added into an angle set, and the angular points in the angle set S are all the angular points in the to-be-detected picture. The detection method has the advantages of being consistent with the real curvature, small in convex / concave angle and low in calculation complexity.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a corner point detection method and a device thereof. Background Art

[0002] Corner detection is an important task in many image analysis applications, such as shape representation, image matching and registration, and camera calibration. Many corner detectors have been developed, which can be roughly divided into the following two categories: intensity-based methods and contour-based methods.

[0003] Intensity-based methods detect corners by utilizing the intensity values ​​of pixels on the input image. Harris detectors use the autocorrelation matrix of local intensity variations to measure the difference between corners and non-corner points. Smith and Brady developed a corner measurement method called the Minimum Single Value Segment Assimilation Kernel (SUSAN), by which corners are identified based on the number of pixels similar to the central pixel within a circular template. Among them, the chord point distance-based method is a promising method that represents a class of discrete curvature models for processing digital curves that have attracted widespread attention due to their robustness to noise and ease of encoding. However, such existing models usually suffer from three problems: inconsistency with the true curvature (i.e., the curvature defined in mathematics), loss of corner convexity / concavity, and high computational complexity. Summary of the invention

[0004] In view of this, the main purpose of the present invention is to provide a corner point detection method and device.

[0005] To achieve the above object, the technical solution of the present invention is implemented as follows: a method for detecting corner points, comprising the following steps: obtaining an image to be detected, initializing a corner set Perform edge detection on the image to be detected Image and obtain an edge map, and extract the curve c from the edge map 1 , c 2 , ..., c M , where M is a natural number; for any curve c j The following processing is performed: fill curve c j The gap in the i and add all the T-junctions found to the angle set S; for curve c j Smoothing is performed, and the curve c j is represented as {point p 1 , click p 2 ,...,point }, calculate any point p i The corresponding curvature product H(i), then based on the curvature products H(1), H(2), ..., H(N j), identify curve c j The corner points in the image are added to the corner set S, where N j , i and j are all natural numbers; 1≤i≤N j , 1≤j≤M; the corner points in the corner set S are all corner points in the image to be detected Image.

[0006] As an improvement of an embodiment of the present invention, the edge detection of the image to be detected Image and obtaining the edge map specifically includes: based on a Canny detector, edge detection of the image to be detected Image and obtaining the edge map. The edge detector is a high-pass filter that can enhance high-frequency components and suppress low-frequency components. Since both edges and noise are high-frequency components, the edge detector tends to amplify the noise.

[0007] As an improvement of the embodiment of the present invention, the calculation of any point p i The corresponding curvature product H(i) specifically includes: using r 1 Measuring point p i The DCPD curvature value h(i; r 1 ), use r 2 Measuring point p i The DCPD curvature value h(i; r 2 ), use r 3 Measuring point p i The DCPD curvature value h(i; r 3 ), based on the DCPD curvature value h(i; r 1 ), h(i; r 2 ) and h(i;r 3 )Calculate point p i The corresponding curvature product H(i), where r 1 , r 2 and r 3 All are natural numbers, 1≤r 1 , r 2 and r 3 <i, function

[0008] x i 、x i-r and x i-r Point p i 、p i-r and p i+r The horizontal axis, y i ,y i-r and i-r Point p i 、p i-r and p i+r The vertical coordinate of .

[0009] As an improvement of the embodiment of the present invention, the DCPD curvature value h(i; r 1 ), h(i; r 2 ) and h(i;r 3 )Calculate point p i The corresponding curvature product H(i) specifically includes: H(i)=h(i; r 1 )·h(i;r 2 )·h(i;r 3 ).

[0010] As an improvement of the embodiment of the present invention, the curvature products H(1), H(2), ..., H(N j ), identify curve c j The corner points in the specific include: by multiplying the curvature products H(1), H(2), ..., H(N j ) performs non-maximum suppression to obtain several corner points with local maximum curvature values, and then selects the corner points with curvature values ​​greater than the predetermined threshold T as the curve c j The corner point in .

[0011] The embodiment of the present invention also provides a corner point detection device, comprising the following modules: an initialization module, for obtaining a picture to be detected Image, initializing a corner point set The processing module is used to perform edge detection on the image to be detected Image and obtain an edge map, and extract the curve c from the edge map 1 , c 2 , ..., c M , where M is a natural number; for any curve c j The following processing is performed: fill curve c j The gap in the i and add all the T-junctions found to the angle set S; for curve c j Smoothing is performed, and the curve c j is represented as {point p 1 , click p 2 ,...,point }, calculate any point p i The corresponding curvature product H(i), then based on the curvature products H(1), H(2), ..., H(N j ), identify curve c j The corner points in the image are added to the corner set S, where N j , i and j are all natural numbers; 1≤i≤N j , 1≤j≤M; a corner point module, which is used for corner points in the corner set S, all of which are corner points in the image to be detected Image.

[0012] As an improvement of the embodiment of the present invention, the processing module is further used to: perform edge detection on the image to be detected Image based on a Canny detector and obtain an edge map.

[0013] As an improvement of the embodiment of the present invention, the processing module is further used to: use r 1 Measuring point p i The DCPD curvature value h(i; r 1 ), use r 2 Measuring point p i The DCPD curvature value h(i; r 2 ), use r 3 Measuring point p i The DCPD curvature value h(i; r 3 ), based on the DCPD curvature value h(i; r 1 ), h(i; r 2 ) and h(i;r 3 )Calculate point p i The corresponding curvature product H(i), where r 1 , r 2 and r 3 All are natural numbers, 1≤r 1 , r 2 and r 3 <i, function

[0014] x i 、x i-r and x i-r Point p i 、p i-r and p i+r The horizontal axis, y i ,y i-r and i-r Point p i 、p i-r and p i+r The vertical coordinate of .

[0015] As an improvement of the embodiment of the present invention, the processing module is further configured to: H(i)=h(i; r 1 )·h(i;r 2 )

[0016] h(i;r 3 ).

[0017] As an improvement of the embodiment of the present invention, the processing module is further used to: multiply the curvature products H(1), H(2), ..., H(N j) performs non-maximum suppression to obtain several corner points with local maximum curvature values, and then selects the corner points with curvature values ​​greater than the predetermined threshold T as the curve c j The corner point in .

[0018] The corner point detection method and device provided by the embodiment of the present invention have the following advantages: The embodiment of the present invention discloses a corner point detection method and device, the detection method comprises the following steps: obtaining a picture to be detected and initializing a corner set; performing edge detection on the picture to be detected and obtaining an edge map, and extracting a curve from the edge map; performing the following processing on each curve: filling the gaps in the curve, finding several T-junctions in the curve and adding all the found T-junctions to the corner set; performing the following processing on the curve c j Smoothing is performed to identify the corners in the curve, and the identified corners are added to the corner set S. The corners in the corner set S are all corners in the image to be detected. This detection method has the advantages of being consistent with the true curvature, small convexity / concavity of the corners, and low computational complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 , Figure 2 , Figure 3A , Figure 3B , Figure 4 , Figure 5 , Fig. 6A , Figure 6B , Fig. 7A , Figure 7B , Figure 8 , Fig. 9 , Fig.10 , Fig.11 , Fig.12 , Fig.13A , Fig. 13B , Fig.14A , Fig. 14B , Fig. 14C , Fig.14D , Fig.14E , Fig.15A and Fig. 15B is a schematic diagram of a method for detecting corner points in an embodiment;

[0020] Fig.16 , Fig.17 , Fig.18 , Fig.19 , Fig. 20 , Fig.21 and Fig. 22 Graph showing experimental results of the corner point detection method in the embodiment;

[0021] Fig.23 Schematic diagram of the process of detecting corner points in the embodiment. DETAILED DESCRIPTION

[0022] The present invention will be described in detail below in conjunction with the embodiments shown in the accompanying drawings. However, the embodiments do not limit the present invention, and any structural, methodological, or functional changes made by a person skilled in the art based on the embodiments are all within the protection scope of the present invention.

[0023] The following description and accompanying drawings fully illustrate the specific embodiments of this article so that those skilled in the art can practice them. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. The scope of the embodiments of this article includes the entire scope of the claims, as well as all available equivalents of the claims. Herein, the terms "first", "second", etc. are only used to distinguish one element from another, without requiring or implying any actual relationship or order between these elements. In fact, the first element can also be called the second element, and vice versa. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the structure, device or equipment including a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also include elements inherent to such structure, device or equipment. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the structure, device or equipment including the elements. Each embodiment is described in a progressive manner herein, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other.

[0024] The terms "longitudinal", "lateral", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. in this document indicate the orientation or position relationship based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing this document and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In the description of this document, unless otherwise specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a mechanical connection or an electrical connection, it can also be the internal communication of two elements, it can be a direct connection, or it can be indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0025] Embodiment 1 of the present invention provides a method for detecting corner points. Fig.23 As shown, the following steps are included:

[0026] Step 2301: Get the image to be detected and initialize the corner set

[0027] Step 2302: Perform edge detection on the image to be detected Image and obtain an edge map, and extract curve c from the edge map 1 , c 2 , ..., c M , where M is a natural number; for any curve c j The following processing is performed: fill curve c j The gap in the i and add all the T-junctions found to the angle set S; for curve c j Smoothing is performed, and the curve c j is represented as {point p 1 , click p 2 ,...,point }, calculate any point p i The corresponding curvature product H(i),

[0028] Then, based on the curvature products H(1), H(2), ..., H(N j ), identify curve c j The corner points in the image are added to the corner set S, where N j , i and j are all natural numbers; 1≤i≤N j , 1≤j≤M;

[0029] Step 2303: The corner points in the corner set S are all corner points in the image to be detected Image.

[0030] The basic idea of ​​chord point distance is to move a span of a fixed number of points along the input curve, and then measure the discrete curvature of the curve based on the perpendicular distances of these points on the arc to their associated chords (i.e., the straight line segments connecting the two ends of the arc).

[0031] like Figure 1 As shown, in the discrete case, the input curve consists of a set of points To represent, the chord of a curve is a straight line segment whose endpoint P i-L+k and P i+k are all located on the curve L, and are L pixels apart from each other. Let L represent the chord, in order to measure point P i The chord L moves along the curve while maintaining P i is an interior point, that is, 1≤k≤L-1. Then, calculate the i The distance to the chord L is denoted by d i,k Finally, click P i The maximum arc chord distance (ACDM) at: denoted by h m (i; L), calculated as follows:

[0032] Chord to Point Distance Accumulation (CPDA) moves the chord along the curve, and then the CPDA curvature is calculated as {d i,k}, that is

[0033] In order to improve the robustness of CPDA, this detection method calculates CPDA at three different scales, that is, using three different parameter L values ​​to estimate the curvature of each point of the curve. j represents the parameter value used for the th time (j=1, 2 and 3). a (i; L j ) is first normalized: j is 1, 2, and 3 respectively. Next, the product of the normalized CPDA values,

[0034] The Curve Chord Ratio (CCR) converts point P i-r and P i+r The ratio of the chord length to the arc length between is used as an estimate of the discrete curvature, such as Figure 2 As shown. CCR can be expressed as follows: in,

[0035] Here, the inventor's working motivation is first explained based on the study of the problems encountered in the existing chord-to-point distance model, and then a new curvature model is proposed, which fully solves the problems of the existing model.

[0036] Motivation,A reasonable discrete curvature model should exhibit consistent,behavior with the true curvature (i.e.,curvature defined in mathematics).,The first typical characteristic of the true curvature is that,small-angle corners have higher curvature than large-angle corners.,Unfortunately, this fundamental property is often violated by,existing models. Figure 3A and Figure 3B This problem about CPDA is shown. Figure 3A As shown in , the input curve used for the test has three corners, namely A, B, and C, where the angle value of corner C is smaller than that of the other two corners. However, when using CPDA, the discrete curvature measured at corner C is lower than that at A and B, as shown in Figure 3B This problem will make it difficult to identify significant corners, resulting in poor corner detection performance.

[0037] Typical problems of existing discrete curvature models: Figure 3AThe test curve shown in the figure has its starting point marked by a dot and its tracing direction indicated by an arrow. Figure 3B The problem is that although the angle value of corner "C" is smaller, its measured curvature is lower than the curvatures of the other two corners "A" and "B".

[0038] In order to further study the above problem, we use a i Sharp corner patterns, such as Figure 4 As shown. At the corner P i The discrete curvature measured at The calculation is as follows:

[0039] Where θ is the angle of the corner and L is a parameter used in CPDA to determine the arc length (in pixels) required to measure the curvature. Without loss of generality, assuming 0≤θ≤π, we then investigate the effect of the measured curvature on the angle θ. How to change. Figure 5 Draw The relationship between θ and θ, where the parameter L is taken as 10 pixels. Although according to the above mathematical properties, the actual curvature of the corner should decrease monotonically with the increase of the angle value θ, it can be seen that the measured curvature exhibits inconsistent and unreasonable behavior - that is, it increases first and then decreases. Figure 5 Demonstrated the use of CPDA in Figure 4 The discrete curvature measured at the corner point of the sharp corner mode shown is plotted for different angle values.

[0040] The second typical characteristic of true curvature is that a flat curve should have low curvature at every point on it. Unfortunately, this property is also often violated by existing chord-to-point distance models. To illustrate this point, Fig. 6A and Figure 6B An illustrative example is given. Fig. 6A The highlighted curve in the edge plot shown is quite flat, so its true curvature is close to zero at every point. However, the measured discrete curvature varies greatly from one point to another, and many points have extremely high curvature values ​​(e.g., close to 1), such as Figure 6B As shown, the measured curvature values ​​range from [0, 1]; that is, 1 is the highest curvature possible and should indicate a significant corner in the curve.

[0041] Concavity is an important feature of corners and can play a key role in many applications, such as shape matching and 3D reconstruction. Mathematically, the concavity of a corner can be determined by the sign of its curvature value. That is, if the curvature is positive, the corner is convex; otherwise, it is concave. As a third problem, some existing models do not have this good property either. For example, the discrete curvature measured by CCR is always greater than 0, so the information of the concavity of the corner is lost.

[0042] In addition, it is worth noting that if the existing chord-to-point distance model only uses a single scale (i.e., arc length L) to measure discrete curvature, its performance may be further inconsistent with the perception of the human visual system (HVS). To illustrate this point, the inventors tested Fig. 7A A curve is shown in . On this curve, a real corner "A" and a false (i.e. meaningless) corner "B" are marked. The angle values ​​of these two corners are the same, but the lengths of their straight arms are different. For HVS, corner "A" is significant, while "B" is negligible. However, if a single-scale model CCR is used, these two points will have the same discrete curvature, as shown in Figure 7B As shown, they will either be detected as corners or missed - resulting in a false positive in the former case and a false negative in the latter.

[0043] In this embodiment, performing edge detection on the image to be detected Image and obtaining an edge map specifically includes: performing edge detection on the image to be detected Image and obtaining an edge map based on a Canny detector.

[0044] In this embodiment, the calculation of any point p i The corresponding curvature product H(i) specifically includes:

[0045] Using r 1 Measuring point p i The DCPD curvature value h(i; r 1 ), use r 2 Measuring point p i The DCPD curvature value h(i; r 2 ), use r 3 Measuring point p i The DCPD curvature value h(i; r 3 ), based on the DCPD curvature value h(i; r 1 ), h(i; r 2 ) and h(i;r 3 )Calculate point p i The corresponding curvature product H(i), where r 1 , r 2 and r 3All are natural numbers, 1≤r 1 , r 2 and r 3 <i, function x i 、x i-r and x i-r Point p i 、p i-r and p i+r The horizontal axis, y i ,y i-r and i-r Point p i 、p i-r and p i+r The vertical coordinate of .

[0046] In order to solve the above problems at the same time, the detection method in the embodiment of the present invention proposes a new discrete curvature model. The basic idea of ​​the detection method is to develop a curvature measurement method that is highly consistent with the real curvature and the perception of the human visual system (HVS). In the discrete case, the input curve is represented by a series of connected points {P j :(x j ,y j )|j=1,2,...,N}. Let P i is the point whose discrete curvature needs to be measured. In this detection method, the chord is taken as the connection point P i-r and P i+r where r is a predetermined parameter that defines the radius of the region of support (ROS, i.e., arc) used to measure the curvature. Then, point p i The discrete curvature at passes directly through point P i To chord P i- r P i+r The vertical distance (see Appendix Figure 8 ) and normalized by r.

[0047] The curvature is called the direct chord to point distance (DCPD) curvature value Among them, r takes the value r 1 , r 2 and r 3 , r is used as a normalization factor so that the value range of the measured curvature h(i; r) is between [-1, 1].

[0048] Compared with the existing chord-to-point distance based models, the proposed DCPD has the following multiple advantages:

[0049] (1) The first advantage of this detection method is that DCPD can show behavior consistent with the actual curvature, that is, corners with smaller angle values ​​have higher curvature. To verify this, Fig. 9 Demonstrated the use of DCPD in Figure 3A The discrete curvature measured on the test curve shown. It can be seen that the corner point "C" with a smaller angle value has a higher discrete curvature than the other two corner points. Fig.10 Further demonstrated in Figure 4 The discrete curvature h measured on the sharp corner pattern shown (θ) , and compared it with the angle value θ. It can be seen that as θ increases, h (θ) is monotonically decreasing, showing the same behavior as the actual curvature. In fact, h (θ) There is a close relationship between and θ, as described in the following theorem.

[0050] Theorem 1: For a sharp corner with an angle of θ, its DCPD (discrete curvature point distance) curvature h (θ) The value measured at this corner point satisfies h (θ) =cos(θ / 2).

[0051] Proof 1: Let d i Point P i To chord P i-r P i+r According to the definition of DCPD (Discrete Curvature Point Distance), h (θ) =d i / r. For the sharp corners considered, it can be easily seen that |P i P i-r |=r(reference Figure 4 ). Therefore, h (θ) =d i / |P i P i-r |=cos(θ / 2).

[0052] More broadly, a rounded corner pattern is studied, e.g. Fig.11 As shown. At the corner point P of this pattern i Around, the curve has a constant curvature, denoted by κ. The following theorem shows that P i There is a roughly linear relationship between the discrete curvature at and the true curvature κ.

[0053] Theorem 2: For a corner point P iFor a rounded corner with constant curvature κ around it, its DCPD (discrete curvature point distance) curvature h(i; r) satisfies h(i; r) ≈ (r / 2)κ, where r is the radius of the ROS (receive operating area), equal to half the arc length, and h(i; r) is the DCPD curvature measured at the corner point.

[0054] Proof 2: Reference Fig.11 , let O be the center of curvature, ρ be the radius of curvature, that is, ρ = 1 / κ. With r = ωρ and d i =(1-cosω)ρ, where ω is the vector OP i and OP i+r Therefore, it can be deduced that h(i;r)=d i / ω=(1-cosω) / ω.

[0055] By performing a second-order Taylor series expansion of cosω at ω = 0 as an approximation of cosω (i.e., cosω ≈ 1-ω 2 / 2), we get h(i;r)≈ω / 2=(r / 2)κ.

[0056] Through the derivation of the above two Theorems 1 and 2, it is mathematically proved that the DCPD (Discrete Curvature Point Distance) in the embodiment of the present invention can exhibit behavior consistent with the true curvature. In particular, prominent corners will have high DCPD curvature and can therefore be easily detected. This means that DCPD will produce fewer false negatives (i.e., missed detections) than using the existing problematic chord-to-point distance-based model.

[0057] (2) The second advantage of this detection method is that it will introduce fewer false positives (i.e., false detections) because according to the above two theorems, the curvature of a flat curve at every point is almost zero. Therefore, the second problem of the existing model (i.e., the possibility of extremely large curvature values ​​on a flat curve) has also been solved. Fig. 6A The maximum value of the discrete curvature measured by DCPD is only 0.0468 for the test curve given in . Therefore, when this detection method is used for corner detection, no real corners will be detected from this curve.

[0058] (3) The third advantage of this detection method is that the convexity of each corner point can be easily identified based on the sign of the measured DCPD curvature. Finally, it is worth pointing out that DCPD is more efficient than existing models in terms of computational complexity. Assume that the input curve has N points. The computational burden of different models relative to the number of square root operations (the main overhead of computing discrete curvature) is recorded in Table 1. CPDA (see paper: JH Han, T. Poston, Chord-to-point distance accumulation and planar curvature: a new approach to discrete curvature, Pattern Recog. Lett. 52022 (10) (2001) 1133–1144) and ACDM (see paper: T.-Y. Phillips, A. Rosenfeld, A method of curve partitioning using arcchord distance, Pattern Recog. Lett. 5 (4) (1987) 285–288) require L-2 square root operations when processing each point of the curve, where L is the length of the chord used. Therefore, both models require a total of (L-2)N square root operations. CCR (see paper: SWTeng, RMNSat, G.Lu, Effective and efficient contour-based corner detectors, Pattern Recog. 48(7) (2015) 2185–2197) takes N-1 square root operations to calculate the arc length, and another N-1 square root operations to calculate the chord length. Therefore, the total number of square root operations is 2N-2. In contrast, the DCPD model has a lower computational burden - it only requires N square root operations, each for calculating the discrete curvature of a point on the curve.

[0059] Table 1: Comparison of computational costs of different chord distance-based models, according to the number of square root operations.

[0060] Discrete curvature model Square root operations CPDA (L-2)N ACDM (L-2)N CCR 2N-2 DCPD N

[0061] In this embodiment, the DCPD curvature value h(i; r 1 ), h(i; r 2 ) and h(i;r 3 )Calculate point p i The corresponding curvature product H(i) specifically includes: H(i)=h(i; r 1 )·h(i;r 2 )·h(i;r3 ).

[0062] here, Fig.12 We show how to use DCPD (Discrete Curvature Point Distance) to compute the discrete curvature of a given point at three scales. First, we choose a set of scale values, denoted by {r 1 ,r 2 ,r 3}. At the current point P i Then construct three chords accordingly, namely Next, the function h(i; r) is used to calculate P i The DCPD curvature h(i; r 1 )、h(i;r 2 ) and h(i;r 3 ). Finally, the product of these three DCPD curvature values ​​is calculated and used as P i Discrete curvature in the multi-scale sense, that is, H(i) = h(i; r 1 )·h(i;r 2 )·h(i;r 3 ).

[0063] In this embodiment, the curvature products H(1), H(2), ..., H(N j ), identify curve c j The corner points in the specific include: by multiplying the curvature products H(1), H(2), ..., H(N j ) performs non-maximum suppression to obtain several corner points with local maximum curvature values, and then selects the corner points with curvature values ​​greater than the predetermined threshold T as the curve c j The corner point in .

[0064] With the curvature product set {H(1), H(2), ..., H(N j )}, the corner points on the curve are identified as follows. First, for {H(1), H(2), ..., H(N j )} Implement non-maximum suppression - only those points with local maximum curvature values ​​are considered as corner candidates. Then, a curvature threshold is used to find "meaningful" corners. That is, only those corner candidates whose curvature values ​​are greater than a predetermined threshold T are considered true corners.

[0065] Fig.13A and Fig. 13BIt shows how to adjust the parameters used in the detection method. Generally speaking, a smaller curvature threshold T will lead to many false positives (i.e., non-corner points are mistakenly identified as corner points), while a larger T may lead to a large number of false negatives (i.e., missing the real corner points). Therefore, it is necessary to determine a suitable threshold. To this end, the inventors constructed five image datasets and used these datasets to adjust the value of T. Fig.13A The F score (a measure of corner detection performance) changes under different T values. Accordingly, the default value of T is set to 0.002, at which the F score reaches the highest value. Similarly, Fig. 13B As shown, the scale set {r 1 ,r 2 ,r 3 The default setting of} is adjusted to {2, 4, 6}.

[0066] Thanks to the use of multi-scale curvature, the distinction between significant corners and insignificant curved structures becomes more distinct and can therefore be more easily detected, e.g. Fig.14A , Fig. 14B , Fig. 14C , Fig.14D and Fig.14E First, the DCPD used is measured on three scales: low, medium, and high. Fig.14A The discrete curvatures of the test curves shown in are Fig. 14B , Fig. 14C and Fig.14D As shown in . It can be seen that in these single-scale cases, the DCPD curvature shows many redundant local maxima, making it difficult to identify the true corner points. Then calculate the product of these three curvatures, as Fig.14E As shown, six meaningful corner points can be easily detected.

[0067] The identification curve c j Another advantage of the method of finding corner points in is that the sign of H(i) can be used to identify the convexity or concavity of the corner point. Specifically, if the sign is positive, the corner point is convex; if the sign is negative, it is concave. Fig.15A and Fig. 15B An illustrative example is given. Fig.15A The curve in has seven convex corners and seven concave corners, and using this identification curve c j The discrete curvature measured by the corner point method is as follows Fig. 15B According to the sign of the measured multi-scale curvature, Fig.15A The convexity and concavity of each convex corner are marked with a dot, and the concave corners are marked with a square.

[0068] experiment

[0069] In the literature, there is a lack of authoritative image datasets to perform corner detection experiments with known evaluation metrics (precision, recall, and F-score), mainly because manually annotating the real corners on an image is very time-consuming. To address this critical issue, the inventors carefully constructed five datasets containing a sufficient number and variety of real-world images from various image and vision tasks, including corner detection, shape matching, image segmentation, and edge detection. These five datasets are labeled Dataset-1 to Dataset-5, as shown in Table 2. For each test image in these datasets, the inventors carefully and as objectively as possible identified the actual corners, such as Fig.16 and Fig.17 The dataset is publicly available online. In addition, two other datasets without true corner reference solutions (labeled as Dataset-A and Dataset-B in Table 2) are adopted to evaluate the corner detection performance using evaluation metrics such as repeatability and positioning error, which have been widely used in existing works.

[0070] Table 2: Image datasets used in our experiments.

[0071]

[0072] Paper 1 is specifically: (1) F.Mokhtarian, R.Suomela, Robust image corner detection through curvature scale space, IEEE Trans.Pattern Anal.Mach.Intell.20(12)(1998)1376–1381; (2) G.Medioni, Y.Yasumoto, Corner detection and curverepresentation using cubic b-splines, Comput.Vis., Graphics, Image Process.39(3)(1987)267–278; (3)

[0073] A. Kalvin, E. Schonberg, JT Schwartz, M. Sharir, Two-dimensional, model-based, boundary matching using footprints, Int. J. of Robotics Res. 5(4) (1986) 38–55.

[0074] Paper 2 is specifically: D. Sharvit, J. Chan, H. Tek, BB Kimia, Symmetry-based indexing of image databases, J. Visual Commun. Image Represent. 9(4) (1998) 366–380.

[0075] Paper 3 is specifically: X.Sun, B.Zhong, Multi-scale corner detection based on direct chordto-point distance (data, codes, and supplementary materials).

[0076] Paper 4 is specifically: (1) F.Mokhtarian, R.Suomela, Robust image corner detection through curvature scale space, IEEE Trans.Pattern Anal.Mach.Intell.20(12)(1998)1376–1381; (2) G.-S.Xia, J.Delon, Y.Gousseau, Accurate junction detection and characterization in natural images, Int.J.of Comput.Vis.106(1)(2014)31–56.

[0077] Paper 5 is specifically: F.Kor, W F¨orstner, eTRIMS image database for interpreting images of man-made scenes, http: / / www.ipb.uni-bonn.de / projects / etrims_db / .

[0078] Paper 6 is specifically: LJLatecki, R.Lakamper, T.Eckhardt, Shape descriptors for non-rigid shapes with a single closed contour, in: Proc.IEEEConf.Comput.Vis.PatternRecog., Vol.1, IEEE, 2000, pp.424–429.

[0079] Paper 7 is specifically: M.Awrangjeb, G.Lu, Robust image corner detection based on the chordto-point distance accumulation technique, IEEE Trans.Multimedia 10(6) (2008)1059–1072.

[0080] In order to test the robustness of corner detection under different situations, five types of image quality attacks are performed on each image in the above seven datasets, and the parameters of each type are set differently as follows. Therefore, a total of 87,344 images will be experimented, including the original selected images, 3,708 rotated images, 52,530 scaled images, 24,720 mixed attack images, 4,120 JPEG compressed images, and 2,060 noisy images.

[0081] Rotation: Each image is rotated by different angles in the range [-90°, 90°] at intervals of 10° (excluding 0° as it corresponds to the original image).

[0082] Scaling: Each image is scaled uniformly and non-uniformly in the range [0.5, 2] with an interval of 0.1, and the scaling factor s x and y Independent and separately selected (excluding s x =s y =1).

[0083] Hybrid attack: A hybrid attack of rotation and scaling is applied to each image. The rotation angle is sampled in the range of [-30°, 30°] at intervals of 10° (excluding 0°), and the scaling factor {s x ,s y} are sampled independently and separately in the range [0.8, 1.2] with an interval of 0.1, and s x ≠s y .

[0084] Lossy compression: Each image is compressed using JPEG lossy compression, with the quality factor sampled in the range [5, 100] with an interval of 5.

[0085] Noise: Add Gaussian white noise with zero mean to each image. The variance of the Gaussian noise is chosen in the range [0.005, 0.05] with an interval of 0.005.

[0086] Evaluation Metrics. In the objective evaluation, six evaluation metrics are used, including the arithmetic mean of precision and recall (APR), F-score, average repetition rate (AR), and localization error (LE). The first two metrics require the use of true corner points to achieve. The average repetition rate measures the average ratio of repeated corner points between the original image and the degraded image, and the localization error is defined as the average of the pixel deviations of repeated corner points, which are specifically defined as follows:

[0087] Where No and Nt represent the number of corner points detected from the input image before and after a given image quality attack, respectively; Nm is the number of matched corner points between them; are the positions of the i-th matching corner point before and after the attack, respectively. The “receptively” here may be a misspelling of “respectively”, meaning “respectively”.

[0088] Objective evaluation

[0089] As we all know, there is a trade-off between precision and recall in the evaluation metrics. That is, for example, a method can easily obtain very high precision by sacrificing recall. Therefore, in order to make a fair comparison, the inventors adjusted the parameters of each compared corner detector to achieve the highest F-score average on Dataset-1 to Dataset-5 (based on the initially selected images), and then applied these parameters to all subsequent experiments.

[0090] True corner

[0091] First, objective evaluation is performed on images initially selected from Dataset-1 to Dataset-5, and the results are recorded in Table 3. Note that Dataset-5 represents a more challenging task compared to the other four datasets because for this dataset, only corners that are meaningful to the human visual system are marked as true corners, while unimportant corners (such as corners on the curves of tree and cloud contours) are ignored. Therefore, on Dataset-5, each compared corner detector tends to show poor performance. The existing chord-to-point distance-based corner detector CPDA performs poorly due to only utilizing a single scale. Its multi-scale version MCPDA as well as another chord-to-point distance-based method CCR achieve competitive performance compared to other types of corner detectors. In contrast, the present detection method performs best on each evaluation metric on each used dataset. This clearly verifies the potential of the proposed discrete curvature model and multi-scale corner detector.

[0092] Then, images subjected to different attacks are tested and the results are reported in Table 4, where the performance of each corner detector is measured in F-score. It can be observed that our method achieves the highest F-score in 17 of the 25 comparison cases (i.e., 5 datasets times 5 types of attacks). Finally, in each attack case, it provides the best performance with the average F-score over all used datasets.

[0093] Table 3: Comparison of different corner detectors on the first 5 datasets in terms of APR (1st item) and F-score (2nd item).

[0094]

[0095] Table 4: Comparison of F-scores of different corner detectors under different image attacks including rotation (item 1), scaling (item 2), hybrid attack (item 3), JPEG lossy compression (item 4), and Gaussian noise (item 5).

[0096]

[0097] The robustness of the proposed detection method is tested on Dataset-A and Dataset-B and compared with existing methods. There is no reference solution of the real corner points in these two datasets, so the evaluation is based on the average repeatability and localization error, which are calculated by comparing the detected corner point sets before and after different attacks. The results are shown in Figure 2. Fig.18 As shown, Fig.18 Comparison of the repeatability (sub-image a) and localization error (sub-image b) of different corner detectors on Dataset-A and Dataset-B under different types of attacks. Note that higher average repeatability and lower localization error indicate better performance. It can be seen that, on average, our detection method achieves the best repeatability and competitive localization error performance.

[0098] Subjective evaluation

[0099] Five test images were selected from the first five datasets for subjective evaluation. The results are shown in Fig.19 , Fig. 20 and Fig.21 For each method used, its detected corner points can be compared with Fig.16 and Fig.17 , to evaluate its performance. It can be seen that the proposed detection method outperforms existing methods because it produces the least number of false positives (i.e., false alarms) and false negatives (i.e., missed corner points). Fig.19Subjective comparison of three binary images using different corner detectors, from Dataset-1, Dataset-2, and Dataset-3. Fig. 20 Subjective comparison of different corner detectors on grayscale images from Dataset-4. Fig.21 Subjective comparison of different corner detectors on color images from Dataset-5.

[0100] exist Fig.16 The figure shows the test images in three datasets (Binary) and manually marked ground. (a) dataset-1, 12 commonly used corner detection images; (b) dataset-2, 25 images from the Kimia25 dataset; (c) dataset-3, 36 images of Arabic numerals and Latin letters.

[0101] Computational efficiency

[0102] In order to compare the computational efficiency of different corner detectors, these detectors are implemented on the first five datasets using the same machine (Intel CPU, 3.1GHZ, 16GB memory). The performance points between average CPU runtime and average F-score are shown in Figure 2. Fig. 22 shown.

[0103] It can be seen that although the detection method performs best in terms of F-score, its CPU running time is only slightly longer than SuperPoint and CCR. Note that SuperPoint requires the use of a GPU to accelerate the calculation, while CCR is a single-scale corner detector. Therefore, it can be concluded that the multi-scale corner detector developed by the inventor has high computational efficiency. In fact, the detection method in this embodiment can achieve real-time performance - the processing time per image on the used dataset is only about 80 milliseconds.

[0104] in conclusion

[0105] Based on a detailed study of the existing chord-to-point distance model, a new discrete curvature measurement method, namely direct chord-to-point distance (DCPD), is proposed. DCPD directly measures the curvature of a digital curve by the maximum chord-to-point distance. Mathematical derivation proves that there is a close relationship between the detection method in this embodiment and the true curvature. Therefore, DCPD can exhibit behavior consistent with the true curvature. In addition, compared with the existing chord-to-point distance model, DCPD has the lowest computational complexity.

[0106] Based on the proposed DCPD, the inventors further developed an efficient multi-scale corner detector. Given the lack of authoritative image datasets for corner detection experiments, another important contribution of the inventors' work is the establishment of a set of image datasets containing a sufficient number and variety of images collected from various image processing tasks. In particular, the inventors marked real corners on five image datasets for public access. In order to test the robustness of each compared corner detector, the inventors' test dataset was also augmented with another 87,138 images by applying five different types of attacks (such as compression) and applying different degrees of each attack. The obtained evaluation results show that the corner detector proposed by the inventors can significantly outperform multiple state-of-the-art detection methods.

[0107] Embodiment 2 of the present invention provides a corner point detection device, comprising the following modules: an initialization module, for obtaining a picture to be detected Image, initializing a corner point set The processing module is used to perform edge detection on the image to be detected Image and obtain an edge map, and extract the curve c from the edge map 1 , c 2 , ..., c M , where M is a natural number; for any curve c j The following processing is performed: fill curve c j The gap in the i and add all the T-junctions found to the angle set S; for curve c j Smoothing is performed, and the curve c j is represented as {point p 1 , click p 2 ,...,point }, calculate any point p i The corresponding curvature product H(i), then based on the curvature products H(1), H(2), ..., H(N j ), identify curve c j The corner points in the image are added to the corner set S, where N j , i and j are all natural numbers; 1≤i≤N j , 1≤j≤M; a corner point module, which is used for corner points in the corner set S, all of which are corner points in the image to be detected Image.

[0108] In this embodiment, the processing module is further used to: perform edge detection on the image to be detected Image based on a Canny detector and obtain an edge map.

[0109] In this embodiment, the processing module is further used to: use r 1 Measuring point p i The DCPD curvature value h(i; r1 ), use r 2 Measuring point p i The DCPD curvature value h(i; r 2 ), use r 3 Measuring point p i The DCPD curvature value h(i; r 3 ), based on the DCPD curvature value h(i; r 1 ), h(i; r 2 ) and h(i;r 3 )Calculate point p i The corresponding curvature product H(i), where r 1 , r 2 and r 3 All are natural numbers, 1≤r 1 , r 2 and r 3 <i, function x i 、x i-r and x i-r Point p i 、p i-r and p i+r The horizontal axis, y i ,y i-r and i-r Point p i 、p i-r and p i+r The vertical coordinate of .

[0110] In this embodiment, the processing module is further used for: H(i)=h(i; r 1 )·h(i;r 2 )·h(i;r 3 ).

[0111] In this embodiment, the processing module is further used to: multiply the curvature products H(1), H(2), ..., H(N j ) performs non-maximum suppression to obtain several corner points with local maximum curvature values, and then selects the corner points with curvature values ​​greater than the predetermined threshold T as the curve c j The corner point in .

[0112] It should be noted that although the above describes the various steps in a specific order, it does not mean that the various steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.

[0113] The present invention may be a system, a method and / or a computer program product. The computer program product may include a readable storage medium carrying computer readable program instructions for causing a processor to implement various aspects of the present invention.

[0114] The readable storage medium can be a tangible device that holds and stores the instructions used by the instruction execution device. The readable storage medium can include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0115] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for detecting corner points, characterized in that: The following steps are involved: Get the image to be detected and initialize the corner set Perform edge detection on the image to be detected Image and obtain an edge map, and extract curves c1, c2, ..., c M , where M is a natural number; for any curve c j The following processing is performed: fill curve c j The gap in the i and add all the T-junctions found to the angle set S; for curve c j Smoothing is performed, and the curve c j is represented as {point p1, point p2, ..., point p Nj }, calculate any point p i The corresponding curvature product H(i), then based on the curvature products H(1), H(2), ..., H(N j ), identify curve c j The corner points in the image are added to the corner set S, where N j , i and j are all natural numbers; 1≤i≤N j , 1≤j≤M; The corner points in the corner set S are all corner points in the image to be detected Image.

2. The detection method according to claim 1, characterized in that: The performing edge detection on the image to be detected Image and obtaining an edge map specifically includes: Based on the Canny detector, edge detection is performed on the image to be detected Image and an edge map is obtained.

3. The detection method according to claim 1, characterized in that: The calculation of any point p i The corresponding curvature product H(i) specifically includes: Use r1 to measure point p i The DCPD curvature value h(i; r1) on the surface is measured using r2 at point p i The DCPD curvature value h(i; r2) on the surface is measured using r3 at point p i The DCPD curvature value h(i; r3) on the point p is calculated based on the DCPD curvature values ​​h(i; r1), h(i; r2) and h(i; r3) i The corresponding curvature product H(i), where r1, r2 and r3 are all natural numbers, 1≤r1, r2 and r3<i, function x i 、x i-r and x i-r Point p i 、p i-r and p i+r The horizontal axis, y i ,y i-r and i-r Point p i 、p i-r and p i+r The vertical coordinate of .

4. The detection method according to claim 3, characterized in that: The DCPD curvature values ​​h(i; r1), h(i; r2) and h(i; r3) are used to calculate the point p. i The corresponding curvature product H(i) specifically includes: H(i)=h(i;r1)·h(i;r2)·h(i;r3).

5. The detection method according to claim 4, characterized in that: The curvature products H(1), H(2), ..., H(N j ), identify curve c j The corner points in include: By multiplying the curvature products H(1), H(2), ..., H(N j ) performs non-maximum suppression to obtain several corner points with local maximum curvature values, and then selects the corner points with curvature values ​​greater than the predetermined threshold T as the curve c j The corner point in .

6. A corner point detection device, characterized in that: Includes the following step modules: Initialization module, used to obtain the image to be detected and initialize the corner set The processing module is used to perform edge detection on the image to be detected Image and obtain an edge map, and extract curves c1, c2, ..., c M , where M is a natural number; for any curve c j The following processing is performed: fill curve c j The gap in the i and add all the T-junctions found to the angle set S; for curve c j Smoothing is performed, and the curve c j is represented as {point p1, point p2, ..., point p Nj }, calculate any point p i The corresponding curvature product H(i), then based on the curvature products H(1), H(2), ..., H(N j ), identify curve c j The corner points in the image are added to the corner set S, where N j , i and j are all natural numbers; 1≤i≤N j , 1≤j≤M; A corner point module is used for corner points in a corner set S, where all corner points are corner points in the image to be detected Image.

7. The detection device according to claim 6, characterized in that: The processing module is also used for: Based on the Canny detector, edge detection is performed on the image to be detected Image and an edge map is obtained.

8. The detection device according to claim 6, characterized in that: The processing module is also used for: Use r1 to measure point p i The DCPD curvature value h(i; r1) on the surface is measured using r2 at point p i The DCPD curvature value h(i; r2) on the surface is measured using r3 at point p i The DCPD curvature value h(i; r3) on the point p is calculated based on the DCPD curvature values ​​h(i; r1), h(i; r2) and h(i; r3) i The corresponding curvature product H(i), where r1, r2 and r3 are all natural numbers, 1≤r1, r2 and r3<i, function x i 、x i-r and x i-r Point p i 、p i-r and p i+r The horizontal axis, y i ,y i-r and i-r Point p i 、p i-r and p i+r The vertical coordinate of .

9. The detection device according to claim 8, characterized in that: The processing module is also used for: H(i)=h(i;r1)·h(i;r2)·h(i;r3).

10. The detection device according to claim 9, characterized in that: The processing module is also used for: By multiplying the curvature products H(1), H(2), ..., H(N j ) performs non-maximum suppression to obtain several corner points with local maximum curvature values, and then selects the corner points with curvature values ​​greater than the predetermined threshold T as the curve c j The corner point in .

Images