Method and device for detecting harris corner points of an image
By using detection algorithms of different scales and non-maximum suppression methods in Harris corner detection, the problem of high computational complexity is solved, and the real-time performance and accuracy are improved.
Patent Information
- Application Number
- CN202311277697.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-28
- Publication Date
- 2026-05-15
- Estimated Expiration
- 2043-09-28
AI Technical Summary
Existing Harris corner detection methods have high computational complexity, which affects system real-time performance and results in redundant calculations.
By pre-screening the target image using detection algorithms at different scales, potential texture points are identified, and Harris corner points are determined from the candidate points using a non-maximum suppression method, reducing computational load while ensuring accuracy.
This reduces the computational complexity of Harris corner detection, ensuring the system's real-time performance while improving the accuracy and effectiveness of candidate point recognition.
Smart Images
Figure CN117315280B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of image processing technology, and in particular relates to a method and apparatus for detecting Harris corner points in an image. Background Technology
[0002] Harris corner detection has a wide range of applications in computer vision, such as autonomous driving, robotics, and AR / VR. By detecting Harris corners, texture information in images can be extracted more efficiently, reducing computational load and ensuring accurate image registration. However, Harris corner detection suffers from disadvantages such as high computational complexity and impacts system real-time performance.
[0003] Existing methods for Harris corner extraction using OpenCV (an open-source computer vision library) involve a large amount of redundant computation, which affects the real-time performance of Harris corner detection methods. Summary of the Invention
[0004] In view of the above problems, this application is proposed to solve the aforementioned problems existing in the prior art.
[0005] The purpose of this application is to provide a method for detecting Harris corners in images. This method obtains potential texture points by pre-screening points in the target image to select candidate points for Harris score calculation, and then identifies Harris corners from these candidate points. This reduces computational load and complexity, thereby ensuring the real-time performance of the system. Furthermore, this application performs pre-screening through multiple recognition steps, thereby reducing redundant computation during Harris corner detection while ensuring high candidate point recognition accuracy, thus guaranteeing the final usability of the detected Harris corners.
[0006] According to a first aspect of this disclosure, a method for detecting Harris corners in an image is provided. The method includes a processor:
[0007] Obtain the target image for Harris corner detection. Using a first detection algorithm at a first scale and a second detection algorithm at a second scale greater than the first scale, preliminarily identify potential texture points from the target image. Based on the preliminarily identified potential texture points, perform at least one expansion to determine candidate neighborhoods. Using points in the candidate neighborhoods as candidate points, calculate the Harris score for each candidate point. Among the candidate points, extract a predetermined number of candidate points with the highest Harris scores as Harris corners using a non-maximum suppression method.
[0008] According to a second aspect of this application, an apparatus for detecting Harris corners of an image is provided. The apparatus includes a processor configured to perform a method for detecting Harris corners of an image according to various embodiments of this application.
[0009] The Harris corner detection method and apparatus of this application perform pre-screening by recognizing points in the target image multiple times at multiple scales to obtain possible texture points, thereby selecting candidate points for Harris score calculation, and determining Harris corners from the selected candidate points. This reduces redundant computation during Harris corner detection while ensuring high candidate point recognition accuracy, meaning that most strong corners are detected. This reduces the computational load and complexity of Harris corner detection, ensuring system real-time performance while guaranteeing the final usability of the detected Harris corners. Attached Figure Description
[0010] In drawings that are not necessarily drawn to scale, the same reference numerals may describe similar parts in different views. The same reference numerals with or without letter suffixes may indicate different instances of similar parts. The drawings illustrate various embodiments generally by way of example rather than limitation, and are used, together with the description and claims, to explain the disclosed embodiments. Where appropriate, the same reference numerals are used in all drawings to refer to the same or similar parts. Such embodiments are illustrative and not intended to be exhaustive or exclusive embodiments of the apparatus or method.
[0011] Figure 1 A first flowchart of a Harris corner detection method for an image according to an embodiment of this application is shown;
[0012] Figure 2 A flowchart of a first detection algorithm according to an embodiment of this application is shown;
[0013] Figure 3 This diagram illustrates a set of surrounding points according to Example 1 of an embodiment of this application.
[0014] Figure 4 A flowchart of a second detection algorithm according to Example 1 of this application is shown;
[0015] Figure 5 A flowchart of a first detection algorithm according to Example 2 of this application is shown;
[0016] Figure 6 This diagram illustrates a peripheral point set according to Example 2 of an embodiment of this application.
[0017] Figure 7A flowchart of a second detection algorithm according to Example 2 of this application is shown;
[0018] Figure 8 A second flowchart illustrating a Harris corner detection method for an image according to an embodiment of this application is shown;
[0019] Figure 9 A first flowchart illustrating the determination of candidate neighborhoods according to Example 1 of an embodiment of this application is shown;
[0020] Figure 10 A schematic diagram of candidate neighborhoods according to an embodiment of this application is shown;
[0021] Figure 11 A second flowchart illustrating the determination of candidate neighborhoods according to Example 1 of an embodiment of this application is shown;
[0022] Figure 12 A first flowchart illustrating Example 2 of an embodiment of this application for determining candidate neighborhoods is shown; and
[0023] Figure 13 A second flowchart illustrating the determination of candidate neighborhoods according to Example 2 of an embodiment of this application is shown. Detailed Implementation
[0024] To enable those skilled in the art to better understand the technical solutions of this disclosure, the disclosure will be described in detail below with reference to the accompanying drawings and specific embodiments. The embodiments of this disclosure will be further described in detail below with reference to the accompanying drawings and specific examples, but this is not intended to limit the disclosure. If there is no necessary sequential relationship between the various steps described herein, the order in which they are described as examples should not be considered a limitation. Those skilled in the art should understand that the order can be adjusted as long as it does not disrupt the logical coherence between them and render the entire process impossible.
[0025] In some embodiments of this application, a method for detecting Harris corners in an image is provided. For example... Figure 1 As shown in Examples 1 and 2 of this application, the flow of the Harris corner detection method for an image includes steps S1 to S5. In this detection method, the processor executes the following steps S1 to S5:
[0026] In step S1, the target image to be subjected to Harris corner detection is acquired.
[0027] In step S2, a first detection algorithm at a first scale and a second detection algorithm at a second scale greater than the first scale are used to initially identify possible texture points from the target image.
[0028] Specifically, when identifying potential texture points in a target image, false detections can occur if the different scale settings of various detection algorithms are not considered. For example, when using a detection algorithm at the first scale for potential texture point identification, potential texture points that can be identified at the second scale may be falsely identified as non-potential texture points. To ensure the accuracy of candidate point identification, two detection algorithms at different scales are used here to detect potential texture points.
[0029] In step S3, at least one expansion is performed based on the initially identified possible texture points to determine candidate neighborhoods.
[0030] Specifically, selecting a region with a high probability of Harris corner detection and detecting Harris corners from points within that region helps improve the detection rate of Harris corners. Since the vicinity of a textured potential point is a region with strong texture, which is also a region with a high probability of Harris corner detection, in order to improve the detection rate of Harris corners, candidate neighborhoods determined by expanding based on textured potential points are used as candidate regions for Harris corner detection.
[0031] In step S4, points in the candidate neighborhood are used as candidate points, and the Harris score of each candidate point is calculated.
[0032] In step S5, among each candidate point, a preset number of candidate points with the highest Harris scores are extracted as Harris corner points using a non-maximum suppression method.
[0033] Specifically, Non-Maximum Suppression (NMS) can be understood as a local maximum search. In the technical solution of this application, "local" can be understood as each candidate neighborhood within a fixed-size region centered on each candidate point. In step S5, for each candidate neighborhood, the candidate point with the highest Harris score (i.e., the maximum value of the Harris score) is extracted using the NMS method and designated as the Harris corner point of that candidate neighborhood.
[0034] The Harris corner detection method and apparatus of this application obtains possible texture points by performing pre-screening through multiple identifications of points in the target image at multiple scales, in order to select candidate points for Harris score calculation, and determines Harris corners from the selected candidate points using NMS. This reduces redundant computation during Harris corner detection while ensuring a high accuracy rate for candidate point recognition, that is, ensuring that most strong corners are detected, thereby reducing computational load and complexity, ensuring the real-time performance of the system, and also ensuring the final usability of the detected Harris corners.
[0035] In some embodiments, the first detection algorithm includes a fast 8-5 point detection algorithm, and the second detection algorithm includes a fast 16-9 point detection algorithm. The detected fast 8-5 points or fast 16-9 points are used as identified potential texture points.
[0036] In Example 1 of this application, a fast 8-5 point detection algorithm is used as the first detection algorithm, and a fast 16-9 point detection algorithm is used as the second detection algorithm.
[0037] In some embodiments, such as Figure 2 As shown, the first detection algorithm includes steps S211 to S213. In Example 1 of this application, the fast 8-5 point detection algorithm performs the following steps S211 to S213 for each point of the target image:
[0038] In step S211, the gray values of each of the first surrounding points of the point and its first scale surrounding point set are obtained.
[0039] For example, such as Figure 3 As shown, let any point in the target image be point P(x, y), then the set of surrounding points with a radius of 1 pixel is point M1-M8. Calculate the gray value Ip of point P and the gray values IM1-IM8 of each point in the first surrounding set M1-M8.
[0040] In step S212, the deviation between the gray value of each first peripheral point and the gray value of that point is calculated.
[0041] For example, the grayscale difference D1-D8 between the first peripheral points M1-M8 and point P is calculated using Equation 1:
[0042] Di = IMi - Ip … Equation 1
[0043] Where Di is the grayscale difference between the i-th point and point P, IMi is the grayscale value of the i-th point, and Ip is the grayscale value of point P.
[0044] In step S213, if the absolute value of the deviation corresponding to the first predetermined number of consecutive first peripheral points in any group is greater than the first threshold, the point is identified as a possible texture point.
[0045] For example, as shown in Table 1, five consecutive points are selected from the eight points M1-M8 of the first perimeter, and the gray-level differences D between these five points and point P are combined into a continuous gray-level difference set. If the absolute value of the gray-level difference between a set of five consecutive points and point P is greater than the first threshold, that is, if the five values in a continuous gray-level difference set are all greater than parameter t1 or all less than parameter -t1, then point P is identified as a fast8-5 point, i.e., a possible texture point Pt(x, y).
[0046] For example, the first threshold value ranges from 5 to 30. Preferably, the first threshold is 20.
[0047] Table 1
[0048]
[0049]
[0050] like Figure 4 As shown, the second detection algorithm includes steps S221 to S223. In Example 1 of this application, the fast 16-9 point detection algorithm performs the following steps S221 to S223 for each point of the target image:
[0051] In step S221, the gray values of each of the second surrounding points of the point and its second-scale surrounding point set are obtained.
[0052] For example, such as Figure 3 As shown, let any point in the target image be point P(x, y), then the set of surrounding points with a radius of 3 pixels is point N1-N16. Calculate the gray value Ip of point P and the gray values IN1-IN16 of each of the second surrounding points N1-N16.
[0053] In step S222, the deviation between the gray value of each second peripheral point and the gray value of that point is calculated.
[0054] For example, the grayscale difference D1-D16 between the second peripheral points N1-N16 and point P is calculated using Equation 2:
[0055] Di = INi - Ip … Equation 2
[0056] Where Di is the grayscale difference between the i-th point and point P, INi is the grayscale value of the i-th point, and Ip is the grayscale value of point P.
[0057] In step S223, if the absolute value of the deviation corresponding to any group of a second predetermined number of consecutive second peripheral points is greater than a second threshold, the point is identified as a possible texture point. The second predetermined number is greater than the first predetermined number.
[0058] For example, nine consecutive points are selected from the 16 points N1-N16 of the second perimeter, and the gray-level differences D between these nine points and point P are combined into a continuous gray-level difference set. If the absolute value of the gray-level differences between a set of nine consecutive points and point P is greater than the second threshold, that is, if the nine values in a continuous gray-level difference set are all greater than parameter t2 or all less than parameter -t2, then point P is identified as a fast16-9 point, i.e., a possible texture point Pt(x, y).
[0059] For example, the second threshold value ranges from 5 to 30. Preferably, the second threshold is 20.
[0060] Here, the first threshold and the second threshold can be the same value or different values.
[0061] As mentioned above, if the different scale settings of various detection algorithms are not considered when identifying potential texture points in a target image, false detections may occur. For example, when using only the fast8-5 point detection algorithm to identify potential texture points, points that can be identified as potential texture points by the fast16-9 point detection algorithm may be falsely identified as non-potential texture points, and vice versa. In the embodiments of this application, for each point in the target image, a fast8-5 point detection algorithm with a radius of 1 pixel and a fast16-9 point detection algorithm with a radius of 3 pixels are used to perform two detections to identify potential texture points in the target image, thereby ensuring the accuracy of candidate point identification.
[0062] In some embodiments, the first detection algorithm includes a Susan-R1 point detection algorithm, and the second detection algorithm includes a Susan-R3 point detection algorithm. The detected Susan-R1 points or Susan-R3 points are used as identified possible texture points.
[0063] In Example 2 of this application, the Susan-R1 point detection algorithm is used as the first detection algorithm, and the Susan-R3 point detection algorithm is used as the second detection algorithm. As mentioned above, if the different scale settings of each detection algorithm are not considered when identifying potential texture points in the target image, false detections will occur. Therefore, in this embodiment, for each point in the target image, the Susan-R1 point detection algorithm corresponding to the first scale with a radius of 1 pixel and the Susan-R3 point detection algorithm corresponding to the second scale with a radius of 3 pixels are used to perform two detections to identify potential texture points in the target image, so as to ensure the accuracy of candidate point identification.
[0064] In some embodiments, such as Figure 5As shown, the first detection algorithm includes steps S231 to S235. In Example 2 of this application, the Susan-R1 point detection algorithm performs the following steps S231 to S235 for each point of the target image:
[0065] In step S231, the gray values of each of the first surrounding points of the point and its first scale surrounding point set are obtained.
[0066] For example, such as Figure 6 As shown, let any point in the target image be point P(x, y), then the set of surrounding points with a radius of 1 pixel is point N29-N36. Calculate the gray value Ip of point P and the gray values IN29-IN36 of each point in the first surrounding point N29-N36.
[0067] In step S232, the absolute value of the deviation between the gray value of each first peripheral point and the gray value of that point is calculated.
[0068] For example, Equation 3 is used to calculate the absolute value of the deviation between the gray values of the first peripheral points N29-N36 and point P, that is, the absolute value of the gray value difference AD29-AD36:
[0069] ADi=|INi-Ip|…Equation 3
[0070] Where ADi is the absolute value of the grayscale difference between the i-th point and point P, INi is the grayscale value of the i-th point, and Ip is the grayscale value of point P.
[0071] In step S233, the number of first peripheral points whose absolute value of the corresponding deviation is not greater than the third threshold is calculated.
[0072] For example, the value range of the third threshold is greater than or equal to 5 and less than or equal to 30. Preferably, the third threshold is 20. The absolute value of the statistical grayscale difference AD is not greater than the number n1 of the first peripheral points of the third threshold.
[0073] In step S234, the ratio K1 of the number of the first peripheral points n1 to the total number of the first peripheral points N1 in the peripheral point set of the first scale is calculated as the first ratio.
[0074] For example, the first ratio K1 is calculated using Equation 4:
[0075]
[0076] In step S235, if the first ratio K1 is less than the fourth threshold, the point is identified as a possible texture point.
[0077] For example, the fourth threshold is preferably 0.5. When K1 is less than 0.5, point P is identified as a possible texture point Pt(x, y).
[0078] like Figure 7 As shown, the second detection algorithm includes steps S241 to S245. In Example 2 of this application, the Susan-R3 point detection algorithm performs the following steps S241 to S245 for each point of the target image:
[0079] In step S241, the gray values of each of the second surrounding points of the point and its second-scale surrounding point set are obtained.
[0080] For example, such as Figure 6 As shown, let any point in the target image be point P(x, y), then the set of surrounding points with a radius of 3 pixels is point N1-N36. Calculate the gray value Ip of point P and the gray values IN1-IN36 of each of the second surrounding points N1-N36.
[0081] In step S242, the absolute value of the deviation between the gray value of each second peripheral point and the gray value of that point is calculated.
[0082] For example, the absolute values of the grayscale values of the second peripheral points N1-N36 and point P are calculated using Equation 3 above, namely, the absolute values of the grayscale differences AD1-AD36.
[0083] In step S243, the number of second peripheral points whose absolute value of the corresponding deviation is not greater than the fifth threshold is calculated.
[0084] For example, the fifth threshold ranges from 5 to 30. Preferably, the fifth threshold is 20. The absolute value of the statistical grayscale difference AD is not greater than the number n2 of the second peripheral points of the fifth threshold.
[0085] In step S244, the ratio of the number of second peripheral points n2 to the total number of second peripheral points N2 in the peripheral point set of the second scale is calculated as the second ratio K2.
[0086] For example, the first ratio K2 is calculated using Equation 5:
[0087]
[0088] In step S245, if the second ratio K2 is less than the sixth threshold, the point is identified as a possible texture point. The total number of second peripheral points is greater than the total number of first peripheral points.
[0089] For example, the sixth threshold is preferably 0.5. When K2 is less than 0.5, point P is identified as a possible texture point Pt(x, y).
[0090] Here, the third and fifth thresholds can be the same or different values. Similarly, the fourth and sixth thresholds can be the same or different values.
[0091] As mentioned above, if the different scale settings of various detection algorithms are not considered when identifying potential texture points in a target image, false detections may occur. For example, when using only the Susan-R1 point detection algorithm to identify potential texture points, points that can be identified as potential texture points by the Susan-R3 point detection algorithm may be falsely identified as non-potential texture points, and vice versa. In the embodiments of this application, for each point in the target image, the Susan-R1 point detection algorithm corresponding to a first scale with a radius of 1 pixel and the Susan-R3 point detection algorithm corresponding to a second scale with a radius of 3 pixels are used to perform two detections to identify potential texture points in the target image, so as to ensure the accuracy of candidate point identification.
[0092] In some embodiments, such as Figure 8 As shown in Examples 1 and 2 of this application, the Harris corner detection method for an image further includes step S25. In this detection method, the processor executes step S25 as follows:
[0093] In step 25, points in the target image that are not identified as possible texture points are excluded from the Harris corner detection process.
[0094] Therefore, in the Harris corner detection process, Harris corner detection is only performed on the texture points that are likely to be identified by the first and second detection algorithms. This can eliminate a large portion of the target image that are not likely to be Harris corners, thereby reducing the computational load when detecting Harris corners.
[0095] In some embodiments, such as Figure 9 As shown, the process of determining candidate neighborhoods based on the initially identified potential texture points includes steps S311 to S315. In Example 1 of this application, the fast8-5 point detection algorithm performs the following steps S311 to S315 based on each potential texture point:
[0096] In step S311, for each identified possible texture point, the consecutive first peripheral points of each group whose absolute value of the deviation from their grayscale value is greater than the first threshold are obtained.
[0097] For example, if the gray level difference Di in groups 1 to 3 in Table 1 all satisfy -t1 < Di < t1, for the possible texture point Pt(x, y), obtain the three first peripheral points corresponding to groups 1 to 3 in Table 1, namely points M1-M5, points M2-M6 and points M3-M7 respectively.
[0098] In step S312, for each group of consecutive first peripheral points, the deviation between each first peripheral point and the possible texture points is calculated, and the deviation with the smallest absolute value is taken as the first representative deviation of each group.
[0099] For example, the representative deviation of group 1 (points M1-M5) is calculated. The gray level differences D1-D5 between points M1-M5 and possible texture points Pt(x,y) are obtained and their absolute values are taken. Then, the gray level difference Di corresponding to the smallest absolute value is taken as the first representative deviation DL1 of group 1.
[0100] In step S313, the maximum value among the first representative deviations of each group is taken as the texture parameter value of the possible texture point.
[0101] For example, if the gray level difference Di in groups 1-3 in Table 1 all satisfy -t1<Di<t1, the first representative deviation DL1-DL3 of groups 1-3 is calculated by step S312, and the maximum value among them is taken as the texture parameter value Spt of the possible texture point Pt(x,y).
[0102] In step S314, the target image is divided into grids of a preset size, and the coordinates of each possible texture point are projected into each grid. If more than one possible texture point falls into each grid, the point with the largest texture parameter value is retained, thereby obtaining the texture interest point.
[0103] For example, such as Figure 10 As shown, the preset size L is set to 4 pixels, and the target image is divided into multiple L×L grids. Then, the coordinates of each possible texture point are projected into each grid. For example, a possible texture point Pt(x, y) is projected into the grid containing the point (x, y), and its projected coordinates are:
[0104] X = floor(x / L)
[0105] Y = floor(y / L)
[0106] If there is only one possible texture point in a raster, that possible texture point is taken as the texture focus of the raster; if there are multiple possible texture points in a raster, only the point with the largest texture parameter value among the multiple possible texture points is retained and taken as the texture focus of the raster, and the other possible texture points are excluded from the Harris corner detection process.
[0107] In step S315, the surrounding area of each texture interest point is used as a candidate neighborhood.
[0108] Specifically, selecting a region with a high probability of Harris corner detection and detecting Harris corners from points within that region helps improve the detection rate. Since the area near a texture interest point is a region with strong texture, and strong texture regions have a higher probability of detecting Harris corners compared to other regions, to further improve the detection rate, the region surrounding the texture interest point, determined through an expansion process, is used as a candidate neighborhood for Harris corner detection. Compared to performing Harris corner detection on all points in the target image, this reduces computational cost and complexity, thus ensuring the system's real-time performance.
[0109] In some embodiments, such as Figure 11 As shown, the process of determining candidate neighborhoods based on the initially identified potential texture points includes steps S321 to S325. In Example 1 of this application, the fast16-9 point detection algorithm performs the following steps S321 to S325 based on each potential texture point:
[0110] In step S321, for each identified possible texture point, the consecutive second peripheral points of each group whose absolute value of the deviation from its gray value is greater than the second threshold are obtained.
[0111] For example, if there are three sets of gray level differences Di (e.g., D1-D9, D2-D10 and D3-D11) in the continuous gray level difference set of the second peripheral points of Pt(x,y) and all satisfy -t2<Di<t2, then for the possible texture point Pt(x,y), obtain the three sets of second peripheral points (e.g., N1-N9, N2-N10 and N3-N11) corresponding to these three sets of continuous gray level difference sets.
[0112] In step S322, for each group of consecutive second peripheral points, the deviation between each second peripheral point and the possible texture points is calculated, and the deviation with the smallest absolute value is taken as the second representative deviation of each group.
[0113] For example, the representative deviation of the continuous gray-level difference set D1-D9 is calculated. The gray-level differences D1-D9 between points N1-N9 and possible texture points Pt(x,y) are obtained and their absolute values are taken. Then, the gray-level difference Di corresponding to the smallest absolute value is taken as the second representative deviation of the continuous gray-level difference set D1-D9.
[0114] In step S323, the maximum value among the second representative deviations of each group is taken as the texture parameter value of the possible texture point.
[0115] For example, if there are three sets of gray level differences Di (e.g., D1-D9, D2-D10 and point D3-D11) in the continuous gray level difference set of the second peripheral points of Pt(x,y) and all satisfy -t2<Di<t2, the second representative deviation of the three sets of continuous gray level difference sets is calculated by step S322, and the maximum value of the three second representative deviations is taken as the texture parameter value Spt of the possible texture point Pt(x,y).
[0116] In step S324, the target image is divided into grids of a preset size, and the coordinates of each possible texture point are projected into each grid. If more than one possible texture point falls into each grid, the point with the largest texture parameter value is retained, thereby obtaining the texture interest point.
[0117] For example, the preset size L is set to 4 pixels, and the target image is divided into multiple L×L grids. Then, the coordinates of each possible texture point are projected into each grid. For instance, a possible texture point Pt(x, y) is projected into the grid containing the point (x, y), and its projected coordinates are:
[0118] X = floor(x / L)
[0119] Y = floor(y / L)
[0120] If there is only one possible texture point in a raster, that possible texture point is taken as the texture focus of the raster; if there are multiple possible texture points in a raster, only the point with the largest texture parameter value among the multiple possible texture points is retained and taken as the texture focus of the raster, and the other possible texture points are excluded from the Harris corner detection process.
[0121] In step S325, the surrounding area of each texture interest point is used as a candidate neighborhood.
[0122] In some embodiments, such as Figure 12 As shown, the process of determining candidate neighborhoods based on the initially identified possible texture points includes steps S331 to S334. In Example 2 of this application, the Susan-R1 point detection algorithm performs the following steps S331 to S334 based on each possible texture point:
[0123] In step S331, a first ratio is obtained for each identified possible texture point.
[0124] For example, for the identified possible texture point Pt(x, y), obtain its corresponding first ratio K1.
[0125] In step S332, the difference between the first ratio and 1 is calculated and used as the texture parameter value of the possible texture point.
[0126] For example, the texture parameter value Spt of a possible texture point Pt(x, y) is calculated using Equation 6:
[0127] Spt = 1 - K1 … Equation 6
[0128] Where K1 is the first ratio K1 of possible texture points Pt(x,y).
[0129] In step S333, the target image is divided into grids of a preset size, and the coordinates of each possible texture point are projected into each grid. If more than one possible texture point falls into each grid, the point with the largest texture parameter value is retained, thereby obtaining the texture interest point.
[0130] For example, the preset size L is set to 4 pixels, and the target image is divided into multiple L×L grids. Then, the coordinates of each possible texture point are projected into each grid. For instance, a possible texture point Pt(x, y) is projected into the grid containing the point (x, y), and its projected coordinates are:
[0131] X = floor(x / L)
[0132] Y = floor(y / L)
[0133] If there is only one possible texture point in a raster, that possible texture point is taken as the texture focus of the raster; if there are multiple possible texture points in a raster, only the point with the largest texture parameter value among the multiple possible texture points is retained and taken as the texture focus of the raster, and the other possible texture points are excluded from the Harris corner detection process.
[0134] In step S334, the surrounding area of each texture interest point is used as a candidate neighborhood.
[0135] In some embodiments, such as Figure 13 As shown, the process of determining candidate neighborhoods based on the initially identified possible texture points includes steps S341 to S344. In Example 2 of this application, the Susan-R3 point detection algorithm performs the following steps S341 to S344 based on each possible texture point:
[0136] In step S341, a second ratio is obtained for each identified possible texture point.
[0137] For example, for the identified texture point Pt(x, y), obtain its corresponding second ratio K2.
[0138] In step S342, the difference between the second ratio and 1 is calculated to serve as the texture parameter value for the possible texture point.
[0139] For example, the texture parameter value Spt of a possible texture point Pt(x, y) is calculated using Equation 7:
[0140] Spt = 1 - K2 … Equation 7
[0141] Where K1 is the second ratio K2 of the possible texture points Pt(x,y).
[0142] In step S343, the target image is divided into grids of a preset size, and the coordinates of each possible texture point are projected into each grid. If more than one possible texture point falls into each grid, the point with the largest texture parameter value is retained, thereby obtaining the texture interest point.
[0143] For example, the preset size L is set to 4 pixels, and the target image is divided into multiple L×L grids. Then, the coordinates of each possible texture point are projected into each grid. For instance, a possible texture point Pt(x, y) is projected into the grid containing the point (x, y), and its projected coordinates are:
[0144] X = floor(x / L)
[0145] Y = floor(y / L)
[0146] If there is only one possible texture point in a raster, that possible texture point is taken as the texture focus of the raster; if there are multiple possible texture points in a raster, only the point with the largest texture parameter value among the multiple possible texture points is retained and taken as the texture focus of the raster, and the other possible texture points are excluded from the Harris corner detection process.
[0147] In step S344, the surrounding area of each texture interest point is used as a candidate neighborhood.
[0148] In some embodiments, the surrounding area is a 1×1 to 7×7 region around each texture point of interest. Preferably, the surrounding area is a 3×3 region around each texture point of interest.
[0149] Taking a 640x480 resolution VGA image as an example, the Harris corner detection method in this application, by setting the candidate neighborhood of a texture interest point as a 3×3 region around the point, only performs Harris score calculation for 9558 candidate points in the candidate neighborhood to perform Harris corner detection. This method reduces the complexity of NMS and can remove some fast points with weak textures. Furthermore, compared with the method of performing Harris corner detection on all 640x480 points in the target image, the computational cost is approximately 3% (9558 / (640x3480) ≈ 3%) of the detection method without pre-screening and using NMS.
[0150] In some embodiments of this application, an apparatus for detecting Harris corners of an image is provided, including a processor. The processor is configured to perform a method for detecting Harris corners of an image according to various embodiments of this application.
[0151] Specifically, the Harris corner detection device for the aforementioned image can perform pre-screening by repeatedly identifying points in the target image at multiple scales using a first detection algorithm such as fast8-5 or Susan-R1, and a second detection algorithm such as fast16-9 or Susan-R3, thereby obtaining potential texture points. Based on the obtained potential texture points, it performs expansion to determine candidate neighborhoods, and uses points in these neighborhoods as candidate points to select candidate points for Harris score calculation. Furthermore, it uses Non-Maximum Score (NMS) to select the point with the highest Harris score from the selected candidate points and identifies it as a Harris corner.
[0152] Taking a 640x480 resolution VGA image as an example, the detection device of this application acquires 9558 candidate points, and finally detects 1423 corner points. The proportion of Harris corner points among the candidate corner points is approximately 1423 / 9558. Compared with the OpenCV method, which directly calculates the Harris score for each point in the image and uses NMS to determine Harris corner points, the detection device of this application can remove a large number of non-corner points through pre-screening, thus reducing redundant calculations, lowering computational complexity, and ensuring the real-time performance of the system. For the same image, the number of corner points detected by the OpenCV method is 903, and the number of corner points present in the detection results of the OpenCV method but not detected by the detection device of this application is 72. That is to say, compared with the existing OpenCV method, the proportion of points lost by the detection device of this application is approximately 7.9% (72 / 903), meaning that most strong corner points are detected, ensuring the effectiveness of corner point detection.
[0153] In summary, the Harris corner detection device for images in this application reduces redundant computation during Harris corner detection while ensuring a high accuracy rate for candidate point recognition, meaning that most strong corners are detected. This reduces computational load and complexity, ensures system real-time performance, and also guarantees the final usability of the detected Harris corners.
[0154] Furthermore, although exemplary embodiments have been described herein, their scope includes any and all embodiments based on this disclosure that have equivalent elements, modifications, omissions, combinations (e.g., schemes involving intersections of various embodiments), adaptations, or alterations. Elements in the claims will be interpreted broadly based on the language used in the claims and are not limited to the examples described in this specification or during the implementation of this application, and such examples will be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered illustrative only, and the true scope and spirit are indicated by the full scope of the following claims and their equivalents.
[0155] The above description is intended to be illustrative and not restrictive. For example, the above examples (or one or more thereof) can be used in combination with each other. Other embodiments may be used by those skilled in the art upon reading the above description. Furthermore, in the above detailed description, various features may be grouped together to simplify this disclosure. This should not be construed as an intention that a disclosed feature, which is not claimed, is necessary for any claim. Rather, the subject matter of the invention may be less than all the features of a particular disclosed embodiment. Thus, the following claims are incorporated herein by reference as examples or embodiments, wherein each claim is independently considered as a separate embodiment, and these embodiments are contemplated as being possible in various combinations or arrangements. The scope of the invention should be determined by reference to the appended claims and the full scope of their equivalents.
Claims
1. A method for detecting Harris corners in an image, characterized in that, Includes, by the processor: Obtain the target image for Harris corner detection; Using a first detection algorithm at a first scale and a second detection algorithm at a second scale greater than the first scale, possible texture points are initially identified from the target image; Based on the initially identified possible texture points, at least one expansion is performed to determine candidate neighborhoods; Using the points in the candidate neighborhood as candidate points, calculate the Harris score for each candidate point; Among the candidate points, a predetermined number of candidate points with the highest Harris scores are extracted as Harris corner points using a non-maximum suppression method. The first detection algorithm includes, for each point of the target image: Obtain the grayscale values of the point and each of the first surrounding points of the surrounding point set at the first scale; Calculate the deviation between the gray value of each of the first peripheral points and the gray value of that point, or the absolute value of the deviation; Based on the deviation or the absolute value of the deviation, the point is identified as a possible texture point.
2. The detection method as described in claim 1, characterized in that, The first detection algorithm includes a fast 8-5 point detection algorithm, and the second detection algorithm includes a fast 16-9 point detection algorithm. The detected fast 8-5 points or fast 16-9 points are used as the identified possible texture points.
3. The detection method as described in claim 1, characterized in that, The first detection algorithm includes, for each point of the target image: Obtain the grayscale values of the point and each of the first surrounding points of the surrounding point set at the first scale; Calculate the deviation between the gray value of each of the first peripheral points and the gray value of that point; If the absolute value of the deviation corresponding to the first predetermined number of consecutive first peripheral points in any group is greater than the first threshold, then the point is identified as a possible texture point. The second detection algorithm includes, for each point in the target image: Obtain the grayscale values of each second surrounding point of the point and its surrounding point set at the second scale; Calculate the deviation between the grayscale value of each of the second peripheral points and the grayscale value of that point; If the absolute value of the deviation corresponding to a second predetermined number of consecutive second peripheral points in any group is greater than a second threshold, the point is identified as a possible texture point, wherein the second predetermined number is greater than the first predetermined number.
4. The detection method as described in claim 1, characterized in that, The first detection algorithm includes a Susan-R1 point detection algorithm, and the second detection algorithm includes a Susan-R3 point detection algorithm. The detected Susan-R1 points or Susan-R3 points are used as the identified possible texture points.
5. The detection method as described in claim 1, characterized in that, The first detection algorithm includes, for each point of the target image: Obtain the grayscale values of the point and each of the first surrounding points of the surrounding point set at the first scale; Calculate the absolute value of the deviation between the gray value of each of the first peripheral points and the gray value of that point; The number of first surrounding points whose absolute value of the corresponding deviation is not greater than the third threshold is calculated. Calculate the ratio of the number of the first peripheral points to the total number of the first peripheral points in the peripheral point set at the first scale, and use it as the first ratio. If the first ratio is less than the fourth threshold, the point is identified as a possible texture point; The second detection algorithm includes, for each point in the target image: Obtain the grayscale values of each second surrounding point of the point and its surrounding point set at the second scale; Calculate the absolute value of the deviation between the gray value of each of the second peripheral points and the gray value of that point; Calculate the number of second peripheral points whose absolute value of the corresponding deviation is not greater than the fifth threshold; Calculate the ratio of the number of the second perimeter points to the total number of the second perimeter points in the perimeter point set at the second scale, and use it as the second ratio. If the second ratio is less than the sixth threshold, the point is identified as a possible texture point, wherein the total number of the second surrounding points is greater than the total number of the first surrounding points.
6. The detection method as described in claim 3 or 5, characterized in that, Also includes: Points in the target image that are not identified as possible texture points are excluded from the Harris corner detection process.
7. The detection method as described in claim 3, characterized in that, Based on the initially identified possible texture points, at least one expansion is performed to determine the candidate neighborhood, specifically including: For each identified possible texture point, obtain the consecutive first peripheral points of each group whose absolute value of the deviation from its gray value is greater than the first threshold. For each group of consecutive first peripheral points, calculate the deviation between each first peripheral point and the possible texture points and take the deviation with the smallest absolute value as the first representative deviation of each group. The maximum value among the first representative deviations of each group is taken as the texture parameter value of the possible point of the texture. The target image is divided into grids of a preset size. The coordinates of each possible texture point are projected into each grid. If more than one possible texture point falls into each grid, the point with the largest texture parameter value is retained, thus obtaining the texture interest point. Use the surrounding area of each texture interest point as the candidate neighborhood.
8. The detection method as described in claim 3, characterized in that, Based on the initially identified possible texture points, at least one expansion is performed to determine the candidate neighborhood, specifically including: For each identified possible texture point, obtain the consecutive second peripheral points of each group whose absolute value of the deviation from its gray value is greater than the second threshold; For each group of consecutive second peripheral points, calculate the deviation between each second peripheral point and the possible texture points and take the deviation with the smallest absolute value as the second representative deviation of each group. The maximum value among the second representative deviations of each group is taken as the texture parameter value of the possible point of the texture. The target image is divided into grids of a preset size. The coordinates of each possible texture point are projected into each grid. If more than one possible texture point falls into each grid, the point with the largest texture parameter value is retained, thus obtaining the texture interest point. Use the surrounding area of each texture interest point as the candidate neighborhood.
9. The detection method as described in claim 5, characterized in that, Based on the initially identified possible texture points, at least one expansion is performed to determine the candidate neighborhood, specifically including: For each identified possible texture point, obtain the first ratio; Calculate the difference between the first ratio and 1, and use it as the texture parameter value for the possible point of the texture; The target image is divided into grids of a preset size. The coordinates of each possible texture point are projected into each grid. If more than one possible texture point falls into each grid, the point with the largest texture parameter value is retained, thus obtaining the texture interest point. Use the surrounding area of each texture interest point as the candidate neighborhood.
10. The detection method as described in claim 5, characterized in that, Based on the initially identified possible texture points, at least one expansion is performed to determine the candidate neighborhood, specifically including: For each identified possible texture point, obtain the second ratio; Calculate the difference between the second ratio and 1, and use it as the texture parameter value for the possible point of the texture; The target image is divided into grids of a preset size. The coordinates of each possible texture point are projected into each grid. If more than one possible texture point falls into each grid, the point with the largest texture parameter value is retained, thus obtaining the texture interest point. Use the surrounding area of each texture interest point as the candidate neighborhood.
11. The detection method according to any one of claims 7-10, characterized in that, The surrounding area refers to a 1×1 to 7×7 region around each texture point of interest.
12. A Harris corner detection device for an image, characterized in that, Includes a processor configured to perform a Harris corner detection method for an image according to any one of claims 1-11.