X corner detection method based on multilevel matching and feature optimization

Abstract

The invention relates to an X corner detection method based on multilevel matching and feature optimization, and belongs to the technical field of computer vision and image processing. The method comprises the following steps: making a high-precision template corresponding to a predetermined X corner mark, so that the high-precision template can be accurately focused on a corner area; carrying out noise reduction and filtering operations on the image before identifying the mark so as to remove noise interference; carrying out template matching on the preprocessed image by adopting a self-adaptive threshold value and a multi-scale search strategy; and performing multi-level detection on the basis of template matching, including ORB feature extraction, Harris corner optimization and sub-pixel level positioning, and outputting the accurate position of the corner and a feature descriptor. The method can still maintain high precision and stability in a complex scene, is suitable for the fields of visual positioning, three-dimensional reconstruction and the like, and has the advantages of high detection efficiency, strong anti-interference capability, wide applicability and the like.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and image processing, and particularly relates to an X-corner detection method based on multi-level matching and feature optimization. Background Art

[0002] Corner detection is a key link in the field of computer vision, and it plays a fundamental and irreplaceable role in tasks such as image feature matching, target recognition, visual navigation, and image analysis and understanding. Corners are points with significant local features in an image, usually located at the intersections of object edges or areas with drastic texture changes, and can provide key structural information for image processing and analysis. However, despite the significant progress made in corner detection technology, existing methods still face many challenges in practical applications. Especially when dealing with complex scenes, noise interference, illumination changes, and scale changes, etc., their detection accuracy, computational efficiency, and robustness will be greatly affected, making it difficult to meet the growing practical needs.

[0003] Traditional corner detection methods such as Harris corner detection, SIFT, and SURF, although performing well in certain specific scenarios, still have obvious limitations. Specifically, the Harris corner detection method is based on the second-order moment matrix of the image gray gradient. It can effectively detect corners, but is sensitive to noise, with insufficient anti-noise performance, and is difficult to effectively handle dynamic environments such as illumination changes and occlusions; the SIFT and SURF algorithms introduce a scale space and feature descriptors, and have a certain degree of anti-noise and scale invariance, but their computational complexity is relatively high, and the processing speed is slow, making it difficult to meet application scenarios with high real-time requirements. In addition, although the algorithm has an advantage in computational efficiency, its detection accuracy is relatively low. Especially under the conditions of large noise or unclear image details, the accuracy and reliability of the detection results will decrease significantly.

[0004] In addition, with the wide application of computer vision technology in fields such as industrial automation, autonomous driving, and robot navigation, higher requirements are put forward for the accuracy and efficiency of corner detection. For example, in the field of industrial automation, high-precision corner detection is a key technology for realizing the positioning and assembly of precision parts, directly affecting production efficiency and product quality. In the field of autonomous driving, corner detection technology provides important support for vehicle visual positioning and obstacle recognition, and is a core link to ensure driving safety. In the field of robot navigation, corner detection technology is the basis for realizing path planning and environmental perception, and can help robots accurately perceive the surrounding environment and make reasonable decisions. Therefore, developing a corner detection method that can achieve fast, accurate, and robust detection in complex scenarios not only has important theoretical value but also has broad practical application significance. Summary of the Invention

[0005] The object of the present invention is to provide an X corner detection method based on multi-level matching and feature optimization in view of the deficiencies of existing corner detection technologies in practical applications. By organically combining the advantages of different algorithms, the limitations of a single algorithm are made up for, so as to achieve corner detection with higher accuracy, faster speed and stronger robustness in complex scenarios.

[0006] To achieve the above object, the technical solution of the present invention is: an X corner detection method based on multi-level matching and feature optimization, including:

[0007] Step S1: Mark the predetermined X corners, and construct a corresponding template image based on the actual features and size parameters of the marks. Through preprocessing operations such as background blurring and corner edge sharpening, ensure that the intersection features in the template are clearly distinguishable, providing clear features for subsequent matching;

[0008] Step S2: Perform Gaussian filtering and noise reduction processing on the actual marked image collected by the binocular camera to remove noise and unnecessary details in the image, while retaining important edge information. This preprocessing step can improve the input quality of subsequent feature extraction, corner detection and matching algorithms, enhancing the detection accuracy and robustness of the system;

[0009] Step S3: Based on the preprocessed image data, use the template matching algorithm to search for the region with the highest similarity to the template. By calculating the similarity metric between the template and the target region, determine the best matching position, providing an initial reference for subsequent precise positioning;

[0010] Step S4: Based on the best matching region of the template, use the ORB (Oriented FAST and Rotated BRIEF) algorithm to detect feature points; this algorithm combines FAST key point detection and BRIEF descriptor, with the characteristics of rotation invariance and high computational efficiency;

[0011] Step S5: Based on the corners detected in Step S4, further optimize using the Harris corner detection algorithm; by setting a response threshold, screen out significant corner features, effectively eliminating false corners and improving the reliability of corner detection;

[0012] Step S6: Based on the corners filtered in Step S5, perform sub-pixel level optimization positioning. Use the cv2.cornerSubPix function, and combine the search window size, zero region and iteration termination conditions to iteratively optimize the positioning of each corner, improving the corner positioning accuracy to the sub-pixel level.

[0013] In an embodiment of the present invention, the specific content of Step S1 is:

[0014] According to the actual features and size of the marker, a template image corresponding to the marker is constructed. To ensure that the template contains clear intersection features, the present invention uses Gaussian blur to achieve background blurring and Laplace operator to achieve corner edge sharpening and other preprocessing operations to optimize the quality of the template image.

[0015] The background of the template image is blurred through Gaussian blur. Gaussian blur is a commonly used image smoothing technique that can effectively reduce the noise in the image while retaining important structural information. The formula for Gaussian blur is:

[0016]

[0017] where I(x′, y′) is the pixel value of the input image at the coordinate (x′, y′), x′, y′ are the integration variables representing the coordinates of all pixels participating in the calculation, σ is the standard deviation of the Gaussian kernel, and I blurred (x, y) is the image after Gaussian blur processing. In the present invention, the size of the Gaussian kernel is 5×5 and the standard deviation is σ = 1.5.

[0018] The corner edges of the template image are sharpened through the Laplace operator, which can enhance the contrast of the corner edges in the image, thereby improving the recognition ability of the template. The second derivative of the image by the Laplace operator is usually realized through convolution, and its formula is:

[0019] I sharpened (x,y) = I(x,y) + Δ 2 I(x,y)

[0020] where Δ 2 I(x, y) is the second derivative of the image by the Laplace operator, usually realized through convolution:

[0021]

[0022] The matrix in the above formula calculates the difference between the central pixel and its four adjacent pixels (up, down, left, right), thereby enhancing the edge.

[0023] In an embodiment of the present invention, the step S2 is specifically:

[0024] The actual marker image collected by the binocular camera is subjected to Gaussian filtering and noise reduction processing. The noise and redundant details in the image are removed through Gaussian filtering, while retaining important edge information, providing a clearer input for subsequent feature extraction, corner detection, and matching algorithms.

[0025]

[0026] where G(x, y) is the Gaussian kernel function, σ is the standard deviation, and x and y are pixel coordinates.

[0027] In one embodiment of the present invention, step S3 is specifically as follows:

[0028] Perform template matching on the preprocessed image data to obtain the region with the highest matching degree with the template in the marks within the camera's field of view. By adopting an adaptive threshold and a multi-scale search strategy, the template image is slid on the target image, and the normalized cross-correlation variance matching (NCC) is used to calculate the similarity between each position in the target image and the template image. Furthermore, the matching regions with a similarity higher than the threshold are screened out by setting a threshold.

[0029] The calculation formula of the normalized cross-correlation variance matching (NCC) is:

[0030]

[0031] In the formula, R(x, y) is the matching result, T represents the template image, I represents the original image, (x, y) represents the horizontal and vertical coordinates of the pixel position in the original image, and (x′, y′) represents the horizontal and vertical coordinates in the template image. This algorithm normalizes the matching result coefficient to -1 to 1. When the template is exactly the same as the search region in the original image, the calculation result is 1, and when it is completely different, the calculation result is -1.

[0032] The multi-scale search strategy is to perform NCC matching at different zoom levels:

[0033]

[0034] where T s is the scaled template, and n represents the zoom level. This method ensures the robustness of the matching, especially when the size of the target image is uncertain.

[0035] Meanwhile, an adaptive threshold strategy is adopted to screen the optimal matching region:

[0036] θ = λ·max(R)

[0037] where R is the similarity response matrix calculated by NCC, λ is the dynamic adjustment factor, and θ is the finally screened similarity threshold.

[0038] In one embodiment of the present invention, step S4 is specifically as follows:

[0039] Based on the template matching region in step S3, use the ORB (Oriented FAST and Rotated BRIEF) algorithm for feature point detection. Create an ORB detector by setting a series of parameters, and use its detectAndCompute method to calculate the feature points and their descriptors for the image in the matching region to achieve efficient feature point detection.

[0040] In one embodiment of the present invention, step S5 is specifically as follows:

[0041] Based on the ORB detection results, the Harris corner detection algorithm is used for optimization to further improve the accuracy and reliability of corner detection. First, for the image within the template matching region, its Harris corner response map is calculated. The core of the Harris corner detection algorithm is to judge corners by calculating the gradient changes of the image in various directions. To enhance the corner response, the response map is processed through a dilation operation. The dilation operation can replace each pixel value with the maximum value within the neighborhood, thereby highlighting the local maximum points. Subsequently, a threshold is set to filter out the pixel points with response values higher than the threshold as Harris corners, effectively removing false corners and improving the reliability of corner detection.

[0042] R = det(M) - k(trace(M)) 2

[0043] The above formula is the definition of the corner response function, where det(M) and trace(M) are the determinant and trace of the matrix respectively, M is the autocorrelation matrix of the image, and is defined as:

[0044]

[0045] where I x and I y are the gradients of the image in the x and y directions respectively, w(x, y) is the window weight function, k is an empirical constant, usually taken between 0.04 and 0.06, and here 0.04 is selected.

[0046] In one embodiment of the present invention, step S6 is specifically as follows:

[0047] Based on the Harris corners filtered in step S5, sub-pixel level optimization is further performed on them. Using the cv2.cornerSubPix function in OpenCV, by setting the search window size, zero region, and iteration termination conditions, the corner positions are iteratively optimized, and finally sub-pixel level precise positioning is achieved. Its iterative formula is:

[0048] P new = P old + ΔP

[0049] where P new and P old are the corner coordinates before and after iterative update respectively, and ΔP is the displacement vector obtained by minimizing the error function. The error function is usually:

[0050]

[0051] Wherein, Ω is the search window, I(x, y) is the pixel value of the image I at the coordinate (x, y), and P represents the current corner position.

[0052] The present invention also provides an X corner detection system based on multi-level matching and feature optimization, including:

[0053] A template making module, which makes a corresponding high-precision template for the predetermined X corner mark, so that it can accurately focus on the corner area;

[0054] An image preprocessing module, which performs operations including noise reduction and filtering on the image before identifying the mark to remove noise interference;

[0055] A template matching module, which performs template matching on the preprocessed image by using an adaptive threshold and a multi-scale search strategy;

[0056] A feature point detection module, which performs multi-level detection on the basis of template matching, including ORB feature extraction, Harris corner optimization and sub-pixel level positioning, and outputs the accurate position and feature descriptor of the corner.

[0057] The system executes the method steps as described in any of the above.

[0058] Compared with the prior art, the present invention has the following beneficial effects: The present invention adopts sub-pixel level positioning, multi-scale and rotation invariant feature extraction, and optimized feature point screening, which significantly improves the detection accuracy and noise resistance; at the same time, through the efficient ORB algorithm and the adaptive template matching strategy, the detection speed is greatly improved, and stable performance can be maintained in complex scenarios such as complex backgrounds, noise interference, illumination changes and scale changes. In addition, this method is applicable to multiple fields such as visual positioning, 3D reconstruction, and image calibration, and has broad practicability and innovation, providing an efficient and reliable corner detection solution for related application scenarios. Description of the Drawings

[0059] Figure 1 It is a flowchart of the method of the present invention. Detailed Embodiments

[0060] Next, in combination with the drawings, the technical solutions of the present invention will be specifically described.

[0061] Glossary:

[0062] X corner: The international chessboard for visual calibration is composed of regions with sudden changes in black and white colors. Among them, the critical points where adjacent black and white chessboard squares meet are the X corners.

[0063] Please refer to Figure 1, the present invention provides an X-corner detection method based on multi-level matching and feature optimization, including the following steps:

[0064] Step S1: Use the original X-corner image as input, perform background blurring on the template image through Gaussian blur to reduce the interference of the background and highlight the corner features. Then, use the Laplace operator to perform corner edge sharpening on the template image to further enhance the visibility of the corners. Through these two image processing techniques, the background of the generated template image is effectively weakened, while the corners are significantly strengthened, thus providing a high-quality basis for subsequent template matching. And in order to adapt to the sizes and features of different target images, the template ratio size can be dynamically adjusted. Specifically, by calculating the size ratio between the target image and the template image, the scaling ratio of the template image is determined, and the size of the template image is adjusted according to this ratio, so as to automatically focus on the interested X-corner area during template matching, in order to reduce the interference of surrounding corners during subsequent corner detection and improve the recognition accuracy.

[0065] Step S2: Perform noise reduction processing on the images in the binocular camera's field of view with a Gaussian filter kernel size of 5×5 and a standard deviation of σ = 1.5 to remove the noise and redundant details in the images and improve the image quality. This process reduces the influence of noise by smoothing the image while retaining important edge information, providing a clearer input for subsequent feature extraction, corner detection, and matching algorithms.

[0066] Step S3: Use the template matching method to locate the preliminary corner area in the image. Adopt an adaptive threshold and a multi-scale search strategy to slide the template image on the target image, use the normalized cross-correlation variance matching (NCC) to calculate the similarity between each position in the target image and the template picture, and obtain the most likely matching position (i.e., max_loc) in the image. The matching position defines a rectangular area (from top_left to bottom_right) through the size (h, w) of the template, and further extracts the possible corner area in the image.

[0067]

[0068] The above formula is the calculation formula for the normalized cross-correlation variance matching (NCC). In the formula, R(x, y) is the matching result, T represents the template image, I represents the original image, (x, y) represents the horizontal and vertical coordinates of the pixel position in the original image, and (x, y) represents the horizontal and vertical coordinates in the template image. This algorithm normalizes the matching result coefficient to -1 to 1. When the template is exactly the same as the search area in the original image, the calculation result is 1, and when it is completely different, the calculation result is -1.

[0069] The multi-scale search strategy is to perform NCC matching at different zoom levels:

[0070]

[0071] Among them, T s is the scaled template, and n represents the scaling level. This method ensures the robustness of the matching, especially in the case where the size of the target image is uncertain.

[0072] Meanwhile, an adaptive threshold strategy is adopted to screen the optimal matching region:

[0073] θ = λ·max(R)

[0074] where R is the similarity response matrix calculated by NCC, λ is the dynamic adjustment factor, and θ is the finally screened similarity threshold.

[0075] Step S4: Use the ORB (Oriented FAST and Rotated BRIEF) algorithm to extract feature points from the image within the template matching region in step S3. Create an ORB detector by setting parameters nfeatures = 100, scaleFactor = 1.2, edgeThreshold = 15, patchSize = 31, and use its detectAndCompute method to calculate the feature points and their descriptors for the matching region image. The ORB algorithm combines the characteristics of multi-scale and rotation invariance, and can stably extract feature points when the scale and angle change, providing an effective local feature expression for subsequent operations such as feature point screening and matching.

[0076] Step S5: In order to improve the quality of the feature points, the present invention uses the Harris corner detection algorithm to screen the feature points extracted by ORB in step S4. First, calculate the Harris corner response map for the image within the template matching region, and enhance the corner response through dilation operation; then, set the threshold to 1% of the maximum value of the response map, and screen out the pixel points with response values higher than the threshold as Harris corners. This process effectively excludes redundant and low-contrast feature points and retains the pixel points with significant corner characteristics.

[0077] Based on the ORB detection results, the Harris corner detection algorithm is used for optimization to further improve the accuracy and reliability of corner detection. First, for the image within the template matching region, calculate its Harris corner response map. The core of the Harris corner detection algorithm is to judge corners by calculating the gradient changes of the image in various directions. In order to enhance the corner response, the response map is processed through dilation operation. The dilation operation can replace each pixel value with the maximum value in the neighborhood, thus highlighting the local maximum points. Subsequently, set the threshold, and screen out the pixel points with response values higher than the threshold as Harris corners, effectively removing false corners and improving the reliability of corner detection.

[0078] R = det(M) - k(trace(M)) 2

[0079] The above formula is the definition of the corner response function, where det(M) and trace(M) are the determinant and trace of the matrix respectively, and M is the autocorrelation matrix of the image, which is defined as:

[0080]

[0081] where I x and I y are the gradients of the image in the x and y directions respectively, w(x, y) is the window weight function, and k is an empirical constant, usually taken between 0.04 and 0.06. Here, 0.04 is selected.

[0082] Step S6: The present invention performs sub-pixel localization on the Harris corners screened in step S5 to further improve the corner localization accuracy. After converting the Harris corner coordinates to floating-point type, the cv2.cornerSubPix function is used, combined with the search window size (11, 11), zero region (-1, -1), and iteration termination condition (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.1), to perform iterative optimization localization on each corner. Through the gradient information of the image, the corner position is accurately adjusted to achieve sub-pixel localization accuracy, effectively reducing the corner localization error caused by image blurring, noise, or insufficient accuracy. Especially in high-noise and complex environments, it can still maintain high accuracy, significantly improving the accuracy and reliability of the entire corner detection system.

[0083] Step S7: Finally, the detected corner positions and corresponding feature descriptors are output for subsequent image processing and analysis.

[0084] The present invention also provides an X corner detection system based on multi-level matching and feature optimization, including:

[0085] A template making module that makes a corresponding high-precision template for the predetermined X corner mark so that it can accurately focus on the corner area;

[0086] An image preprocessing module that performs operations including noise reduction and filtering on the image before identifying the mark to remove noise interference;

[0087] A template matching module that performs template matching on the preprocessed image using an adaptive threshold and a multi-scale search strategy;

[0088] The feature point detection module performs multi-level detection based on template matching, including ORB feature extraction, Harris corner optimization, and sub-pixel positioning, and outputs the exact positions of the corner points and the feature descriptors.

[0089] The system executes the method steps as described in any one of the above.

[0090] The above are the preferred embodiments of the present invention. All changes made according to the technical solution of the present invention, when the functions and effects produced do not exceed the scope of the technical solution of the present invention, shall fall within the protection scope of the present invention.

Claims

1. An X-corner detection method based on multi-level matching and feature optimization, characterized in that Including: Making a corresponding high-precision template for a predetermined X corner mark so that it can accurately focus on the corner area; Performing operations including noise reduction and filtering on the image before identifying the mark to remove noise interference; Adopting an adaptive threshold and multi-scale search strategy to perform template matching on the preprocessed image; Performing multi-level detection on the basis of template matching, including ORB feature extraction, Harris corner optimization, and sub-pixel level positioning, and outputting the accurate position and feature descriptor of the corner.

2. The X-corner detection method based on multi-level matching and feature optimization according to claim 1, wherein Making a corresponding high-precision template for a predetermined X corner mark so that it can accurately focus on the corner area. Specifically, that is: optimizing the template quality through techniques including background blurring and corner edge sharpening, and dynamically adjusting the scale of the template according to the target image to ensure that the matching area can be automatically and correctly identified during the template matching process to reduce peripheral interference.

3. A method for detecting X corner points based on multi-level matching and feature optimization according to claim 1, characterized in that, Using the Gaussian filtering method to perform noise reduction and filtering processing on the image to remove noise interference while retaining key edge information.

4. A method for detecting X corner points based on multi-level matching and feature optimization according to claim 1, characterized in that, Adopting an adaptive threshold and multi-scale search strategy, sliding the template image on the target image, using the normalized cross-correlation variance matching NCC algorithm to calculate the similarity between each position in the target image and the template image, screening out the most likely matching position by setting the similarity threshold, and defining a rectangular matching area according to the template size.

5. The X-corner detection method based on multi-level matching and feature optimization according to claim 4, wherein The calculation formula of the normalized cross-correlation variance matching NCC algorithm is as follows: In the formula, R(x, y) is the matching result, T represents the template image, I represents the original image, (x, y) represents the horizontal and vertical coordinates of the pixel position in the original image, and (x′, y′) represents the horizontal and vertical coordinates in the template image; the normalized cross-correlation variance matching NCC algorithm normalizes the matching result coefficient to -1 to 1. When the search area in the template image is exactly the same as that in the original image, the calculation result is 1, and when it is completely different, the calculation result is -1; The multi-scale search strategy is to perform NCC matching at different zoom levels: Among them, T s is the scaled template, and n represents the scaling level; At the same time, adopting an adaptive threshold strategy to screen the optimal matching area: θ = λ·max(R) Where, R is the similarity response matrix calculated by NCC, λ is the dynamic adjustment factor, and θ is the finally screened similarity threshold.

6. The X-corner detection method based on multi-level matching and feature optimization according to claim 4, wherein, Using the ORB feature detection algorithm to extract the feature points of the matching area, creating an ORB detector by setting relevant parameters, and using the detectAndCompute method to calculate the feature points and their descriptors.

7. A method for detecting X corner points based on multi-level matching and feature optimization according to claim 6, characterized in that, Adopting the Harris corner detection algorithm to optimize and screen the feature points extracted by the ORB feature detection algorithm. By calculating the Harris corner response map within the template matching area and performing a dilation operation on it to enhance the corner response, and then screening out significant corners by setting a threshold to remove redundant and low-contrast feature points.

8. A method for detecting X corner points based on multi-level matching and feature optimization according to claim 7, characterized in that Performing sub-pixel level positioning optimization on the screened Harris corners. Through a high-precision positioning algorithm, accurately adjusting the corner position with the image gradient information to make it accurate to the sub-pixel level. The entire process completes the X corner detection, and finally outputs the accurate position and feature descriptor of the detected corner.

9. The X-corner detection method based on multi-level matching and feature optimization according to claim 8, characterized in that Using the cv2.cornerSubPix function in OpenCV, by setting the search window size, zero zone, and iteration termination conditions, the positions of the filtered Harris corner points are iteratively optimized to finally achieve sub-pixel level precise positioning; its iterative formula is: P new = P old + ΔP where P new and P old are the corner coordinates before and after iterative update respectively, and ΔP is the displacement vector obtained by minimizing the error function; the error function is: Among them, Ω is the search window, I(x, y) is the pixel value of the image I at the coordinate (x, y), and P represents the current corner point position.

10. An X-corner detection system based on multi-level matching and feature optimization, characterized in that, It includes: A template making module that makes a corresponding high-precision template for the predetermined X corner mark so that it can accurately focus on the corner area; An image preprocessing module that performs operations including noise reduction and filtering on the image before identifying the mark to remove noise interference; A template matching module that performs template matching on the preprocessed image using an adaptive threshold and a multi-scale search strategy; A feature point detection module that performs multi-level detection based on template matching, including ORB feature extraction, Harris corner optimization, and sub-pixel level positioning, and outputs the precise position and feature descriptor of the corner points.

Images