Weak texture feature matching method and system

Abstract

This invention relates to the field of feature detection technology and discloses a weak texture feature matching method and system. The method utilizes information entropy to detect weak texture regions, calculates the homography matrix of the weak texture regions by extracting the edge lines of these regions, and then calculates the correspondence between any points within the weak texture region based on the homography matrix. This invention solves the problem in existing technologies where effective feature points cannot be extracted from textureless or weak texture regions, leading to holes in the weak texture regions in the reconstructed model.

Description

Technical Field

[0001] This invention relates to the field of feature detection technology, specifically a weak texture feature matching method and system. Background Technology

[0002] Feature detection's core idea is to find unique, stable, and repeatable feature points in key regions of an image. Therefore, these are typically points with significant grayscale changes, certain geometric characteristics, or topological structures, making them highly discriminative. Initially, corner features were identified based on gradient recognition, lacking orientation and rotation invariance. The well-known Harris corner detection method was proposed to address anisotropy and computational complexity by using a second-order matrix to find the two directions of fastest and slowest grayscale changes, thus achieving orientation and illumination invariance and providing more reliable repeatability and discriminability. Later, comparison-based corner detection methods were proposed, simplifying gradient calculation by comparing the brightness of the center pixel with surrounding pixels. Leveraging its binary nature, it has been widely used in applications with limited storage and real-time requirements. The most famous example is FAST corner detection. To achieve both the efficiency of FAST and the reliability of Harris, Rublee et al. proposed an ORB (Object-Oriented Base) for matching, which includes feature detection and descriptor generation. The ORB selects a certain number of FAST corners based on the Harris response, and the main direction of the ORB is represented by the line connecting the center of a local block and its grayscale centroid. Multi-scale corner detector (MSFD) based on segmentation has been proposed for wide baseline matching and reconstruction. MSFD feature points are detected by identifying the intersections of three or more region edges using existing segmentation methods. MSFD can generate rich and accurate corner features. Blob features typically represent a closed region with relatively uniform internal pixels, distinct from its neighborhood. They can be broadly categorized into two types: those based on second-order partial derivatives and those based on segmentation. In methods based on second-order partial derivatives, a Gaussian Laplace function (LoG) is used in the scale space. Gaussian convolution is used for preprocessing to reduce noise. Different standard deviations can be used to detect scale-invariant blobs by searching for extrema in the multi-scale space. DoG can approximate LoG filtering, accelerating LoG computation. Another classic blob detection strategy is based on the Hessian matrix DoH determinant, which has better radiometric invariance. Feature point detection using DoG and DoH is widely applied. The well-known SIFT extracts local extrema points from the DoG pyramid as feature points and uses a Hessian matrix for filtering. SURF simplifies the template for second-order partial derivatives by approximating the Hessian matrix using Haar wavelets on the integral image, thus accelerating SIFT. ASIFT proposes affine-invariant SIFT, improving efficiency by approximating the Laplacian calculation through bilateral filtering. DARTs uses piecewise triangular filtering to effectively approximate DoH. Unlike circular Gaussian functions, KAZE uses nonlinear partial differential equations and nonlinear diffusion filtering for blob feature detection. Its accelerated version, AKAZE, embeds fast explicit diffusion within a pyramid framework to accelerate nonlinear scale-space feature detection. MSER is one of the most well-known segmentation-based blob detectors; this method does not require additional processing for scale estimation and is robust to large viewpoint changes.Kimmel et al. extended MSER using shape and structure cues. This type of feature detection is rarely used for feature matching and is gradually evolving towards saliency detection and segmentation in computer vision.

[0003] Existing feature point extraction and matching methods typically rely on local gradient information in images. For example, algorithms such as SIFT, SURF, and ORB all extract feature points based on local gradient information, focusing on points with drastic changes in grayscale values, certain geometric characteristics, or topological structures. While these methods can extract a large number of high-quality feature points in strongly textured regions, they struggle to extract effective feature points in weakly textured regions, leading to holes in weakly textured areas during subsequent SFM 3D reconstruction.

[0004] The main reasons are as follows: 1. Low gradient values: In weakly textured regions, the gradient values ​​of the image are usually very small, even close to zero. Traditional feature point extraction algorithms require a certain gradient value threshold to be considered a valid feature point, but the gradient values ​​in weakly textured regions often fail to meet this threshold requirement, thus failing to extract the feature points. 2. Instability: In weakly textured regions, the texture information in the image is very blurry and easily affected by noise and interference, resulting in the inability to extract a sufficient number of effective feature points. 3. Repetitiveness: In weakly textured regions, the texture information in the image is often very simple and repetitive, reducing the distinguishability and uniqueness of the feature points.

[0005] Image-based 3D reconstruction is a core problem in the field of 3D reconstruction, aiming to recover the 3D structure of a scene from 2D images from multiple perspectives. Feature point extraction and matching are key steps; however, existing methods cannot extract effective feature points in textureless or weakly textured regions, resulting in holes in weakly textured areas in the reconstructed model. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies, this invention provides a weak texture feature matching method and system, which solves the problem that existing technologies cannot extract effective feature points in areas with no texture or weak texture, resulting in holes in weak texture areas in the reconstructed model.

[0007] The technical solution adopted by the present invention to solve the above problems is:

[0008] A weak texture feature matching method is proposed. It uses information entropy to detect weak texture regions, extracts the edge lines of the weak texture regions to calculate the homography matrix of the weak texture plane, and then calculates the correspondence of any point in the plane based on the homography matrix.

[0009] As a preferred technical solution, the steps include:

[0010] S1, Calculate the fundamental matrix: Calculate the fundamental matrix F between two images, denoted as image A and image B respectively;

[0011] S2, Weak texture region detection: Use information entropy to distinguish weak texture regions in image A and image B;

[0012] S3, Edge line matching: Extract the edge lines of the weak texture region from images A and B respectively, and match the extracted edge lines;

[0013] S4, Mismatch Removal: Remove mismatched edge lines from images A and B respectively;

[0014] S5, Homography Matrix Calculation: Calculate the homography matrix H between the weak texture regions in image A and the weak texture regions in image B.

[0015] As a preferred technical solution, step S2 includes the following steps:

[0016] S21, calculate the information entropy of image A and image B with each pixel as the center in a window of a set size, and obtain the information entropy map corresponding to image A and image B;

[0017] S22, combine the difference between the maximum and minimum pixel values ​​in the window of the set size to jointly determine the probability that the pixels in this window are weak textures;

[0018] S23, adjust the information entropy of images A and B according to the set threshold. Figure 2 Value-enhanced mapping involves marking weak texture regions as 1 and strong texture regions as 0 to obtain a binary image.

[0019] As a preferred technical solution, step S3 includes the following steps:

[0020] S31, perform EDlines line detection on the binary images corresponding to image A and image B respectively, and extract the edge lines of the weak texture region;

[0021] S32, calculate the intersection point between any two edge lines in image A, and calculate the intersection point between any two edge lines in image B;

[0022] S33, Match the weak texture edge lines extracted from image A with the weak texture edge lines extracted from image B using the LBD line descriptor.

[0023] As a preferred technical solution, step S4 includes the following steps:

[0024] S41, If ​​two edge lines in image A match two edge lines in image B respectively, then the intersection of the two edge lines in image A and the intersection of the two edge lines in image B are the matching points.

[0025] S42 uses the property that the matching points satisfy the constraints of the fundamental matrix F to back-verify the correctness of the edge line matching, filters out the edge lines with incorrect matching, and obtains multiple lines describing the weak texture region in images A and B.

[0026] As a preferred technical solution, step S5 includes the following steps:

[0027] S51, calculate the homography matrix H between the weak texture regions in image A and image B by selecting straight line pairs of weak texture regions;

[0028] S52, calculates the matching relationship between any point in the weak texture region in image A and the weak texture region in image B from different perspectives using H.

[0029] As a preferred technical solution, in step S1, the feature points extracted by SIFT are used for matching to calculate the fundamental matrix F between image A and image B.

[0030] As a preferred technical solution, in step S1, the image originates from one or more of the following scenarios: object tracking, road and lane line recognition and tracking, and traffic sign and marking recognition and tracking.

[0031] A weak texture feature matching system, used to implement the aforementioned weak texture feature matching method, comprises the following modules connected in sequence:

[0032] Fundamental Matrix Calculation Module: Used to calculate the fundamental matrix F between two images, denoted as image A and image B respectively;

[0033] Weak texture region detection module: used to distinguish weak texture regions in image A and image B using information entropy;

[0034] Edge line matching module: used to extract edge lines of weak texture regions in images A and B respectively, and to match the extracted edge lines;

[0035] Mismatch Removal Module: Used to filter out mismatched edge lines in images A and B respectively;

[0036] Homography matrix calculation module: Used to calculate the homography matrix H between weak texture regions in image A and weak texture regions in image B, and then calculate the correspondence between any points in the weak texture region based on the homography matrix.

[0037] Compared with the prior art, the present invention has the following advantages:

[0038] (1) This invention mainly solves the problem of holes in weak texture regions in three-dimensional reconstruction models. It proposes a feature matching method based on planar priors, which can add enough effective feature points to weak texture regions, thereby enabling the weak texture regions to be effectively reconstructed.

[0039] (2) This invention is a flexible mechanism that can be effectively combined with existing feature extraction and matching methods, making it highly applicable and performant under different computing resources and application scenarios. Attached Figure Description

[0040] Figure 1 This is a flowchart of a weak texture feature matching method according to the present invention;

[0041] Figure 2 This is a schematic diagram illustrating the method for calculating the homography matrix H.

[0042] Figure 3 for Figure 2 One of the magnified views of a section;

[0043] Figure 4 for Figure 2 The second enlarged view of a part. Detailed Implementation

[0044] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0045] Example 1

[0046] like Figures 1 to 4 As shown, this invention aims to leverage the prior knowledge that most weakly textured regions in nature are planar, proposing a weakly textured feature matching method based on planar priors. This method extracts a sufficient number of effective correspondences within weakly textured planar regions, addressing the problem of hollowness in weakly textured region reconstruction models. The planar prior refers to the assumption that most weakly textured objects in reality are man-made regular objects, such as walls and tabletops; therefore, we assume that weakly textured regions in a scene are planar. Point-to-point matching of weakly textured regions is achieved by utilizing the characteristic that point-to-point mapping between planes can be described by a homography matrix H.

[0047] This invention aims to utilize the prior knowledge of weakly textured regions in nature, which are usually man-made planes such as roads, tiles, tables, and wooden boards, to calculate the homography matrix of the plane, calculate the correspondence of each point on the plane, and extract a sufficient number of effective matching points, thereby solving the problem of holes in weakly textured regions of 3D reconstruction models.

[0048] Observation revealed that the reconstructed scenes generally contain both weak texture areas and texture areas, and are mostly flat surfaces such as walls and tabletops. Therefore, we simplified the problem to the premise that "the weak texture area is a plane". Under the premise of a plane, as long as we can match any four non-collinear points on the plane, we can obtain the matching point of any point on the plane.

[0049] Motivated by this, and addressing the challenge of feature extraction and matching in weakly textured regions, a feature extraction and matching algorithm for weakly textured images based on shape prior is proposed. The overall process is as follows: Figure 1 As shown. The aim is to increase the number of matching points in areas with weak textures.

[0050] (1) Calculate the fundamental matrix F between the two images. First, the fundamental matrix F between the two images is calculated by matching the feature points extracted by SIFT, which can provide a constraint for the subsequent matching of weak texture regions.

[0051] (2) Weak Texture Region Detection. Weak texture regions contain little information. We use information entropy to distinguish weak texture regions in an image. More specifically, we first calculate the information entropy of a 3*3 window centered on each pixel in the image, and then combine the difference between the maximum and minimum pixel values ​​in the window to determine the probability that a pixel in this window is a weak texture. Then, we apply a threshold to the information entropy. Figure 2 Values ​​are assigned, with weak texture regions marked as 1 and strong texture regions marked as 0.

[0052] (3) Weak texture edge line matching. For the binary image obtained in (2), we perform EDlines line detection to obtain the edges of the weak texture region and calculate the intersection points between the two lines. Next, we match the lines extracted from the two images using the LBD line descriptor.

[0053] (4) Removal of incorrect line matching. If the two lines in the figure below match the two lines in the figure above, then their intersection is also a matching point. Since the matching point needs to satisfy the constraint of the basic matrix F, that is, the matching point is on the polar line of each other, this property can be used to verify the correctness of the line matching in reverse and filter out some incorrect line matching.

[0054] (5) Calculation of the homography matrix H between two planes. Specifically, through the above steps, multiple straight lines describing the weak texture plane are obtained. The homography matrix H of the weak texture plane is calculated by selecting the corresponding pair of straight lines. The matching relationship between any point between two planes from different perspectives can be calculated through H.

[0055] by Figure 2For example, the two images above and below represent two images (image A below and image B above). The quadrilateral represents the edge of the weakly textured region, and the dashed line in the upper image represents the epipolar line of the point in the lower image. Since l1 and l2 match, and We have determined that p1 matches and They are also matched. To solve for the H matrix, we need three more pairs of matching points, namely... Figure 2 p4, p5, and p6 in the table. Next, we will explain in detail how to calculate the above three points in... Figure 2 Corresponding points in the upper and lower diagrams: When it is known that two matching lines l in the two diagrams are... Given the fundamental matrix F between the two graphs, we can solve for the matching point of any point p on l in the other graph. Since p lies on line l, then the corresponding point of p... It must be on a straight line in another graph. It is on the polar line of p, and it must also be on the polar line of p. Two straight lines determine a unique point of intersection, therefore... Intersection with the polar line in another figure p This corresponds to the point p2. In summary, the correspondence between points on a line can be determined by the correspondence between the lines from the two perspectives and the fundamental matrix F.

[0056] In the above steps: SIFT (Scale-Invariant Feature Transform) is a widely used computer vision algorithm for detecting and describing local features in images. It was first proposed by David Lowe in 1999, and the principle and implementation of the algorithm were described in detail in the paper "Distinctive image features from scale-invariant keypoints" published in 2004; EDlines (Edge Drawing Lines) is a method proposed in the paper "A real-time line segment detector with a false detection control"; LBD (Line Band Descriptor) is proposed in the paper "An efficient and robust line segment matching approach based on LBD descriptor and pairwise geometric consistency", which uses relative position and gradient information to describe the local features of a line.

[0057] This invention has the following features:

[0058] 1. This invention proposes to detect weak texture regions by means of information entropy. Weak texture regions usually have small color changes, which can be reflected in the information entropy, which is small.

[0059] 2. This invention proposes to extract weak texture planes in an image by detecting and matching lines based on weak texture region detection. Conventional line detection and matching are performed on the original color image, which can lead to the extraction of lines from non-weak texture regions. Compared to subsequent filtering through weak texture detection, this invention can more effectively extract and match lines from weak texture regions.

[0060] 3. This invention uses geometric constraints to filter out unreliable line matching relationships, thereby extracting the enclosing lines of the weak texture plane more stably and determining the position of the weak texture plane.

[0061] 4. This invention calculates the homography matrix of a weakly textured plane by extracting the enclosing lines of the weakly textured plane, and then calculates the correspondence between any points in the plane based on the homography matrix.

[0062] This invention has the following application prospects:

[0063] The inability to extract robust feature points in weakly textured scenes is a challenging problem in 3D reconstruction, object tracking, and target detection. This invention proposes an algorithm to address the hole problem in the reconstruction of weakly textured scenes in 3D, effectively solving the lack of matching points in weakly textured regions and significantly improving the reconstruction results. Notably, this invention can be combined with other feature extraction and matching algorithms, embedding itself into other algorithms to improve their performance.

[0064] The proposed feature point extraction algorithm based on planar priors has the advantage of extracting a sufficient number of feature points in weakly textured regions. Unlike traditional feature point extraction algorithms, our algorithm utilizes planar information from adjacent regions, thereby extracting more feature points in weakly textured areas. In the field of computer vision, feature points are a fundamental visual feature used in tasks such as object detection, tracking, and 3D reconstruction. In weakly textured regions, traditional algorithms often fail to extract enough feature points, while our algorithm fills this gap, improving the accuracy and efficiency of these tasks.

[0065] This algorithm has broad application prospects. In object tracking, extracting more feature points makes the tracker more stable and accurate. In 3D reconstruction, feature points can be used for matching and localization; extracting more feature points can improve the accuracy and efficiency of reconstruction. In autonomous driving, feature points can be used for road and lane line recognition and tracking; extracting more key feature points can improve the safety and stability of autonomous driving. In intelligent transportation, feature points can be used for traffic sign and marking recognition and tracking; extracting more feature points can enable intelligent traffic management and control.

[0066] In applications with weakly textured regions, traditional feature point extraction algorithms often struggle to extract enough feature points, hindering high-precision tasks. Our algorithm, however, leverages planar information from adjacent regions to extract more feature points in these weakly textured areas, offering broad application prospects. For example, in industrial manufacturing, parts on production lines may exhibit weakly textured regions, such as smooth metal or plastic parts. By using a planar prior-based feature point extraction algorithm, sufficient feature points can be extracted from these areas for tasks like localization, tracking, and quality control. In medical imaging, such as CT and MRI scans, weakly textured regions are common, including organ surfaces and blood vessel walls. Planar prior-based feature point extraction algorithms can extract feature points from these regions for medical image analysis and image registration. In robot navigation, robots need to move in indoor or outdoor environments, often containing weakly textured areas like smooth walls and glass. Planar prior-based feature point extraction algorithms can extract feature points from these areas for robot position estimation and navigation. In architectural design, building surfaces may contain areas with weak textures, such as glass curtain walls and metal curtain walls. Feature point extraction algorithms based on planar priors can extract feature points from these areas for tasks such as 3D building reconstruction and structural analysis. In autonomous driving, road surfaces often contain various areas with weak textures, such as zebra crossings and rearview mirrors. Our algorithm can extract more feature points from these areas, improving the safety and stability of autonomous driving.

[0067] The feature point extraction algorithm based on planar prior proposed in this invention can bring more opportunities and challenges to the development of computer vision, autonomous driving, intelligent transportation and other fields.

[0068] As described above, the present invention can be implemented well.

[0069] All features disclosed in all embodiments of this specification, or steps in all methods or processes implied in the disclosure, may be combined and / or extended or replaced in any way, except for mutually exclusive features and / or steps.

[0070] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Based on the technical essence of the present invention, any simple modifications, equivalent substitutions, and improvements made to the above embodiments within the spirit and principles of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A weak texture feature matching method, characterized in that, Includes the following steps: S1, Fundamental Matrix Calculation: Using the feature points extracted by SIFT for matching, calculate the fundamental matrix F between the two images, denoted as image A and image B respectively; S2, Weak Texture Region Detection: Weak texture regions in images A and B are identified using information entropy. Step S2 includes the following steps: S21, calculating the information entropy of images A and B within a window of a set size, centered on each pixel, to obtain the information entropy maps corresponding to images A and B; S22, combining the difference between the maximum and minimum pixel values ​​within the set size window to jointly determine the probability that a pixel in this window represents a weak texture; S23, binarizing the information entropy maps corresponding to images A and B according to a set threshold, marking weak texture regions as 1 and strong texture regions as 0, to obtain a binary map. S3, Edge Line Matching: Extract edge lines of weak texture regions from images A and B respectively, and match the extracted edge lines. Step S3 includes the following steps: S31, perform EDlines detection on the binary images corresponding to images A and B respectively to extract edge lines of weak texture regions; S32, calculate the intersection point between any two edge lines in image A, and calculate the intersection point between any two edge lines in image B; S33, match the weak texture edge lines extracted from image A and image B using LBD line descriptors. S4, Mismatch Removal: Remove mismatched edge lines from images A and B respectively. Step S4 includes the following steps: S41, if two edge lines in image A match two edge lines in image B, then the intersection of the two edge lines in image A and the intersection of the two edge lines in image B are the matching points; S42, using the property that the matching points satisfy the constraints of the fundamental matrix F, the correctness of the edge line matching is verified in reverse, and mismatched edge lines are removed, resulting in multiple lines describing weak texture regions in images A and B. S5, Homography Matrix Calculation: Calculate the homography matrix H between the weak texture regions in image A and image B, and then calculate the dense correspondence of any point within the weak texture region based on the homography matrix to fill the holes in the 3D reconstruction model. This includes: S51, selecting the matching line pairs of weak texture regions obtained in step S4, and calculating the homography matrix H between the weak texture regions in image A and the corresponding weak texture regions in image B using the geometric relationship of the line pairs themselves; S52, calculating the matching relationship between any point between the weak texture regions in image A and the corresponding weak texture regions in image B from different perspectives using H, and generating dense feature points within the weak texture regions.

2. The weak texture feature matching method according to claim 1, characterized in that, In step S1, the image originates from one or more of the following scenarios: object tracking, road and lane line recognition and tracking, and traffic sign and marking recognition and tracking.

3. A weak texture feature matching system, characterized in that, A method for implementing a weak texture feature matching method according to any one of claims 1 to 2 includes the following modules connected in sequence: Fundamental Matrix Calculation Module: Used to calculate the fundamental matrix F between two images, denoted as image A and image B respectively; Weak texture region detection module: used to distinguish weak texture regions in image A and image B using information entropy; Edge line matching module: used to extract edge lines of weak texture regions in images A and B respectively, and to match the extracted edge lines; Mismatch Removal Module: Used to filter out mismatched edge lines in images A and B respectively; Homography matrix calculation module: used to calculate the homography matrix H between weak texture regions in image A and weak texture regions in image B, and then calculate the correspondence between any points in the weak texture region based on the homography matrix.

Images