Image line feature classification matching method without prior condition

Abstract

The application relates to a kind of image line feature classification matching methods without prior condition, belong to image matching technical field.The application first connects adjacent line features to form V-shaped line pair, compares the matching effect of reduced images under different transformation parameters, corrects the original image using the best parameters, carries out line pair intersection matching, and determines the same name line feature and the same name V-shaped line pair in combination with angle constraint;For unmatched line segment, construct coplanar region around the same name V-shaped line pair and restore local homography, according to homography constraint, filter candidate matching line in coplanar region, and realize single line segment grouping matching.The application combines the best transformation parameters calculated by simulating camera view point and line pair intersection matching result, provides strong geometric condition, replaces prior information constraint, and realizes line feature matching between pure images without prior information.

Description

Technical Field

[0001] This invention relates to an image line feature classification and matching method that does not require prior conditions, belonging to the field of image matching technology. Background Technology

[0002] Wide-view imagery allows for observation of surface morphology and acquisition of target elevation information from multiple angles, providing a more intuitive and complete representation of the target. Line features, as a commonly used feature, possess rich semantic information and can effectively express target structure. Therefore, establishing the correlation between wide-view imagery through line feature matching is of great significance for target 3D reconstruction.

[0003] For images with significant viewpoint changes, the inconsistency in the content of the neighboring images on one side of the line feature leads to a decrease in the similarity of line descriptors constructed based on the line support domain. Existing methods typically establish transformation relationships between images through attitude data to mitigate the influence of complex geometric deformations between images; or determine the local homography of candidate line pairs using a given fundamental matrix and construct a homography map, combining reweighted random walks and constrained greedy algorithms to achieve reliable matching in cross-viewpoint images; alternatively, geometric constraints can be established using the disparity between stereo image pairs to limit the matching search range of line features and achieve better matching results. However, all of the above line feature matching methods require additional prior knowledge to enhance the stability of the matching, which has certain limitations. Summary of the Invention

[0004] The purpose of this invention is to provide an image line feature classification and matching method that does not require prior conditions, so as to solve the problem that the current cross-view image matching method has great limitations due to the need for prior information constraints.

[0005] To solve the above-mentioned technical problems, this invention provides an image line feature classification and matching method that does not require prior conditions. This classification and matching method includes the following steps:

[0006] 1) Obtain the image to be classified and matched, and construct V-shaped line pairs based on the distribution of line features in the obtained image;

[0007] 2) Perform image transformation on each image in the image pair to be classified and matched, determine the best transformation parameters for each image in the form of multi-parameter simulation matching, and transform the images according to the best transformation parameters.

[0008] 3) Judge all the corresponding intersection points of the transformed image pairs, delete the corresponding intersection points that cannot obtain the corresponding V-shaped line pairs, obtain the corresponding V-shaped line pair groups, and take the line segments in the corresponding V-shaped line pairs as the matching line segments.

[0009] 4) Construct local coplanar regions around the same V-shaped line pairs, group the unmatched line features into single line segments, restore the homography relationship of the corresponding regions by combining the basic matrix, and select candidate matching lines according to homography constraints to achieve single line feature matching within the group.

[0010] This invention combines the optimal transformation parameters calculated from the simulated camera viewpoint with the line pair intersection matching results, providing powerful geometric conditions that replace prior information constraints. It achieves pure image-to-image line feature matching without prior information, effectively solving the problem of needing to construct additional constraints based on image prior information to obtain reliable matching line pairs due to large geometric deformation of cross-view images.

[0011] Furthermore, the process of constructing the V-shaped line pairs in step 1) is as follows:

[0012] In each image, a search region is established for each line segment. Within this search region, the line features that form V-shaped line pairs with the segment are identified. The conditions for forming V-shaped line pairs include:

[0013] One endpoint of the line feature is located within the square search area; the angle between the line feature and the line segment is greater than 30 degrees and less than 130 degrees; the intersection of the line feature and the line segment is located within the search area.

[0014] This invention uses an online feature search region to find adjacent line features that satisfy the angle constraints, thereby enabling the rapid and accurate construction of V-shaped line pairs.

[0015] Furthermore, in step 3), the same-name V-shaped line pair refers to a V-shaped line pair whose angle difference between the two line pairs with the same intersection point is within a set threshold range.

[0016] Furthermore, the homography recovery process in step 4) is as follows:

[0017] Establish a system of linear equations about the fundamental matrix using all intersections of the same name, and solve for the fundamental matrix;

[0018] Construct non-homogeneous constraints on the homography matrix based on the compatibility constraints between the fundamental matrix and the homography matrix;

[0019] Based on the mapping relationship of the features of the same line, the relevant parameters of the homography matrix are calculated, thereby obtaining the homography matrix of the coplanar region.

[0020] Based on the compatibility constraints between the fundamental matrix and the homography matrix, this invention allows for the determination of the homography of a given planar region corresponding to a fundamental matrix F by solving three parameters, thus facilitating the solution of the homography matrix.

[0021] Furthermore, the single-segment matching process in step 4) is as follows:

[0022] The candidate matching line segments belonging to the coplanar region of the same V-shaped line pair are respectively calculated as the mapping lines in the corresponding images through the local homography relationship;

[0023] Determine whether the candidate matching line segment and its corresponding mapping line satisfy the angle constraint and the proximity constraint.

[0024] Candidate matching line segments that simultaneously satisfy both angle constraints and proximity constraints are designated as line segments with the same name.

[0025] Furthermore, the angle constraint condition is as follows:

[0026]

[0027] Where α and α′ are the angles of the candidate matching line segments, β and β′ are the angles of the mapping lines of the candidate matching line segments, and T β To set the angle threshold.

[0028] Furthermore, the proximity constraint condition is as follows: a support domain for candidate matching line segments is constructed in each of the respective images. If any endpoint of the mapping line of the candidate matching line segment is within the support domain of the matching line segment in the mapped image, then the proximity constraint is satisfied.

[0029] This invention, by constructing angle constraints and proximity constraints, can accurately find line features that are consistent in direction and spatially adjacent, thereby improving the accuracy of matching line segments with the same name.

[0030] Furthermore, when the candidate matching line segments are one-to-many or many-to-many, the matching line with the smallest mapping distance is selected as the final line segment with the same name.

[0031] This invention further filters one-to-many or many-to-many matching results by mapping distance. The smaller the distance, the stronger the correlation between the two lines, and the more likely they are line segments with the same name. Therefore, it can find the final matching result of line segments with the same name from one-to-many or many-to-many matching results, which greatly improves the accuracy of matching.

[0032] Furthermore, when the image line feature to be matched is a curve, a virtual V-shaped line pair is constructed, and the virtual V-shaped line pair is matched according to the method of V-shaped line pair; the construction process of the virtual V-shaped line pair is as follows: find the adjacent curve in the search area of ​​the curve, connect the endpoints of the two adjacent curves and their subsequent edge points to construct virtual straight line segments. If the included angle of the two virtual straight line segments is within the set range, then the two virtual straight line segments of these two adjacent curves constitute a virtual V-shaped line pair.

[0033] This invention targets curve features and constructs virtual V-shaped line pairs for line feature matching, which can solve the problem of needing prior information in the curve feature matching process. Attached Figure Description

[0034] Figure 1 This is a flowchart of the image line feature classification and matching method of the present invention that does not require prior conditions;

[0035] Figure 2 This is a schematic diagram of the V-shaped line pair construction principle used in this invention;

[0036] Figure 3 This is a schematic diagram of the locally coplanar region of the V-shaped line used in this invention;

[0037] Figure 4 This is a schematic diagram of the single-segment matching proximity constraint used in this invention;

[0038] Figure 5 This is a schematic diagram illustrating the construction principle of the virtual V-shaped line pair used in curve matching of this invention. Detailed Implementation

[0039] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0040] This invention consists of two parts: V-shaped line pair matching and single line segment matching. In the V-shaped line pair matching part, adjacent line features are first connected to form V-shaped line pairs. The matching effect of the scaled-down image under different transformation parameters is compared. The original image is corrected using the optimal parameters, and the intersection points of the line pairs are matched. Angle constraints are then used to determine the corresponding line features and corresponding V-shaped line pairs. In the single line segment matching part, a coplanar region is first constructed around the corresponding V-shaped line pairs, and local homography is restored. Then, candidate matching lines are selected within the coplanar region based on homography constraints, achieving single line segment group matching. This invention effectively solves the problem of needing to construct additional constraints based on prior image information to obtain reliable matching line pairs due to large geometric deformation of cross-view images, without requiring prior information limitations. The implementation process of this method is as follows: Figure 1 As shown below, the specific implementation process of this procedure will be explained in detail.

[0041] 1. Construct V-shaped line pairs

[0042] The images to be matched are obtained, including a reference image and a target image. V-shaped line pairs are constructed for the line features in both the reference and target images. Two coplanar line features and their intersection points form new matching primitives, and V-shaped line pairs are formed by connecting adjacent line features. Specifically, as follows... Figure 2 As shown, for line segment l, square search areas of size 20×20 are defined with its two endpoints as the center. For line features that satisfy conditions (1) to (3), they are paired with line segment l to form V-shaped line pairs.

[0043] (1) A line feature has one endpoint located within a square search area;

[0044] (2) The angle between the line feature and l is greater than 30 degrees and less than 130 degrees;

[0045] (3) The intersection of the line feature and l is located within the square search area.

[0046] Figure 2 In the case of l5, neither of the two endpoints of l is located within the two search areas of line segment l. The angle between l2 and l does not satisfy the angle constraint condition. Although l3 satisfies the first two conditions, its intersection point does not meet the third condition. Therefore, only l1 and l4 can form a V-shaped line pair with l.

[0047] 2. Determine the optimal image transformation parameters

[0048] To overcome the poor matching effect caused by perspective issues, this invention requires determining the optimal image transformation parameters and transforming the image to be matched according to these parameters. The specific process is as follows:

[0049] First, images with sides exceeding 200 pixels are scaled down proportionally. The formula for calculating the scaling parameter λ is as follows:

[0050]

[0051] In the formula, W and H represent the width and height of the original image, respectively. In this embodiment, T s It is 200.

[0052] Then, set the tilt angle t to 1. 2. For each inclination degree, samples are taken in the range [0,π] at intervals of 72 / t to obtain the corresponding longitude values. Finally, 20 sets of simulation parameters were obtained. The 20 sets of set parameters were substituted into equation (2) in turn to transform the reduced image and establish two sets of simulation images.

[0053]

[0054] Arbitrarily pairwise combinations of simulated images from different groups are performed using the Superpoint algorithm. The values ​​A and A′ with the highest number of matching points are recorded as the optimal transformation parameters. These optimal transformation parameters are then used to transform the two original images, resulting in the transformed image I. A and I′ A′ .

[0055] 3. Match the V-shaped line pairs according to the transformed image.

[0056] First, the line pair is transformed to I using the optimal transformation parameters. A and I′ A′In the middle, taking the intersection point of the same name as the origin, the vertical downward direction is defined as the 0 direction, and the counterclockwise angle between the 0 direction and the line feature is defined as the angle of the line feature. If the difference between the angles of the two line pairs is within (20, 340), then the V-shaped line pair formed by them is considered to be a V-shaped line pair of the same name.

[0057] Based on the above method, all intersections with the same name are judged, and intersections with the same name that cannot be used to obtain a V-shaped line pair are removed, thus forming a group of V-shaped line pairs with the same name. in The intersection of the same name is the point of intersection. and There are two pairs of matching line segments with the same name, and m is the number of matching V-shaped line pairs.

[0058] 4. Perform single-segment matching

[0059] Line features that do not obtain reliable matching are matched as single line segments in this stage. First, single line segments are grouped into local coplanar regions around the same V-shaped line pairs. Then, the homography relationship of the corresponding regions is restored by combining the basic matrix. Finally, candidate matching lines are selected according to the homography constraints to achieve single line feature matching within the group.

[0060] 1) Grouping of single line segments

[0061] Using the V-shaped lines of the same name obtained in step 3, construct locally coplanar regions and group them into single-line segments. Assume the V-shaped lines of the same name obtained in step 3 are... and First, around and Local coplanar regions are constructed in both the reference and target images. Taking V as an example (e.g.) Figure 3 As shown), p1 and p2 are The endpoints farthest from the intersection, a and b, represent the points with... Parallel and passing through points respectively The straight line between p2 and c and d represents the line with p2. Parallel and passing through points respectively The straight line between p1 and p2. Place a and b along... Translate outwards by d 22 and d 21 Pixels serve as the left and right boundaries, with c and d respectively along... Translation in both directions d 22 and d 21 Pixels are used as the upper and lower boundaries. Where d 21 =l2 / 2,d 22 =l2 / 4,d 11 =l1 / 2,d 12 =l1 / 4, where l1 and l2 are linear features. and The length of the line segment is then determined. Then, in both the reference and target images, single line segments whose endpoints lie within a coplanar region are identified, forming corresponding groups of single line segments. and

[0062] 2) Perform localized homography restoration

[0063] First, establish a system of linear equations about F using all intersection points with the same name. The fundamental matrix F is solved using the least squares method.

[0064] Then, the local homography of the corresponding plane is estimated by combining the fundamental matrix F and the homography matrix H, and the compatibility constraint H between the fundamental matrix F and the homography matrix H is used. T F+F T H = 0 constructs 5 non-homogeneous constraints on H. Therefore, the eight parameters of the homography matrix H can be simplified to three parameters for calculation. Arbitrary decomposition of F yields A and e′. Substituting the two pairs of corresponding line features into formula (3) establishes 4 equations. The three-parameter matrix b of H is solved using the least squares method, finally yielding the local homography matrix H and its inverse matrix H′.

[0065]

[0066] 3) Obtain line segments with the same name

[0067] Candidate matching lines are selected based on homography constraints, and single line segments within the group are matched. For example... Figure 4 As shown, l and l′ are a pair of candidate matching line segments corresponding to single line segment groups κ and κ′, respectively, belonging to the coplanar region where the corresponding V-shaped line pairs (V, V′) are located. Based on local homography, the mapping lines l′ of l and l′ in the corresponding images are calculated. H and l H′ If l and l′ are a correctly matched line pair, l′ H It should be located near l′ and have a similar angle to l′; similarly, l H′ It should also be located near l and have a similar angle to l. Therefore, we check l, l′, l′ in sequence. H ,l H′ Do the angle and proximity constraints meet?

[0068] (1) Angle constraint conditions: l and l H′ Angle α and β, l′ and l′ H If the angles α′ and β′ satisfy equation (4), then l and l′ are considered to have directional consistency, and l and l′ are taken as candidate matching lines; where α Let β be the angle between the directions I and 0, and let l be the angle between I and 0. H′ The angle between the zero direction and the zero direction, which is the zero direction defined in step 3.

[0069]

[0070] In this embodiment, T β The value of T is 20 degrees. In other implementation methods, T... β Adjustments can also be made.

[0071] (2) Proximity constraint: Construct rectangular support domains around l and l′ respectively (e.g., Figure 4 (dashed box area), when and l H′ If any endpoint of l is located within the supporting domain, then l and l′ are considered to have spatial proximity.

[0072] Line features that are in the same direction and spatially adjacent are selected as candidate line features, and the mapping distance d of the candidate line features is calculated according to formula (5). H The smaller the distance, the stronger the correlation between l and l′, and the more likely they are line segments with the same name. Therefore, for "one-to-many" or "many-to-many" phenomena in candidate matching, this invention only retains the one with the smallest mapping distance that is less than the threshold T. H The matching line is used as the final line segment with the same name.

[0073]

[0074] Where d1 and d2 are respectively l H′ The distances from the two endpoints to l, d1′ and d2′ are respectively the distances from l′ to l. H The distance from the two endpoints to l′ (e.g.) Figure 4 (As shown).

[0075] Thus, through the above process, the present invention can achieve the matching of line features in images, where line features mainly refer to straight line features. For curved features in images, the present invention can construct virtual V-shaped line pairs of curves and match the constructed virtual V-shaped line pairs according to the method in steps 1-3 to achieve the matching of curved line features in images.

[0076] Specifically, such as Figure 5 As shown, with curves Define a square search region of ω×ω centered on the endpoints, and treat any curve whose endpoint lies within the search region as a neighboring curve (e.g., ...). Then, virtual straight line segments l1 and l2 are constructed by connecting the endpoints of two adjacent curves and the 10th edge point thereafter. If the angle between l1 and l2 is within the range of 20 to 160 degrees, a virtual V-shaped line pair V = (p, l1, l2) is formed. The intersection points of the virtual V-shaped line pair are matched in the transformed image to determine the correspondence between the two pairs of curves, thus achieving image curve matching.

Claims

1. An image line feature classification and matching method without prior conditions, characterized in that, This classification matching method includes the following steps: 1) Obtain the image to be classified and matched, and construct V-shaped line pairs based on the distribution of line features in the obtained image; 2) Perform image transformation on each image in the image pair to be classified and matched, determine the optimal transformation parameters for each image using multi-parameter simulation matching, and transform the images according to the optimal transformation parameters; set different tilt angles, and sample the images to be classified and matched at set intervals under each tilt angle as corresponding longitude values ​​to obtain multiple sets of simulation parameters; substitute the multiple sets of simulation parameters into the images for transformation to establish two sets of simulation images; combine any two simulated images from different sets and match them using the Superpoint algorithm; record the parameters with the maximum number of matching points as the optimal transformation parameters; 3) Judge all the corresponding intersection points of the transformed image pairs, delete the corresponding intersection points that cannot obtain the corresponding V-shaped line pairs, obtain the corresponding V-shaped line pair groups, and take the line segments in the corresponding V-shaped line pairs as the matching line segments. 4) Construct local coplanar regions around the same-named V-shaped line pairs. Group unmatched line features into single line segments, recover the homography relationship of the corresponding regions using the fundamental matrix, and filter candidate matching lines according to homography constraints to achieve single-line feature matching within the group. The same-named V-shaped line pairs are... and , and The intersection of the same name is the point of intersection. For matching line segments with the same name, for The endpoints farthest from the intersection, a and b, represent the points with... Parallel and passing through points respectively and The straight lines, c and d represent the lines with... Parallel and passing through points respectively and The straight line; place a and b along respectively Shift outwards and Pixels serve as the left and right boundaries of the coplanar region, with c and d respectively along... Translation in two directions and Pixels serve as the upper and lower boundaries of the coplanar region; , and Line features and The length.

2. The image line feature classification and matching method without prior conditions according to claim 1, characterized in that, The process of constructing the V-shaped line pairs in step 1) is as follows: In each image, a search region is established for each line segment. Within this search region, the line features that form V-shaped line pairs with the segment are identified. The conditions for forming V-shaped line pairs include: One endpoint of the line feature is located within the square search area; the angle between the line feature and the line segment is greater than 30 degrees and less than 130 degrees; the intersection of the line feature and the line segment is located within the search area.

3. The image line feature classification and matching method without prior conditions according to claim 1, characterized in that, In step 3), the same-name V-shaped line pair refers to a V-shaped line pair whose angle difference between the two line pairs with the same intersection point is within the set threshold range.

4. The image line feature classification and matching method without prior conditions according to claim 1, characterized in that, The homography recovery process in step 4) is as follows: Establish a system of linear equations about the fundamental matrix using all intersections of the same name, and solve for the fundamental matrix; Construct non-homogeneous constraints on the homography matrix based on the compatibility constraints between the fundamental matrix and the homography matrix; Based on the mapping relationship of the features of the same line, the relevant parameters of the homography matrix are calculated, thereby obtaining the homography matrix of the coplanar region.

5. The image line feature classification and matching method without prior conditions according to claim 4, characterized in that, The single-segment matching process in step 4) is as follows: The candidate matching line segments belonging to the coplanar region of the same V-shaped line pair are respectively calculated as the mapping lines in the corresponding images through the local homography relationship; Determine whether the candidate matching line segment and its corresponding mapping line satisfy the angle constraint and the proximity constraint. Candidate matching line segments that simultaneously satisfy both angle constraints and proximity constraints are designated as line segments with the same name.

6. The image line feature classification and matching method without prior conditions according to claim 5, characterized in that, The angle constraint condition is as follows: Where α and α′ are the angles of the candidate matching line segments, β and β′ are the angles of the mapping lines of the candidate matching line segments, and T β To set the angle threshold.

7. The image line feature classification and matching method without prior conditions according to claim 5, characterized in that, The proximity constraint is as follows: a support domain for candidate matching line segments is constructed in each image. If any endpoint of the mapping line of a candidate matching line segment is within the support domain of the matching line segment in the mapped image, then the proximity constraint is satisfied.

8. The image line feature classification and matching method without prior conditions according to claim 5, characterized in that, When the candidate matching line segments are one-to-many or many-to-many, the matching line with the smallest mapping distance is selected as the final line segment with the same name.

9. The image line feature classification and matching method without prior conditions according to any one of claims 1-8, characterized in that, When the image line feature to be matched is a curve, a virtual V-shaped line pair is constructed, and the virtual V-shaped line pair is matched according to the method of V-shaped line pair. The construction process of the virtual V-shaped line pair is as follows: find the adjacent curve in the search area of ​​the curve, and connect the endpoints of the two adjacent curves and their subsequent edge points to construct virtual straight line segments. If the included angle of the two virtual straight line segments is within the set range, then the two virtual straight line segments of these two adjacent curves constitute a virtual V-shaped line pair.

Images