A high-resolution remote sensing image matching method fusing line features and geometric constraints

Abstract

The application discloses a high-resolution remote sensing image matching method fusing line features and geometric constraints, comprising the following steps: step 1, extracting line segments in a reference image and a to-be-matched image; step 2, constructing a multi-scale geometric feature descriptor according to geometric information between the line segments and adjacent line segments, and obtaining a candidate straight line set through similarity measurement screening; step 3, establishing an initial matching relationship based on the candidate straight line set obtained in step 2; and step 4, verifying the matching result through iterative geometric constraint verification based on the initial matching relationship obtained in step 3, and obtaining a final matching result. The method can solve the problems of insufficient stable feature points and weak line structure expression capability of a current local feature description method in a high-resolution remote sensing low-texture and large repeated edge scene, and simultaneously improves the candidate search efficiency, matching accuracy and robustness of geometric consistency verification in a high-resolution remote sensing image with scale change, rotation change, light change and more local repeated structures.

Description

Technical Field

[0001] This invention relates to the field of image feature matching technology, and specifically to a high-resolution remote sensing image matching method that integrates line features and geometric constraints. Background Technology

[0002] High-resolution remote sensing image matching is an important research area in high-resolution remote sensing image processing. High-resolution remote sensing image matching technology maps an image of a specific scene acquired by a sensor to one or more images of the same scene, establishing spatial correspondences at the pixel or feature level, and achieving precise alignment of the same ground objects in the two images.

[0003] High-resolution remote sensing images exhibit significant differences in spatial resolution, temporal characteristics, imaging perspective, and background information. Traditional point feature matching algorithms often struggle to effectively detect stable keypoints, and their feature descriptors lack uniqueness, leading to a significantly higher false-match rate. Compared to the micro-texture dependence of point features, line features, by expressing macro-geometric structures such as feature edges and region boundaries, can effectively overcome the representation bottleneck of low-texture areas. Furthermore, geometric information such as the direction, length, and cross-linking patterns between line segments provides complementary matching constraints.

[0004] However, in heterogeneous remote sensing scenarios, if matching is based solely on geometric information such as line segment length, direction, and adjacency, different ground objects are prone to mismatches due to similar local geometric structures. Conversely, directly introducing regional grayscale or gradient information to match all line segments one by one increases computational complexity, making it difficult to balance matching efficiency and accuracy. Therefore, a high-resolution remote sensing image matching method that balances candidate selection efficiency, local discrimination capability, and overall geometric consistency is urgently needed. Summary of the Invention

[0005] Purpose of the invention: The technical problem to be solved by the present invention is to provide a high-resolution remote sensing image matching method that integrates line features and geometric constraints, addressing the shortcomings of the existing technology.

[0006] To address the aforementioned technical problems, this invention discloses a high-resolution remote sensing image matching method that fuses line features and geometric constraints, comprising the following steps:

[0007] Step 1: Extract line segments from the reference image and the image to be matched;

[0008] Step 2: For line segments in the reference image and the image to be matched, construct multi-scale geometric feature descriptors based on their geometric information with neighboring line segments, and obtain a set of candidate lines by filtering through similarity measurement;

[0009] Step 3: Select supporting regions in the neighborhood of the line segment and construct regional gradient feature descriptors, and establish initial matching relationships using the nearest neighbor and second nearest neighbor algorithms;

[0010] Step 4: Verify the matching results through iterative geometric constraint verification to obtain the final line matching result.

[0011] Furthermore, the specific steps of step 1 include:

[0012] Step 1-1: Calculate the gradient magnitude and gradient direction of each pixel in the reference image and the image to be matched;

[0013] Step 1-2: Perform region growth based on the consistency of gradient direction, and aggregate connected pixels with approximate gradient direction to form line support regions;

[0014] Steps 1-3: Perform rectangular approximation on each line support region to obtain a rectangular region representing the line segment, and calculate its center, length, width and angle;

[0015] Steps 1-4: Calculate the false alarm number (NFA) for the rectangular region based on the Helmholtz principle, verify the validity of the fitted rectangular line segments, and retain the NFA. The line segments are used as the final line segment extraction result.

[0016] Furthermore, the specific steps of step 2 include:

[0017] Step 2-1: For the line segment The length of its line segment, midpoint, and direction angle are respectively expressed as: , , .by midpoint Let the coordinates be the center of the circle. For the diameter, the scale factor is defined as: The set of neighboring line segments of a circular neighborhood is defined as follows:

[0018]

[0019] in, for The midpoint.

[0020] Step 2-2: For Its relative The normalized relative geometric features are :

[0021]

[0022]

[0023]

[0024] in, , They are respectively The length and direction angle of the line segment; for The relative diameter, representing With describing the neighborhood Scale proportions; for and The radial relative distance; It is the angle between the direction angles of the two lines;

[0025] Steps 2-3: and The relationship between the median line segments can be expressed as:

[0026]

[0027] Will , , The range of values Divide into 4 equal intervals and calculate. Histogram description:

[0028]

[0029]

[0030] in, The first in the histogram One value;

[0031] use (Chi-square statistic) Statistical implementation of geometric feature descriptor similarity measurement:

[0032]

[0033] in, Represents a line segment in a reference image. This represents candidate line segments in the image to be matched;

[0034] Steps 2-4: Use multiple sets of scaling factors Repeat steps 2-1 to 2-3 above to construct multi-scale geometric feature descriptors, sort them from high to low similarity, and select the top... The candidate straight lines are used as straight line segments. The set of candidate lines, where Multi-scale geometric feature descriptors are constructed for each line segment in the reference image and each line segment in the image to be matched, thereby generating their respective candidate line sets;

[0035] Furthermore, step 3 includes the following specific steps:

[0036] Step 3-1: Divide the straight line segment Evenly divided into Line segments , with each line segment The midpoint is the center of the circle. Multiple local circular support areas are established with the line segment length as the diameter. ,in Take 8. The support region is not used independently for point feature description, but is connected in sequence according to the line segment direction to form a line segment neighborhood support region sequence;

[0037] Step 3-2: Calculate the number of pixels in each support region. gradient magnitude With direction Based on gradient direction The pixels within the support region are divided into 8 equally divided intervals. The gradient magnitudes of each interval are accumulated to generate an directional gradient histogram. The direction of the peak of this histogram corresponds to the main direction of the gradient feature of the current region. All intervals are traversed counterclockwise. The summation of gradient magnitudes in each directional interval can be expressed as:

[0038]

[0039] Step 3-3: From the main direction, the position of the... Starting with the interval, follow the interval traversal order and use... Constructing feature description vectors To maintain the consistency of the descriptor's representation under different rotation angles;

[0040] Steps 3-4: Use the nearest neighbor and second nearest neighbor algorithm to match regional gradient feature descriptors and establish initial matching relationships.

[0041] Furthermore, step 4 includes the following specific steps:

[0042] Step 4-1: Calculate the initial homography matrix based on the midpoint correspondence of the initial line segment matching. The length of the target line segment is obtained through homography transformation. Direction angle and midpoint ;

[0043] Step 4-2: For any set of matching line segments, calculate the line segment scale projection error. Angular projection error Midpoint projection error :

[0044]

[0045]

[0046]

[0047] and with , and The homography matrix is ​​calculated with the goal of minimizing the weighted sum. :

[0048]

[0049] Among them, the summation symbol This indicates that all matching line segments in the current matching set are accumulated; , , They are respectively , and The weighting coefficients, and in order to make the midpoint projection error have a higher constraint effect in the mismatch elimination process, can be taken as:

[0050]

[0051] Further eliminate mismatched lines and iteratively optimize the homography matrix to obtain the final matching result.

[0052] Beneficial effects:

[0053] This invention does not simply combine line segment geometric features, regional gradient features, and geometric consistency verification in parallel. Instead, it constructs a hierarchical and collaborative matching mechanism. It utilizes multi-scale geometric relationships between line segments and neighboring line segments to pre-screen candidate line segments, reducing computational complexity and improving candidate search efficiency. Instead of establishing support regions for isolated feature points, this invention establishes segmented local circular support regions with spatial order constraints along the line segment direction. It then uses gradient direction statistics to form regional gradient feature description vectors, enhancing the ability to distinguish line segments with similar local geometric structures but different true attributes, thus improving matching accuracy in complex scenes. The homography matrix in this invention is not obtained by looking up a table based on preset camera angles, nor does it rely on a calibration board or manual annotation. Instead, it is estimated from the initial line segment matching relationship and iteratively geometrically verified by combining line segment scale projection error, direction angle projection error, and midpoint projection error. This enables dynamic removal of mismatched line segment pairs, thereby improving the robustness and stability of the final matching result.

[0054] Therefore, this invention can solve the problems of insufficient stable feature points and weak ability to express linear structures in existing point feature local description methods in high-resolution remote sensing scenes with low texture and a large number of repeated edges. It can also solve the problem that the pre-corrected homography matrix method relies on fixed imaging equipment, fixed relative angles and pre-stored matrices, making it difficult to apply to non-fixed remote sensing image pairs. Through the above-mentioned hierarchical collaborative processing, this invention can simultaneously improve candidate search efficiency, matching accuracy and robustness of geometric consistency verification in high-resolution remote sensing images with scale changes, rotation changes, illumination changes and a large number of local repeated structures. Attached Figure Description

[0055] Figure 1 A schematic diagram of constructing a multi-scale geometric feature descriptor.

[0056] Figure 2 A schematic diagram of constructing a region gradient feature descriptor.

[0057] Figure 3 This is a schematic diagram for iterative geometric constraint verification.

[0058] Figure 4 This is a flowchart of the present invention.

[0059] Figure 5 This is a reference image used in this embodiment.

[0060] Figure 6 This is the image to be matched used in this embodiment.

[0061] Figure 7 shows the matching results of this embodiment. Detailed Implementation

[0062] Step 1: Extract line segments from the reference image and the image to be matched;

[0063] Step 2: For line segments in the reference image and the image to be matched, construct multi-scale geometric descriptors based on their geometric information with neighboring line segments, and obtain a set of candidate lines by filtering through similarity measurement;

[0064] Step 3: Select supporting regions in the neighborhood of the line segment and construct regional gradient feature descriptors, and establish initial matching relationships using the nearest neighbor and second nearest neighbor algorithms;

[0065] Step 4: Verify the matching results through iterative geometric constraint verification to obtain the final line matching result.

[0066] Furthermore, the specific steps of step 1 include:

[0067] Step 1-1: Calculate the gradient magnitude and gradient direction of each pixel in the reference image and the image to be matched;

[0068] Step 1-2: Perform region growth based on the consistency of gradient direction, and aggregate connected pixels with approximate gradient direction to form line support regions;

[0069] Steps 1-3: Perform rectangular approximation on each line support region to obtain a rectangular region representing the line segment, and calculate its center, length, width and angle;

[0070] Steps 1-4: Calculate the false alarm number (NFA) for the rectangular region based on the Helmholtz principle, verify the validity of the fitted rectangular line segments, and retain the NFA. The line segments are used as the final line segment extraction result;

[0071] Furthermore, the specific steps for constructing the multi-scale geometric feature descriptor in step 2 include:

[0072] Step 2-1: For the line segment The length of its line segment, midpoint, and direction angle are respectively expressed as: , , .by midpoint Let the coordinates be the center of the circle. For the diameter, the scale factor is defined as: The set of neighboring line segments of a circular neighborhood is defined as follows:

[0073]

[0074] in, for The midpoint.

[0075] Step 2-2: For Its relative The normalized relative geometric features are :

[0076]

[0077]

[0078]

[0079] in, , They are respectively The length and direction angle of the line segment; for The relative diameter, representing With describing the neighborhood Scale proportions; for and The radial relative distance; It is the angle between the direction angles of the two lines;

[0080] Steps 2-3: Set of adjacent line segments The relationship between the median line segments can be expressed as:

[0081]

[0082] Will , , The range of values Divide into 4 equal intervals and calculate. Histogram description:

[0083]

[0084]

[0085] in, The first in the histogram One value;

[0086] use Statistical implementation of geometric feature descriptor similarity measurement:

[0087]

[0088] in, Represents a line segment in a reference image. This represents candidate line segments in the image to be matched;

[0089] Steps 2-4: Use multiple sets of scaling factors Repeat steps 2-1 to 2-4 above to construct multi-scale geometric feature descriptors, sort them from high to low similarity, and select the top... The candidate straight lines are used as straight line segments. The set of candidate lines, where Multi-scale geometric feature descriptors are constructed for each line segment in the reference image and each line segment in the image to be matched, thereby generating their respective candidate line sets;

[0090] Furthermore, the specific steps for constructing the region gradient feature descriptor in step 3 include:

[0091] Step 3-1: For the line segments in the reference image and their candidate line segments in the image to be matched, divide the line segments evenly into... Line segments , with each line segment The midpoint is the center of the circle. Multiple local circular support areas are established with the line segment length as the diameter. ,in Take 8. The support region is not used independently for point feature description, but is connected in sequence according to the line segment direction to form a line segment neighborhood support region sequence;

[0092] Step 3-2: Calculate the number of pixels in each support region. gradient magnitude With direction Based on gradient direction The pixels within the support region are divided into eight equal intervals. The gradient magnitudes of each interval are accumulated to generate an directional gradient histogram. The direction of the peak value of this histogram corresponds to the main direction of the gradient feature in the current region.

[0093] Traverse all intervals counterclockwise, the first... The summation of gradient magnitudes in each directional interval can be expressed as:

[0094]

[0095] Where n represents the total number of pixels whose gradient direction falls into the i-th direction interval within the p-th local circular support region;

[0096] Step 3-3: From the main direction, the position of the... Starting with the interval, follow the interval traversal order and use... Constructing regional gradient feature description vectors This ensures that the descriptor maintains consistent representation under different rotation angles;

[0097] Steps 3-4: For line segments in the reference image The distance between the region gradient feature description vector of the line segment and the region gradient feature description vector of each line segment in the candidate line segment set in the image to be matched is calculated, and matching is performed according to the nearest neighbor and second nearest neighbor criteria. If the ratio of the optimal matching distance to the second optimal matching distance is less than a preset threshold, the line segment in the reference image is determined to be in the reference image. If a candidate line segment in the corresponding image to be matched is successfully matched, an initial line segment matching relationship is established; the initial line segment matching set is composed of all line segment pairs that satisfy the matching criteria.

[0098] Furthermore, the specific steps in step 4 for verifying the matching results using iterative geometric constraints include:

[0099] Step 4-1: Calculate the homography matrix based on the midpoint correspondence in the initial line segment matching set obtained in Step 3-4. :

[0100]

[0101] in, For reference image number The coordinates of the midpoint of the line segment This represents the coordinates of the midpoint of the line segment in the image to be matched. Using the scaling factor, the homography matrix is ​​solved based on the correspondence of midpoints of at least four initial line segment matching points; the length of the target line segment is obtained through homography transformation. Direction angle and midpoint ;

[0102] Step 4-2: For any set of matching line segments, calculate the line segment scale projection error. Angular projection error Midpoint projection error :

[0103]

[0104]

[0105]

[0106] and with , and The homography matrix is ​​calculated with the goal of minimizing the weighted sum. :

[0107]

[0108] Among them, the summation symbol This indicates that all matching line segments in the current matching set are accumulated; , , They are respectively , and The weighting coefficients, and in order to make the midpoint projection error have a higher constraint effect in the mismatch elimination process, can be taken as:

[0109]

[0110] Further eliminate mismatched lines and iteratively optimize the homography matrix to obtain the final matching result.

[0111] To verify the effectiveness of the method of the present invention, a high-resolution remote sensing image was selected as a reference image, such as... Figure 5 As shown; a matching image with scale variations, illumination variations, and rotation variations is generated based on the reference image, such as... Figure 6As shown in Figure 7, which is the matching result diagram of this embodiment, even when there are complex changes in the image such as scale, illumination and rotation, the method of the present invention can still achieve relatively accurate matching in linear structural areas such as road edges and building outlines, indicating that the method of the present invention has good matching accuracy and robustness.

[0112] This invention provides a high-resolution remote sensing image matching method that integrates line features and geometric constraints. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A method for matching high-resolution remote sensing images by fusing line features and geometric constraints, characterized in that, Includes the following steps: Step 1: Extract line segments from the reference image and the image to be matched; Step 2: Based on the line segments extracted in Step 1, construct multi-scale geometric feature descriptors according to their geometric information with neighboring line segments, and obtain a set of candidate lines by filtering through similarity measurement; Step 3: Based on the candidate line set obtained in Step 2, establish an initial matching relationship; Step 4: Based on the initial matching relationship obtained in Step 3, verify the matching result through iterative geometric constraint verification to obtain the final matching result.

2. The high-resolution remote sensing image matching method based on the fusion of line features and geometric constraints according to claim 1, characterized in that, The specific steps for extracting line segments from the reference image and the image to be matched in step 1 include: Step 1-1: Calculate the gradient magnitude and gradient direction of each pixel in the reference image and the image to be matched; Step 1-2: Perform region growth based on the consistency of gradient direction, and aggregate connected pixels with approximate gradient direction to form line support regions; Steps 1-3: Perform rectangular approximation on each line support region to obtain a rectangular region representing the line segment, and calculate its center, length, width and angle; Steps 1-4: Validate the fitted rectangular line segments to obtain the final line segment extraction results.

3. The high-resolution remote sensing image matching method based on the fusion of line features and geometric constraints according to claim 2, characterized in that, Steps 1-4 are as follows: Calculate the false alarm number (NFA) of the rectangular region based on the Helmholtz principle, verify the validity of the fitted rectangular line segments, and retain the line segments with NFA less than the threshold as the final line segment extraction results.

4. The high-resolution remote sensing image matching method based on the fusion of line features and geometric constraints according to claim 1, characterized in that, The multi-scale geometric feature descriptor construction step in step 2 includes: Step 2-1: For the line segment The length of its line segment, midpoint, and direction angle are respectively expressed as: , , ,by midpoint Let the coordinates be the center of the circle. For the diameter, the scale factor is defined as: The set of neighboring line segments of a circular neighborhood is defined as follows: in, for The midpoint; Step 2-2: For the line segment Its relative to the straight line segment The normalized geometric features are : in, , They are respectively The length and direction angle of the line segment; for The relative diameter, representing With describing the neighborhood Scale proportions; for and The radial relative distance; The angle between the direction angles of the two line segments; Steps 2-3: Line Segments Set of adjacent line segments The relationship between line segments is expressed as follows: relative diameter Relative distance , Direction angle The range of values Divide the line into N intervals, based on the relationship between the line segments. To obtain the line segment Histogram description.

5. The high-resolution remote sensing image matching method based on the fusion of line features and geometric constraints according to claim 4, characterized in that, Line segments in steps 2-3 The histogram is described as follows: in, The first in the histogram Values.

6. The high-resolution remote sensing image matching method according to claim 4, characterized in that, The similarity metric screening in step 2 to obtain the candidate line set specifically involves: Based on the calculated histogram descriptions of the line segments, the chi-square statistic is used to measure geometric feature similarity. in, Represents a line segment in a reference image. This represents candidate line segments in the image to be matched. For line segments The first in the histogram One value; Using multiple sets of scaling coefficients Repeat steps 2-1 to 2-3 above to calculate multiple sets of geometric feature similarity measures, realizing multi-scale geometric feature descriptor similarity measurement. Sort the similarities from high to low and select the top... The candidate straight lines are used as straight line segments. The candidate line set is generated by constructing multi-scale geometric feature descriptors for each line segment in the reference image and each line segment in the image to be matched, thereby generating their respective candidate line sets.

7. The high-resolution remote sensing image matching method based on line segment geometric information and region description features according to claim 5, characterized in that, The specific steps for constructing and matching the region gradient feature descriptor in step 3 include: Step 3-1: Divide the straight line segment Evenly divided into Line segments , with each line segment The midpoint is the center of the circle. Multiple local circular support regions are established with the length of the line segment as the diameter. These support regions are connected in sequence according to the direction of the line segment to form a sequence of support regions in the neighborhood of the line segment. Step 3-2: Calculate the number of pixels in each support region. gradient magnitude With direction Based on gradient direction The pixels within the support region are assigned to S equally divided intervals, and the gradient magnitudes of each interval are accumulated to generate a directional gradient statistical histogram. Step 3-3: From the main direction, the position of the... Starting from the interval, use the first interval in the interval traversal order. The sum of gradient magnitudes in each directional interval Constructing feature description vectors To maintain the consistency of the descriptor's representation under different rotation angles; Steps 3-4: Based on the candidate line set, perform regional gradient feature descriptor matching using the nearest neighbor and second nearest neighbor algorithm to establish initial matching relationships.

8. The high-resolution remote sensing image matching method based on line segment geometric information and region description features according to claim 7, characterized in that, The direction of the peak value of the histogram in step 3-2 is the main direction of the gradient feature of the current region.

9. The high-resolution remote sensing image matching method based on fused line features and geometric constraints according to claim 7, characterized in that, The first step described in step 3-3 The sum of gradient magnitudes in each directional interval The solution is as follows: 。 10. The high-resolution remote sensing image matching method according to claim 1, characterized in that, The specific steps for verifying the matching results using iterative geometric constraints in step 4 include: Step 4-1: Calculate the initial homography matrix based on the midpoint correspondence of the initial line segment matching. The length of the target line segment is obtained through homography transformation. Direction angle and midpoint ; Step 4-2: For any set of matching line segments, calculate the line segment scale projection error. Angular projection error Midpoint projection error : and with , and The homography matrix is ​​calculated with the goal of minimizing the weighted sum. : Among them, the summation symbol This indicates that all matching line segments in the current matching set are accumulated; , , They are respectively , and The weighting coefficients are used to eliminate mismatched lines, and the homography matrix is ​​iteratively optimized to obtain the final matching result.

Images