A remote sensing image registration model construction method based on motion consistency and ternary relationship
By introducing ternary relationships and fine spatial search strategies in image registration, combined with DBSCAN clustering algorithm, the problem of wrong matching in image registration in the existing technology is solved, efficient and accurate remote sensing image stitching is achieved, and image matching is improved Accurate robustness and accuracy.
Patent Information
- Application Number
- CN202411413655.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-11
- Publication Date
- 2024-12-27
- Estimated Expiration
- 2044-10-11
AI Technical Summary
The prior art is difficult to effectively remove mismatches caused by low-quality images and occlusions in image registration. Especially when there are complex spatial relationships and a large number of outliers, the geometric transformation model becomes unreliable, resulting in misalignment and heavy image stitching. Shadow and other issues.
Using a remote sensing image registration model based on motion consistency and ternary relationship, the ternary relationship principle and ternary space fine search strategy are introduced in the DBSCAN clustering algorithm to improve the robustness of feature matching and image registration accuracy, and eliminate the Outliers, achieving efficient and accurate image stitching.
It improves the accuracy and robustness of image registration, effectively eliminates error matching, ensures the accuracy and reliability of image stitching, and improves the overall quality of remote sensing image stitching.
Smart Images
Figure CN118941604B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of image registration, and in particular relates to a remote sensing image registration model construction method based on motion consistency and ternary relationship. Background Art
[0002] In recent years, with the development of aerospace technology, the use of aircraft combined with remote sensing equipment can replace manual work to efficiently complete some terrain detection tasks (such as mine detection, crop monitoring, and other tasks that require the acquisition of large-scale regional information). Therefore, remote sensing technology has also been welcomed by many remote sensing-related fields. Among them, image stitching technology is an important means to obtain remote sensing image information in a large area. Image stitching is divided into two stages: image registration and image fusion. Among them, the feature matching of image registration is the key to ensuring that the subsequent image stitching can be accurately fused. The matching problem usually includes two steps. First, a data set containing multiple sets of assumed matches is constructed between images through feature extraction algorithms (for example, scale-invariant feature transform (SIFT)), and then false matches are removed from the data set. However, in addition to correct matches (inliers), the assumed set also includes many false matches (outliers) caused by the ambiguity of local descriptors (especially when the image is affected by low quality and occlusion). These outliers provide incorrect coordinate information, so that the same places between pictures are not accurately aligned during image registration, which leads to misalignment, ghosting, and other phenomena after image stitching. Therefore, it is crucial to design a robust method to remove outliers to improve the matching reliability.
[0003] In the past few decades, false match removal methods have been developed, most of which assume that the putative matches satisfy an underlying geometric transformation model (e.g., homology, extrinsic geometry, affine), and then they adopt a geometric constraint to remove mismatches that do not meet the assumption. However, when there are complex spatial relationships between image pairs (especially for severe non-rigid deformations) or there are a large number of outliers, the geometric transformation model becomes unreliable. Summary of the invention
[0004] The purpose of the present invention is to improve the accuracy of regional image stitching in remote sensing tasks, provide a remote sensing image registration model construction method based on motion consistency and ternary relationship, which is applied to a remote sensing image platform for panoramic image stitching, and improve the outlier filtering capability and image registration accuracy by improving the feature matching algorithm, thereby realizing efficient and accurate stitching of remote sensing images.
[0005] To achieve the above object, the technical solution of the present invention is: a method for constructing a remote sensing image registration model based on motion consistency and ternary relationship, comprising:
[0006] The data set consisting of the data point pairs of two remote sensing images is expressed using motion consistency;
[0007] The ternary relationship principle is introduced into the DBSCAN clustering algorithm, and a ternary space fine search strategy is proposed to obtain the correctly matched data point pairs;
[0008] The correctly matched data point pairs are used for image registration and image fusion to obtain panoramic remote sensing images.
[0009] In one embodiment of the present invention, a data set composed of data point pairs of two remote sensing images is constructed by inputting two remote sensing images, performing feature extraction on the two remote sensing images using a feature extraction algorithm, and obtaining data point pairs corresponding to the two remote sensing images, thereby constructing a data set.
[0010] In one embodiment of the present invention, the specific implementation method of expressing the data set composed of the data point pairs of two remote sensing images by using motion consistency is as follows:
[0011] For the data set consisting of two remote sensing image data point pairs , and They are the first i The coordinate information of the data points, N is the number of data points, and express and The motion vector between D Expressed as , p i Representation dataset D The i data points; data set D The motion consistency between data points in the weighted distance It is expressed as follows:
[0012]
[0013] in, is the Euclidean distance between data points in the remote sensing image, is the weight parameter, Used to enhance the motion consistency between adjacent data points, It means taking the minimum value of the Euclidean distance between the data points in two remote sensing images.
[0014] In one embodiment of the present invention, the ternary relationship principle is introduced into the DBSCAN clustering algorithm, and a ternary space fine search strategy is proposed to obtain the correct matching data point pairs. The specific implementation method is as follows:
[0015] Based on weighted distance , by defining the radius threshold r to find a given data point p i The set of adjacent points ,like ,but p i is a candidate core point, where Representation calculation The number of midpoints, is the preset minimum quantity;
[0016] Introduce the ternary relationship principle to find alternative core points based on candidate core points;
[0017] A ternary space fine search strategy is proposed to search for core points based on candidate core points, generate clusters based on core points, remove outliers, and obtain correctly matched data point pairs in the data set composed of data point pairs of two remote sensing images.
[0018] In one embodiment of the present invention, the ternary relationship principle, that is, in the process of finding the core point, the candidate core point p i A point can be identified as a core point only when it forms a stable triplet relationship with adjacent data points.
[0019] In one embodiment of the present invention, the introduction of the ternary relationship principle and the search for candidate core points based on candidate core points are specifically implemented as follows:
[0020] Based on candidate core points p i Spatial Relationship Search K-Nearest Neighbors ,when There are p j , p k and p i When the following relationship is satisfied, these three points form a stable triple , at this time the candidate core point p i For alternative core points;
[0021]
[0022] in, express p i belong p j The K-nearest neighbors, Indicates when If true, the value is 1, otherwise it is 0.
[0023] In one embodiment of the present invention, a three-dimensional space fine search strategy is proposed, and the specific implementation method of searching for core points based on candidate core points is as follows:
[0024] The formula for the ternary space fine search strategy is defined as:
[0025]
[0026] in, Indicates the candidate core point p i The new point set, , Indicates that it can be combined with alternative core points p i A set of data points that form a stable triple; the above formula determines the data point q Is it the alternative core point? p i The data points that form a stable triplet and are also at the candidate core points p i The set of adjacent points If so, then as A point in ;
[0027] like There are alternative core points p i A set of data points that form a stable triplet, then the candidate core point p i That is the core point.
[0028] The present invention also provides a remote sensing image registration model construction system based on motion consistency and ternary relationship, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps described above can be implemented.
[0029] Compared with the prior art, the present invention has the following beneficial effects:
[0030] (1) This paper proposes a criterion for ternary relations, combining DBSCAN with ternary relations to improve the robustness of the feature matching method.
[0031] (2) This paper proposes an improved triple space search strategy to further constrain the triple relationship. This strategy only uses data points within the neighborhood radius to guide the triple relationship, making the identified core points more accurate and reliable. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1The present invention is a flowchart of a remote sensing image stitching method based on the present invention.
[0033] Figure 2 It is the ternary relationship guidance process; among them, (a) is the process of DBSCAN searching for core points, and (b) represents the introduction of ternary relationships in the process of searching for core points.
[0034] Figure 3 is a refined search strategy; among them, (a) is guided by ternary relations, and (b) introduces a refined search strategy into the ternary relations.
[0035] Figure 4 The remote sensing image stitching process; wherein, (a) is two input remote sensing images; (b) is the feature matching pairs between the images obtained through feature extraction; (c) is the feature matching pairs after outliers are eliminated by the method of the present invention; and (d) is the remote sensing panoramic image after image fusion.
[0036] Figure 5 is a cumulative distribution plot of the method results.
[0037] Figure 6 This is the partial splicing result in the dataset.
[0038] Figure 7 It is a semi-physical simulation verification platform. DETAILED DESCRIPTION
[0039] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings.
[0040] The present invention provides a remote sensing image registration model construction method based on motion consistency and ternary relationship, comprising:
[0041] The data set consisting of the data point pairs of two remote sensing images is expressed using motion consistency;
[0042] The ternary relationship principle is introduced into the DBSCAN clustering algorithm, and a ternary space fine search strategy is proposed to obtain the correctly matched data point pairs;
[0043] The correctly matched data point pairs are used for image registration and image fusion to obtain panoramic remote sensing images.
[0044] The present invention also provides a remote sensing image registration model construction system based on motion consistency and ternary relationship, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps described above can be implemented.
[0045] The following is the specific implementation process of the present invention.
[0046] like Figure 1As shown, the present invention proposes a remote sensing image registration model construction method based on motion consistency and ternary relationship, which is applied to a remote sensing image platform for panoramic image stitching. The outlier filtering capability and image registration accuracy are improved by improving the feature matching algorithm, thereby realizing efficient and accurate stitching of remote sensing images.
[0047] The method of the present invention: firstly, the motion consistency relationship of data points is used to provide information for clustering; then, a ternary relationship principle is incorporated when clustering is performed using DBSCAN, and the ternary relationship principle further uses the spatial relationship between data to search for inner points that have a more stable relationship with neighboring points; then, a ternary space fine search strategy is proposed, which is used to prevent the introduction of points that do not belong to the neighborhood radius when searching for ternary relationships, thereby further enhancing the reliability of the searched core points.
[0048] 1. Motion consistency expression
[0049] Since two adjacent feature points have similar motion properties, and similar motion properties usually indicate that they are the same type of data points. Therefore, a set of input data containing N pairs of data points is By using motion consistency to express, the data can be converted into a clustering problem. Specifically, for the data set consisting of data point pairs of two remote sensing images S of and is two-dimensional data, which respectively represents the coordinate information of the two input images. express and Therefore, the data set containing motion vector information is represented as , and use the weighted distance of formula (1) to represent the motion consistency between data points.
[0050] (1)
[0051] in, is the Euclidean distance between data points, is the weight parameter, Used to enhance the motion consistency between adjacent feature points. It means taking the minimum value of the Euclidean distance between the data points in two remote sensing images.
[0052] At this point, we can calculate the data set D The motion consistency relationship between each data point and other data points provides data information for the subsequent application of clustering algorithm.
[0053] 2. DBSCAN
[0054] Through formula (1), the present invention obtains the weighted distance between a data point and other data points. Therefore, the DBSACN clustering algorithm is used to generate clusters to eliminate outliers. It involves several key definitions: based on the weighted distance, the radius threshold r is defined to find the given point p i The set of adjacent points , and then identify the core points. ( Representation calculation The number of midpoints, is the minimum number of sample points), then p i is the core point. ,and p i is a core point, then point p j From the core point p i Direct density reachable. Three points p i , p k , p j ,if p i and p k is the core point, and p k is from p i Direct density reachable, p j is from p k Direct density is reachable, then p j is from p i Density is reachable. If the core point p i for p j and p k are all density-reachable, then p j and p k are density connected and belong to the same cluster. Non-density connected points, either core points or points directly density reachable from core points, are assigned to different clusters; otherwise, they are considered outliers.
[0055] However, due to the neighborhood radius threshold r and the minimum number of samples in DBSCAN is fixed, the algorithm faces the following problem when dealing with a large number of outliers: the existence of outliers within the radius threshold r can lead to unexpected results, that is, causing data points that meet the minimum number of samples to be mistakenly identified as core points.
[0056] In order to solve this problem, the present invention further utilizes the spatial relationship between data points and introduces a triple relationship to guide the core point search. That is, based on the weighted distance and radius threshold r, a given point is found. p i Neighboring points , as candidate core points, and then use the spatial relationship between data points and introduce triple relationships to guide the core point search.
[0057] 3. The ternary relationship principle
[0058] Specifically, the data points p i A point can only be identified as a core point if it forms a stable triplet relationship with its neighboring points. The detailed explanation of the triplet relationship is as follows:
[0059] In the dataset Based on the data points p i The spatial relationship between them searches for their K nearest neighbor sets Only if the following conditions are met p i , p j and p k The three points can form a stable triplet only when there is a relationship between .
[0060]
[0061] in, express p i belong p j The K-nearest neighbors, Indicates when If true, the value is 1, otherwise it is 0.
[0062] After being guided by the ternary relationship, the core points with stable relationships can be obtained. The principle is as follows Figure 2 As shown. Figure 2 In , it is assumed that the minimum number of sample points is 4 (including core points), where Figure 2 (a) is the process of DBSCAN searching for core points. Figure 2 (b) shows that the ternary relationship is introduced in the process of searching for core points. Figure 2 In (a), due top The minimum number of sample points is satisfied within the neighborhood radius of 1, so p 1 is considered as the core point; and p 4 also satisfies the minimum number of sample points within the neighborhood radius, then p 4 is also considered a core point, so p 5 is incorrectly classified as an inlier. Figure 2 (b) p The nearest neighbor of 1 is p 2. p 3. p The nearest neighbor of 2 is p 1. p 3. p The nearest neighbors of 3 are p 1. p 2, therefore p 1. p 2. p 3 can form a stable set of triples, then p 1 is regarded as the core point; p 4. Due to p The nearest neighbors of 4 are p 3. p 5. But p 3's neighbors do not include p 4 and p 5 (i.e. p 4 cannot form a stable triplet), so p 4 cannot be considered as a core point, which means p 4 and p 5. It is impossible to form a direct density-reachable relationship, thereby effectively filtering p 5.
[0063] This can effectively overcome the interference of outliers when there are a large number of outliers in the data. However, in the process of using the ternary relationship to guide the core point, since the search for K-nearest neighbor points uses a global search, when the number of K nearest neighbor points is greater than the number of points within the neighborhood radius, the points outside the radius that can be connected to the core point will be discarded. p i The points that form a ternary relationship are merged into the data subset and counted into the sample points for judging the core points. This result will affect the incorrect judgment of the core points. Assuming the minimum number of sample points =4, and the number of neighbor points K=5, such as Figure 3 As shown in (a), for a set of data points , where p 1 cannot form a triple with other points within the neighborhood radius, but its sum p 4. p 5 forms a triple relationship, which will lead to p 1 is mistakenly considered as a core point.
[0064] Therefore, the present invention proposes a ternary space fine search strategy to guide the constraint ternary relationship. That is, based on the ternary relationship principle, after obtaining the candidate core points, the core points are selected through the ternary space fine search strategy.
[0065] 4. Ternary Space Detailed Search Strategy
[0066] This strategy evaluation can be done with p i The points that form a triple (denoted by ).if B i If a point in does not belong to range r, it is deleted. Therefore, it only retains the triplet relationship within the neighborhood radius r.
[0067] The formula for this strategy is defined below.
[0068] (2)
[0069] in, express p i The new point set, express p i The point set within the surrounding radius r, and Need to meet:
[0070]
[0071]
[0072] in, Indicates that it can be p i The set of points that form triples (with p i More than one triplet is associated). q is a data point. Figure 3 As shown in (b), formula (2) is used to restrict the determination of the ternary relationship and to determine whether the data points of the formed triplet are within the neighborhood radius. p 5 is not a data point within the neighborhood radius, so the triple is judged is unreliable, and p 1 does not have any data that can form a triple with it, so it is considered p 1 cannot be used as a core point.
[0073] Through this strategy, the present invention further determines data points that can form reliable relationships, further improving the reliability of the inliers.
[0074] 5. Principle of remote sensing image stitching based on image registration
[0075] The image stitching of the present invention is based on image registration. By extracting and pairing the feature points of the two input remote sensing images, the feature matching pairs between the two images are screened out, and then pixel alignment is performed based on the feature matching pairs. Then, the image fusion method is used to eliminate the color difference, thereby realizing the stitching of the remote sensing scene. The basic process of image stitching is as follows: Figure 4 As shown. Figure 4 (a) shows two input remote sensing images; Figure 4 (b) is the feature matching pair between images obtained through feature extraction; Figure 4 (c) is a feature matching pair after outliers are removed by the method of the present invention; Figure 4 (d) in the middle is the remote sensing panoramic image after image fusion.
[0076] 6. Simulation analysis
[0077] To demonstrate the effectiveness of the proposed algorithm, we compared it with five state-of-the-art feature matching methods using two remote sensing datasets: RANSAC, RANSAC++, LAF, LGSC, and EES. We evaluated each algorithm based on accuracy, recall, F-score, and running time. In addition, a cumulative distribution plot was used to intuitively represent the matching results of each method. The results are shown in Figure 2. Figure 5 shown.
[0078] From the above experimental results, it can be seen that LGSC and LAF have high accuracy. However, the strict constraint relationship of LGSC and LAF makes them sensitive to outliers, resulting in low recall. RANSAC and EES have loose constraints on data points, have strong recall ability, and can effectively retain inliers. However, this also leads to the retention of many outliers, reducing their accuracy and F scores. The method of the present invention achieves high accuracy (i.e., CIAP: 99.98%) while ensuring high recall (i.e., CIAP: 98.70%) without significantly sacrificing the correct match rate. This is because the method of the present invention utilizes triple relationships to reduce the interference of outliers. In addition, the improved spatial triple search strategy further enhances the reliability of the identified core points, thereby ensuring more reliable inlier detection. Therefore, the method of the present invention obtains the highest F score in both CIAP and UAV datasets. In contrast, the method of the present invention utilizes triple relationships to achieve high accuracy while maximizing the retention of correct matches (inliers).
[0079] Figure 6 This is a partial splicing result of a data set using the method of the present invention.
[0080] At the same time, the remote sensing image registration model construction method can be verified by semi-physical simulation based on the Links-UAV-MiniBench, a quadrotor gantry system developed by Links Chuangqi. Figure 7 Specifically, the real scene is photographed by the gimbal camera carried by the drone, and then the acquired images are transmitted to the host computer through the image transmission device, and the input images are processed by Matlab; after the experiment, all image registration results will be exported in Matlab and stored in the specified folder of the host computer, which is convenient for analyzing, evaluating and using the registration results of the proposed remote sensing image registration model construction method based on motion consistency and ternary relationship.
[0081] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions do not exceed the scope of the technical solution of the present invention, belong to the protection scope of the present invention.
Claims
1. A method for constructing a remote sensing image registration model based on motion consistency and ternary relationship, characterized in that: include: The data set consisting of the data point pairs of two remote sensing images is expressed using motion consistency; The ternary relationship principle is introduced into the DBSCAN clustering algorithm, and a ternary space fine search strategy is proposed to obtain the correctly matched data point pairs; Use correctly matched data point pairs to perform image registration and image fusion to obtain panoramic remote sensing images; The specific implementation method of expressing the data set composed of the data point pairs of two remote sensing images by using motion consistency is as follows: For the data set consisting of two remote sensing image data point pairs , and They are the first i The coordinate information of the data points, N is the number of data points, and express and The motion vector between D Expressed as , p i Representation dataset D The i data points; Dataset D The motion consistency between data points in the weighted distance It is expressed as follows: ； in, is the Euclidean distance between data points in the remote sensing image, is the weight parameter, Used to enhance the motion consistency between adjacent data points, It means taking the minimum value of the Euclidean distance between each data point in two remote sensing images; The ternary relationship principle is introduced into the DBSCAN clustering algorithm, and a ternary space fine search strategy is proposed to obtain the specific implementation method of the correctly matched data point pairs as follows: Based on weighted distance , by defining the radius threshold r to find a given data point p i The set of adjacent points ,like ,but p i is a candidate core point, where Representation calculation The number of midpoints, is the preset minimum quantity; Introduce the ternary relationship principle to find alternative core points based on candidate core points; A three-dimensional space fine search strategy is proposed to search for core points based on candidate core points, generate clusters based on core points, remove outliers, and obtain correctly matched data point pairs in the data set composed of data point pairs of two remote sensing images. The ternary relationship principle means that in the process of finding the core point, the candidate core point p i A point can be identified as a core point only when it forms a stable triplet relationship with adjacent data points.
2. The method for constructing a remote sensing image registration model based on motion consistency and ternary relationship according to claim 1, characterized in that: The data set composed of data point pairs of two remote sensing images is constructed as follows: two remote sensing images are input, feature extraction is performed on the two remote sensing images using a feature extraction algorithm, and data point pairs corresponding to the two remote sensing images are obtained to construct the data set.
3. The method for constructing a remote sensing image registration model based on motion consistency and ternary relationship according to claim 1, characterized in that: The specific implementation of introducing the ternary relationship principle and finding candidate core points based on candidate core points is as follows: Based on candidate core points p i Spatial Relationship Search K-Nearest Neighbor Set ,when There are p j , p k and p i When the following relationship is satisfied, these three points form a stable triple , at this time the candidate core point p i For alternative core points; ； in, express p i belong p j The K-nearest neighbors, Indicates when If true, the value is 1, otherwise it is 0.
4. The method for constructing a remote sensing image registration model based on motion consistency and ternary relationship according to claim 3, characterized in that: The proposed three-dimensional space fine search strategy is specifically implemented as follows: The formula for the ternary space fine search strategy is defined as: ； in, Indicates the candidate core point p i The new point set, , Indicates that it can be combined with alternative core points p i A set of data points that form a stable triple; the above formula determines the data point q Is it the alternative core point? p i The data points that form a stable triplet and are also at the candidate core points p i The set of adjacent points If so, then as A point in ; like There are alternative core points p i A set of data points that form a stable triplet, then the candidate core point p i That is the core point.