A method for matching multi-satellite remote sensing images with constraints of coral reef features

Abstract

The present application relates to a kind of coral reef feature constrained multi-satellite remote sensing image matching method, steps include: based on coral reef reference range, the image to be matched is carried out overlay analysis and cutting, obtains the set of to-be-processed tile and reference set;The set of to-be-processed tile is carried out median filtering, index calculation and area threshold screening, extracts the set of to-be-matched coral reef range and image set;Respectively extract the global feature and local feature of coral reef;Through multi-feature similarity measure criterion, coarse match of coral reef is carried out, constructs geometric transformation model, is corrected using local feature point, and finally realizes local precision matching by least square gray matching.It solves the matching difficult problem caused by the texture of coral reef medium resolution remote sensing image is poor, repeat, unstable and scale difference, realizes the automatic matching of multi-satellite remote sensing image in far sea coral reef area, and after matching, RMSE can be reduced to 1.5 pixels, provides technical and data support for coral reef water and land change monitoring.

Description

Technical Field

[0001] This invention relates to a multi-satellite remote sensing image matching method constrained by coral reef features, and in particular to a hierarchical matching strategy that integrates global and local features. This method can solve the matching difficulties caused by the lack of texture, repetition, and instability of regional high-resolution remote sensing images in offshore coral reef areas, as well as the large spatial resolution span of multi-satellite images. It enables automatic matching of multi-source, multi-scale remote sensing images such as Sentinel, Landsat, GF, and HJ. Background Technology

[0002] Image matching is a prerequisite for constructing time-series change monitoring using collaborative multi-satellite remote sensing imagery. Compared to field surveys, multi-satellite remote sensing imagery enables low-cost, high-frequency studies of wide-area coral reefs, making it suitable for long-term coral reef monitoring. To improve the consistency of geometric accuracy between Sentinel-2 and Landsat imagery products, the United States Geological Survey (USGS) has established over 5 million control points globally. However, no control points have been deployed in marine islands, resulting in some geometric errors in Landsat imagery products even after geometric correction. Although domestically produced multi-source remote sensing imagery has undergone systematic geometric correction, the lack of reference images and control points in marine islands leads to inconsistent positioning errors between domestic satellite remote sensing imagery and Sentinel-2 MSI remote sensing imagery.

[0003] Coral reef medium-resolution remote sensing images exhibit poor, repetitive, and unstable textures. The large scale differences in multi-satellite remote sensing images easily lead to "ill-conditioned solutions" in matching, making them difficult areas for matching. Specifically, poor texture exists in areas with small gray-scale differences, such as the deep sea, most reef flats, and beaches. Similar or identical substrates within coral reefs appear as repetitive textures in the images. Natural / human factors such as clouds, waves, sandbar migration, and land reclamation areas cause significant differences in texture information that is rarely seen in remote sensing images of different time phases and spatial resolutions. General matching algorithms are not suitable for matching coral reef medium-resolution multi-satellite remote sensing images, resulting in uneven distribution of matching pairs, numerous mismatches, and no matching pairs.

[0004] Currently, many hierarchical matching methods that integrate different features have been used for image matching in special texture scenes and have proven applicable to multi-source, multi-scale, and spatially differentiated image data. However, there is still little research on the matching problem between multi-satellite remote sensing images of coral reefs. The outline of coral reefs can be clearly distinguished in remote sensing images, and is minimally affected by natural and human activities, exhibiting near-invariant characteristics, making it suitable as an invariant feature constraint for multi-satellite remote sensing image matching. Furthermore, the high positioning accuracy of Sentinel-2 image products allows the coral reef extent extracted from Sentinel-2 satellite images and the synthesized optimal image to serve as a reference benchmark for multi-satellite image matching of coral reefs in marine archipelagos. Therefore, it is urgent to address the following issues: which feature(s) should be selected as auxiliary constraints to improve the accuracy of coral reef matching; and how to quickly and stably extract and describe the constraining features in medium-resolution multi-satellite remote sensing images of coral reefs. Summary of the Invention

[0005] The technical problem this invention aims to solve is to overcome the shortcomings of existing general matching algorithms and address the "ill-conditioned solutions" in medium-resolution remote sensing images of coral reefs, such as uneven distribution of matching pairs, numerous mismatches, and no matching pairs caused by poor texture, repetition, instability, and large scale differences in multi-satellite remote sensing images. Compared to traditional methods, this invention fully considers the near-invariant shape of coral reefs and proposes a hierarchical matching strategy for multi-satellite remote sensing images that integrates multiple feature constraints of coral reefs, using coral reefs as the matching primitive.

[0006] To address the aforementioned technical issues, this invention uses the Sentinel-2 reef extent and imagery as a reference benchmark to construct a multi-feature similarity criterion that integrates global features (geometric, texture, and spectral features) and local features (locally invariant features) to achieve coarse matching of reefs with the same name. Furthermore, it utilizes the geometric features and locally invariant feature points of the reefs with the same name to construct and correct a geometric transformation model, and employs a grayscale matching method to achieve fine matching of local reefs.

[0007] The proposed multi-feature hierarchical matching strategy for coral reefs has the following advantages:

[0008] (1) High-precision automatic matching is achieved, the matching points are evenly distributed, avoiding the distortion problem caused by "ill-conditioned solutions" after transformation. The normalized mutual information (NMI) is significantly improved after matching, and the root mean square error (RMSE) is reduced from a large pixel error before matching to within the pixel level.

[0009] (2) This invention integrates global and local features, solves the problem of automatic image matching across scales, and can correctly match the vast majority of coral reefs.

[0010] (3) The present invention has a high matching success rate. In the automatic matching test of a large number of Landsat series satellite images of key islands and reefs in the sea area, the matching success rate is significantly improved compared with SIFT and Boosted Efficient Binary Local Image Descriptor (BEBLID).

[0011] To meet the needs of long-term time-series monitoring of key islands and reefs in marine areas, it is necessary to collaboratively process remote sensing images from multiple sources and at multiple scales, specifically involving Landsat-4 / 5 TM, Landsat-7 ETM+, Landsat-8 OLI, GF-1 / 6 WFV, GF-4 PMI, and HJ-1A / B CCD, totaling 9 satellites and 6 types of sensors. These sensor images from different sources exhibit significant spatial resolution variations and substantial geometric location differences. This invention solves the matching challenge of large geometric differences and wide spatial resolution variations encountered when matching these images in areas with scarce texture features, such as coral reefs. It achieves automatic matching of these remote sensing images of key islands and reefs in marine archipelagoes and proposes a reliable hierarchical matching strategy that integrates multiple feature constraints of coral reefs, providing technical and data support for monitoring land-water changes in key islands and reefs in marine areas. This invention solves the matching problem of large geometric differences, lack of texture features, and large spatial resolution span of multi-satellite remote sensing images of coral reefs. It realizes automatic matching of remote sensing images of key islands and reefs in the marine archipelago from 9 satellites of the Landsat, GF, and HJ series and 6 types of sensors. It proposes a reliable hierarchical matching strategy that integrates multi-feature constraints of coral reefs, providing technical and data support for monitoring water and land changes of key islands and reefs in the marine area. Attached Figure Description

[0012] The invention will now be further described with reference to the accompanying drawings.

[0013] Figure 1 This is a flowchart illustrating the overall process of an example of the present invention.

[0014] Figure 2 This is a schematic diagram of the coral reef matching patch analysis process in an example of the present invention.

[0015] Figure 3 This is a schematic diagram illustrating the SIFT local invariant feature point extraction method as an example of the present invention.

[0016] Figure 4 This is a schematic diagram showing the transformation of the geometric contour features of the coral reef with the same name into pairs of points of the same name in an example of the present invention.

[0017] Figure 5 This is a schematic diagram of local feature point pairs that satisfy the ratio threshold and point pairs within the geometric contour of the same coral reef, representing an example of the present invention.

[0018] Figure 6 This is a schematic diagram illustrating the determination of the initial window for grayscale matching of a local coral reef in an example of the present invention.

[0019] Figure 7 This is a schematic diagram of a partial coral reef fine matching in an example of the present invention.

[0020] Figure 8 The figure shows the matching results of different methods for the GF-6 WFV remote sensing image of Zhenghe Reefs on March 19, 2021, as an example of this invention. The numbers in the figure represent the final number of matching pairs and the total number of matching pairs.

[0021] Figure 9 This invention provides an example of a comparison of local details between a GF-6 WFV remote sensing image of the Zhenghe Reefs taken on March 19, 2021, after transformation using different methods, and a Sentinel-2 MSI reference image.

[0022] Figure 10 This is an example of the error situation in the remote sensing image of Zhenghe Reefs (March 20, 2015, GF-1 WFV remote sensing image). The numbers in the figure represent the final number of matching pairs and the total number of matching pairs.

[0023] Figure 11 This is an example of the Zhenghe Reefs remote sensing image without matching pairs (January 22, 2020, GF-4 PMI remote sensing image). The numbers in the figure represent the final number of matching pairs and the total number of matching pairs.

[0024] Figure 12 The accuracy of matching pairs using different methods and sensors is shown in the example of the Zhenghe Reefs of this invention.

[0025] Figure 13 The NMI calculation results are shown in the example of the Zhenghe Reefs of this invention after matching different methods and different sensors. Detailed Implementation

[0026] The present invention will now be described in detail with reference to the accompanying drawings, which will make the technical approach and operation steps of the present invention clearer.

[0027] This invention uses a medium-resolution remote sensing image of a group of reefs in a certain sea area as an example to illustrate a multi-satellite remote sensing image matching method constrained by coral reef features. The remote sensing images of these areas suffer from problems such as poor texture, repetition, and instability, and there are often significant scale differences between the matched images, making them difficult to match.

[0028] This example uses this research area as a case study to illustrate the method of the present invention, as shown in the flowchart. Figure 1 As shown, the specific steps include:

[0029] Step 1: Candidate Reef Analysis – Based on the Sentinel-2 reef extent, spatial analysis methods are used to crop the image to be matched into multiple patches. The specific process is as follows:

[0030] (1) Analysis of the Coral Reef Range Buffer Zone. Visually, within the archipelago area, considering Landsat-4 / 5TM, Landsat-7 ETM+, Landsat-8 OLI, GF-1 / 6 WFV, GF-4 PMI, and HJ-1A / B CCD, a total of 9 satellites and 6 sensor types, the GF-4 PMI imagery showed the largest geometric deviation compared to Sentinel-2 MSI imagery, but the deviation was generally no more than 5 km. Therefore, a buffer zone with a radius of 5 km was established for the coral reef area.

[0031] (2) Clipping of the set of tiles to be matched. For example... Figure 2 As shown, the coverage area of ​​the entire image to be matched is overlaid with the coral reef buffer result in (1). If the two intersect, the entire image to be matched is cropped into several image blocks to be matched, and all image blocks to be matched form a set of image blocks to be processed. Otherwise, skip that image.

[0032] (3) Reference range and image set selection. For example... Figure 2 As shown, based on the set of tiles to be processed The coral reef extent is compared with the Sentinel-2 MSI remote sensing image, and a reference coral reef extent set is selected. Reference coral reef image set If the quality of the Sentinel-2 MSI remote sensing image is poor, for example, if there are clouds or cloud shadows within the coral reef area, the best Sentinel-2 MSI remote sensing image can be synthesized from multiple images.

[0033] Step 2, Single-phase coral reef extent extraction – To achieve efficient matching later, it is necessary to extract the extent from the set of tiles to be processed. To quickly and accurately extract coral reefs from single-temporal remote sensing images, this invention designs a workflow for extracting coral reefs based on single-temporal remote sensing images. This workflow mainly uses NDRI, NDWI, and MNDWI indices, combined with clustering and mathematical morphology methods, to quickly and accurately extract the coral reef range in the image to be matched, and remove interference noise such as solar flares, wave spray, and broken clouds. The specific workflow is as follows:

[0034] (1) Image denoising. Although atmospheric correction has been performed in data preprocessing, it is still not possible to effectively eliminate or reduce noise such as solar flares in the image. Since the distribution of interference noise such as flares, broken clouds, and ships is irregular, they are mostly isolated points in medium resolution images. In order to minimize the mis-extraction of background noise, a 3×3 window median filter is used.

[0035] (2) Index Calculation. Calculate the NDRI, NDWI, and MNDWI indices for the patches to be matched. If the shortwave infrared band is missing when calculating MNDWI, only NDRI and NDWI are calculated. The formulas for each index are as follows:

[0036]

[0037]

[0038]

[0039] in, The surface reflectance is in the near-infrared band. The surface reflectance is in the red-edge band. The surface reflectance is in the green light band. This represents the surface reflectance in the shortwave infrared band.

[0040] (3) Coral Reef Extraction and Optimization. The Fuzzy c-Means algorithm was used to divide the NDRI, NDWI, and MNDWI index results into foreground coral reefs and background deep sea, respectively. Subsequently, the foreground results of the three indices were merged, and opening and closing operations were constructed using erosion and dilation operations in morphological filtering. This eliminated isolated noise and fine cracks in the segmentation results while preserving the shape and position of the segmentation results. The process can be described as follows:

[0041]

[0042]

[0043]

[0044]

[0045] in, The input image is the classification result based on the index. For structural elements; For erosion operations; For expansion operation; This refers to the position of the structuring element after translation; Indicates an inclusion relationship; For the result set; for The reflection; The intersection of sets; The intersection is not empty; , They represent opening and closing operations, respectively.

[0046] To fill the foreground voids caused by misclassification of the lagoon floor in the atoll, this embodiment uses the maximum connectivity analysis method to extract and optimize the reef extent. Specifically, by extracting the maximum connectivity of the background deep sea, the inverted result is used as the extraction result of the foreground reef.

[0047] (4) Area threshold screening. Extracting coral reefs from single-temporal images inevitably leads to the mis-extraction of interfering factors such as broken clouds and ships. To reduce invalid calculations of subsequent multi-feature images, a reference coral reef range set is used. The minimum area of ​​the exposed reef is used as a threshold, and erroneous extractions with areas smaller than the threshold are removed.

[0048] Following the above process, extract the set of tiles to be processed. The range of medium coral reefs is used as the set of coral reef ranges to be matched. To match the coral reef range set Tile set to be processed The cropped result serves as the image set of coral reefs to be matched. .

[0049] Step 3: Multi-feature extraction and description of coral reefs – using the range set of coral reefs to be matched Coral reef image set to be matched Coral reefs in the image are used as matching primitives to describe their global features (geometric features, spectral features, texture features) and local features (local invariant features), which makes up for the inherent defects of poor texture, repetition and instability in coral reef images, and solves the problem of difficulty in matching based solely on local invariant features.

[0050] The extraction of global and local features is detailed below:

[0051] (1) Geometric feature extraction. To ensure that the extracted features are invariant to translation, rotation, and scale, Hu geometric invariant moments are selected to describe the geometric features of the coral reef spatial extent. The relevant parameter calculation formulas are as follows:

[0052]

[0053]

[0054]

[0055]

[0056]

[0057]

[0058]

[0059] in, This represents the i-th geometrically invariant moment of Hu. The normalized central moment is expressed by the following formula:

[0060]

[0061] in, Represents the coordinates of a pixel. The centroid coordinates representing the outline of the coral reef are the first geometric moment, calculated using the following formula:

[0062]

[0063]

[0064] in, These are the coordinates of the centroids of the n finite simple figures that have been subdivided. It is the area of ​​the n finite simple shapes that are divided into. The coordinates are the center of gravity of the reef.

[0065] Since the calculated seven geometrically invariant moments differ greatly in magnitude and may even be negative, a method using the natural logarithm is employed to compress the data of the seven geometrically invariant moments without altering their translational, rotational, and scale-invariant properties. The calculation formula is as follows:

[0066]

[0067] Reference coral reef vector range set Set of coral reefs to be matched The reference coral reef geometric feature set was calculated separately. Geometric feature set of coral reefs to be matched .

[0068] (2) Spectral Feature Description. Histogram statistics were used to describe the spectral features of the coral reef area. Considering the diverse underwater landforms and surface cover of different coral reefs, and taking into account the use of equal wavelengths across different sensors, grayscale histograms were calculated for the blue, green, red, and near-infrared bands. The blue and green bands provide richer underwater landform information, while the red and near-infrared bands, although less penetrating, can supplement more information on surface cover. Therefore, grayscale histograms were calculated for each of the four bands, as follows:

[0069]

[0070] Where L is the number of grayscale colors after quantization. L is the number of pixels with value k in the grayscale image, and N is the total number of pixels in the grayscale image. The larger the value of L, the more levels of grayscale histogram quantization there are, and the more accurate the description of the coral reef area. A value of 256 was chosen for L.

[0071] Reference coral reef image set The spectral characteristics of blue, green, red, and near-infrared light bands were statistically analyzed to form a reference coral reef spectral feature set. Similarly, calculate the set of coral reef images to be matched. The spectral feature set of the coral reef to be matched was obtained. .

[0072] (3) Texture Feature Description. The texture within coral reefs reflects the spatial structural characteristics of different landforms. Different coral reefs are affected by environmental factors such as nature, sedimentation, and hydrodynamics during their growth and development, resulting in different texture features in remote sensing images. To describe the overall texture features within a coral reef area, this study introduces the GIST feature descriptor. The calculation process is as follows: First, define m-scale, n-direction Gabor filters and convolve them on the input image to obtain m×n feature maps of the same size as the input image. Second, divide each feature map into r×c blocks, calculate the mean of each block, and statistically obtain m×n×r×c dimensional GIST features. r, c, and m are set to 4, and n is set to 8. The input image uses a blue light band grayscale image that penetrates deeper into the water and reflects more texture information, ultimately resulting in 512-dimensional GIST features.

[0073] Following the above procedure, the reference coral reef image set was processed. The blue light band texture features are calculated to form a reference coral reef texture feature set. Calculate the set of coral reef images to be matched The blue light band was used to obtain the coral reef texture feature set to be matched. .

[0074] (4) Local Invariant Feature Descriptor. To discover and match as many corresponding points as possible, the robust SIFT local invariant feature descriptor algorithm was selected. For example... Figure 3 As shown, a blue light band grayscale image is still used. First, a scale space is constructed and extreme points are detected to locate feature points in the grayscale image. Then, the gradient magnitude and direction in the neighborhood of the feature point are counted. The direction of the maximum value in the histogram is selected as the main direction. Finally, the 16×16 neighborhood of the feature point is divided into 16 4×4 windows. The histogram composed of 8 gradient directions in each window is counted to form the 128-dimensional feature of the feature point.

[0075] Step 4: Hierarchical Matching Strategy – Based on the extracted global and local features (geometric features, spectral features, texture features, and local invariant features), a hierarchical matching strategy is constructed, divided into coarse matching and fine matching. First, the geometric, spectral, and texture similarities between the reef range set and the reef image set to be matched are calculated, and coarse matching of reefs with the same name is performed according to the multi-feature similarity metric. Then, an initial geometric transformation model is constructed using the geometric features of the reefs with the same name, and the model is corrected by fusing the selected local invariant feature point pairs. Finally, an initial window is established based on the corrected model, and iterative matching of local reefs is completed using grayscale information and the least squares method.

[0076] The specific process for this step is as follows:

[0077] (1) Coarse matching of reefs with the same name. A regional multi-feature similarity measurement criterion integrating geometric features, spectral features, and texture features is constructed. The similarity between two reefs is calculated, which is transformed into measuring the distance or similarity between features. Commonly used methods include Euclidean distance and cosine similarity. Euclidean distance is simple and convenient to use, but its effect becomes smaller and smaller as the data dimension increases. Cosine similarity can effectively offset the above problems, but it ignores the magnitude of the vector while considering the vector direction. This invention selects the adjusted cosine similarity algorithm to measure the similarity between features. Unlike the cosine similarity measurement algorithm, the adjusted algorithm uses features with zero mean to perform cosine similarity calculation. For k-dimensional features and The adjusted cosine similarity formula is as follows:

[0078]

[0079] in, and Features and The mean. Represents the first of the range sets of coral reefs to be matched. One characteristic, This represents the feature mean of the set of coral reef ranges to be matched. This indicates the first image in the coral reef image set to be matched. One characteristic, This represents the feature mean of the coral reef image set to be matched. This represents the feature dimension. The adjusted cosine similarity ranges from [-1, 1]. and The more similar the two numbers are, the larger the value. When they are completely identical, the maximum value is 1, and when they are completely opposite, the minimum value is -1.

[0080] Based on the obtained reference coral reef geometry Geometric features of the coral reef to be matched Substituting the adjusted cosine similarity algorithm, the geometric feature similarity can be expressed as:

[0081]

[0082] in, and The range of the i-th reference coral reef is respectively and the range of the j-th unmatched coral reef The l-th geometrically invariant moment, and These are the ranges of the i-th reference coral reef. and the range of the j-th unmatched coral reef The mean of the geometric invariant moments. The value range is [-1, 1].

[0083] Based on the spectral characteristics of coral reefs in the blue, green, red, and near-infrared bands, the calculation weights of different bands are obtained by statistically analyzing histograms of different bands. The similarity results of the four bands are then combined to obtain the spectral feature similarity results between the coral reef to be matched and the reference coral reef region. The calculation formula is shown below:

[0084]

[0085]

[0086]

[0087] in, It is the weight of the l-th band, and , is the maximum frequency value of the histogram of the c-th band. The four bands used for statistical analysis are the blue light band, green light band, red light band, and near-infrared band. In the formula, It is the i-th reference coral reef image and the j-th coral reef image to be matched The cosine similarity in the l-th band, and These represent the values ​​at the u-th level in the histogram corresponding to the l-th band. and These are the histogram mean values ​​corresponding to the l-th band. It is the i-th reference coral reef image and the j-th coral reef image to be matched The spectral feature similarity is in the range of [-1, 1].

[0088] Based on the obtained reference coral reef texture features Coral reef texture features to be matched Substituting the adjusted cosine similarity algorithm, texture feature similarity can be expressed as:

[0089]

[0090] in, and These are the i-th reference coral reef images. and the j-th coral reef image to be matched The value at level l and These are the i-th reference coral reef images. and the j-th coral reef image to be matched The mean of texture features. It is the i-th reference coral reef image and the j-th coral reef image to be matched The texture feature similarity has a value range of [-1, 1].

[0091] Based on the above similarity calculations for different features, a multi-feature similarity criterion for coral reefs, combining geometric, spectral, and textural features, is constructed. The core idea is to utilize the fact that the outline morphology and underwater features of coral reefs remain largely unchanged, while the aforementioned features differ between different coral reefs. This multi-feature similarity criterion can be described as follows:

[0092]

[0093] Represents the i-th reference coral reef image Matching the j-th coral reef image Multi-feature similarity between them It is a symbolic function; , , These represent geometric feature similarity, spectral feature similarity, and texture feature similarity, respectively. In establishing a multi-feature similarity metric, the contribution of different features needs to be considered. For example, there may be cases where the texture structure of the reference coral reef image and the coral reef image to be matched is flat and the texture regions are not obvious. In such cases, spectral features should be given higher weight. Therefore, these are used respectively... and The weights representing spectral and texture features can be adaptively calculated using the spectral histogram. and :

[0094]

[0095]

[0096] in, and These are the i-th reference coral reef images. and the j-th coral reef image to be matched The maximum frequency of the histogram in the blue light band. Compared to other bands, the blue light band has the strongest water penetration ability and can acquire more underwater information. When the texture structure is obvious, the spectral histogram is relatively dispersed, and the maximum frequency of the histogram is small; when the texture structure is flat and the texture area is not obvious, the spectral histogram is relatively concentrated, and the maximum frequency of the histogram is large.

[0097] Determining adaptive spectral weights and texture weight Then, the reference coral reef image was calculated. and the coral reef image to be matched Multi-feature similarity among various coral reefs This indicates whether the minimum value among geometric feature similarity, spectral feature similarity, and texture feature similarity is negative. If a negative value is found, then the multi-feature similarity is... Since it is negative, its range of values ​​is [value missing]. Finally, using the maximum likelihood method, the coral reefs with the highest similarity are grouped into coral reef pairs with the same name.

[0098] To solve the geometric transformation model between the reference image and the image to be matched, the corresponding coral reef pairs constructed in the coarse matching need to be simplified from geometric contours to corresponding point pairs. During the transformation process, to ensure translation, rotation, and scale invariance between the corresponding point pairs, five corresponding point pairs were selected for each pair of corresponding coral reefs, including the four corner points of the smallest bounding rectangle of the coral reef contour and the centroid of the coral reef contour. For example... Figure 4 As shown, the orange lines represent the corresponding point pairs between the reference reef image and the reef image to be matched. Ultimately, for the reference reef image... and the coral reef image to be matched There are N coral reefs in the area, which generate a total of 5×N pairs of points with the same name.

[0099] (2) Matching Coral Reefs with Coral Reefs. For two images on the same plane, the reference image and the image to be matched, they have homography in space. The geometric transformation model for the same point pairs of coral reefs is the projection transformation model. Therefore, the homography matrix H is chosen as the geometric transformation model for the solution. The formula is as follows:

[0100]

[0101] The Least Median of Squares (LMedS) method is commonly used for fitting and selecting corresponding point pairs in image matching. Specifically, it employs the Monte Carlo method, selecting four pairs of corresponding points each time, solving for the homography matrix parameters, and calculating the median of the squared residuals after projection transformation of all corresponding point pairs as the median error. Iteratively, it selects the subset of corresponding point pairs with the smallest median error and their corresponding homography matrix H. At this point, the homography matrix H obtained through iteration distinguishes corresponding point pairs into interior and exterior point pairs.

[0102] To minimize the impact of coral reef contour extraction accuracy and contour variations, the extraction of local invariant feature points of the coral reef is incorporated to correct the geometric transformation model.

[0103] First, for locally invariant feature points in reef pairs (the locally invariant features extracted in step 3), Lowe's suggested ratio of 0.8 is used as a threshold for the ratio of nearest neighbor to second nearest neighbor for filtering. Figure 5 The gray line represents the discarded pairs of corresponding points, and the green line represents the retained pairs of corresponding points. If the local feature point matching pairs after filtering meet the condition of no less than 4 sets, the geometric transformation model is further corrected. Then, the local feature point pairs that meet the ratio threshold are merged with the corresponding coral reef geometric contour point pairs solved in the previous step. The corrected homography matrix H (corrected geometric transformation model) is calculated iteratively, and then the corrected geometric transformation model is used for image matching.

[0104] The calculated geometric transformation model has two limitations. First, it uses the geometric features of the coral reef outline under the premise that the outline remains stable and unchanged. If the extraction of the coral reef outline is affected by factors such as clouds in the image, the accuracy of the calculated geometric transformation model cannot meet the pixel-level accuracy requirements of subsequent change monitoring. Second, medium-resolution remote sensing images have a large swath width, and a single geometric transformation model may not be suitable for matching all coral reefs within the image. Therefore, this invention uses a grayscale matching method for precise matching of local coral reefs and solves the problems of computational complexity and large influence from input image rotation or scale changes in grayscale matching algorithms.

[0105] To address the issue of high computational cost in grayscale matching, which stems from the fact that grayscale matching involves a global search across the entire image, we have solved the problem of a large search area for grayscale matching by performing coarse matching of reef pairs with the same name and correcting the calculation using the geometric transformation model. This has significantly reduced the computational cost of grayscale matching.

[0106] For problems involving image rotation or scale changes, the parameters are reduced using a modified homography matrix H (a modified geometric transformation model) for solution. The parameters in the homography matrix H are... , , To satisfy the homogeneity parameter of the matrix, the normalized result is reduced to a 2×3 affine transformation matrix F, which serves as the initial geometric transformation model for grayscale matching. The affine transformation matrix F is shown below:

[0107]

[0108] in, and Here, Δx and Δy are the scale parameters in the principal and secondary directions, respectively; α is the rotation parameter; and Δx and Δy are the translation parameters. For example... Figure 6 As shown, the i-th coral reef reference image is... The coral reef area is used as the grayscale matching window, corresponding to the yellow rectangle in the image. After affine transformation, the initial matching window position of the corresponding coral reef pair in the image to be matched is obtained, i.e. Figure 6 The blue rectangle in the middle.

[0109] The core idea of ​​grayscale matching is to compare the grayscale differences between the reference image and the image to be matched after geometric and radiometric transformations within the matching window.

[0110]

[0111] in, It matches the range of the window. It is the reference image within the matching window. It is a radiation transformation model. It is a geometric transformation model. It is the image to be matched after geometric transformation. This is the grayscale difference value. It is assumed that all images have undergone atmospheric correction. The initial values ​​are chosen from the modified geometric transformation model. Therefore, the affine transformation matrix F can be transformed into:

[0112]

[0113] In the formula, and These represent the x and y coordinates of the corresponding points in the reference image, respectively. and These represent the horizontal and vertical coordinates of the corresponding points in the image to be matched.

[0114] After determining the initial matching window position, the least squares method is used to calculate the geometric transformation (geometric transformation model) between the reference image and the image to be matched within the matching window. ) and radiation transformation (radiation transformation model) The grayscale difference value is calculated and the image to be matched within the window is corrected (by translation). During the correction process, bilinear interpolation resampling is used. Iterative calculation is performed until the grayscale difference value within the matching window is minimized, that is, F reaches its minimum value, thus completing the fine matching of the local coral reef. Figure 7 As shown, the blue rectangle represents the initial grayscale matching window, the red rectangle represents the matching window during iterative calculation, and the green rectangle represents the matching window after the iteration is complete.

[0115] To verify the accuracy and reliability of the method of the present invention, the following example will be used for further explanation. This example takes a group of reefs in an archipelago in a certain sea area as an example, using Sentinel-2 MSI imagery as a reference, and matching it with Landsat-4 / 5 TM, Landsat-8 OLI, GF-1 / 6 WFV, GF-4 PMI, and HJ-1A / B CCD remote sensing images respectively. We selected and compared our algorithm with other stable, advanced, or specialized feature description and matching algorithms suitable for specific texture images, including SIFT, Oriented FAST and Rotated BRIEF (ORB), BEBLID feature descriptors and Random Sampling Consensus (RANSAC), Grid-based Motion Statistics (GMS), and Adaptive Locally-Affine Matching (AdaLAM). The comparison was conducted primarily from qualitative and quantitative perspectives. Qualitative analysis examined the reasonableness of the number and distribution of matching pairs. Quantitative evaluation employed three criteria: matching pair accuracy, NMI, and RMSE. RMSE was calculated by manually collecting evaluation points (primarily clearly identifiable typical landforms and boundaries in the images, such as point reefs, lagoon slopes, and reef flat boundaries) in each image involved in the matching process.

[0116] Qualitative analysis results as follows Figure 8 , Figure 9 , Figure 10 and Figure 11As shown in the results, the proposed multi-feature hierarchical matching strategy for coral reefs achieved correct matching and uniform distribution of matching points in tests conducted on images from different regions and sensors. In contrast, other comparison methods suffer from geometric distortion after image transformation due to uneven feature point distribution, or almost all matching pairs are incorrect due to interference from fine noise such as clouds and solar flares in the images. Furthermore, the lack of clear texture features and complex noise can lead to the inability to obtain high-confidence matching pairs, resulting in the problem of not finding matching pairs at all. This invention uses coral reefs as the matching primitive and relies on a similarity metric that fuses multiple features to effectively avoid the above problems.

[0117] The quantitative matching RMSE results for images from different methods and sensors are shown in the table below:

[0118] RMSE results of different matching strategies for multi-satellite remote sensing images of Zhenghe Reefs (unit: pixels)

[0119]

[0120] Note: ↑ indicates an increase in RMSE, and ↓ indicates a decrease in RMSE.

[0121] As can be seen, the multi-feature hierarchical matching strategy for coral reefs proposed in this invention can reduce the RMSE to within 1.5 pixels. In contrast, other comparison methods exhibit higher registration errors due to their lower correct matching rates, resulting in increased RMSE ("↑"), distortion ("∞"), or failure to find a matching pair ("-").

[0122] In terms of matching accuracy ( Figure 12 This invention's strategy achieves a higher matching accuracy. For example, in the Zhenghe Reefs GF-6 image test, the accuracy of this invention is 94.29%, compared to 92.11% for BEBLID+AdaLAM and only 40.29% for BEBLID+RANSAC.

[0123] Regarding NMI ( Figure 13 The strategy of this invention improves the NMI of the transformed image by up to 0.17 compared to that before matching.

[0124] The multi-satellite remote sensing image matching method constrained by coral reef features of this invention is not limited to the specific technical solutions described in the above embodiments. All technical solutions formed by equivalent substitutions are within the scope of protection claimed by this invention.

Claims

1. A method for matching multi-satellite remote sensing images constrained by coral reef features, comprising the following steps: Step 1: Candidate coral reef analysis – Generate a buffer based on the coral reef reference range in the Sentinel-2 image; Overlay the image to be matched with the buffer, crop the image to be matched to obtain a set of images to be processed consisting of the image blocks to be matched, and select the corresponding set of reference coral reef ranges and reference coral reef images in the Sentinel-2 image. Step 2: Single-phase coral reef extent extraction – First, median filtering is used to denoise the image patches to be matched. Then, the Normalized Difference Red Edge Index (NDRI), Normalized Difference Water Index (NDWI), and Modified Normalized Difference Water Index (MNDWI) are calculated for the image patches to be matched. Next, the fuzzy C-cluster mean algorithm is used to divide the results of NDRI, NDWI, and MNDWI into foreground coral reef and background deep sea, and the foreground results of the three indices are merged. Then, morphological filtering and maximum connected component analysis are used to extract and optimize the coral reef extent. Finally, area filtering is performed using the minimum exposed reef area of ​​the reference coral reef extent set as a threshold to remove incorrectly extracted portions. Following the above process, the coral reef extents in the image patch set to be processed are extracted as the coral reef extent set to be matched. The result of cropping the image patch set to be processed from the coral reef extent set to be matched is used as the coral reef image set to be matched. Step 3, Coral Reef Multi-Feature Extraction and Description—Using the coral reef range set and the coral reef image set to be matched as matching primitives, describe their global and local features. The global features include geometric features, spectral features, and texture features, and the local features include local invariant features. Step 4, Layered Matching Strategy – First, calculate the geometric, spectral, and textural similarities between the reef range set and the reef image set to be matched, and perform coarse matching of reefs with the same name based on the multi-feature similarity metric. Then, construct an initial geometric transformation model using the geometric features of the reefs with the same name, correct the initial geometric transformation model using selected local invariant feature points, and use the corrected geometric transformation model for image matching. Finally, correct the image to be matched by minimizing the grayscale difference between the reference image and the image to be matched after geometric and radiometric transformations within the matching window, thus completing the fine matching of local reefs.

2. The multi-satellite remote sensing image matching method constrained by coral reef features according to claim 1, characterized in that: In step 2, the calculation formulas for the Normalized Difference Red Edge Index (NDRI), the Normalized Difference Water Body Index (NDWI), and the Modified Normalized Difference Water Body Index (MNDWI) are as follows: The surface reflectance is in the near-infrared band. The surface reflectance is in the red-edge band. The surface reflectance is in the green light band. This represents the surface reflectance in the shortwave infrared band.

3. The multi-satellite remote sensing image matching method constrained by coral reef features according to claim 1, characterized in that: In step 2, morphological filtering includes erosion and dilation operations to construct opening and closing operations, as shown in the following formulas: in, Input image; For structural elements; For erosion operations; For expansion operation; This refers to the position of the structuring element after translation; Indicates an inclusion relationship; For the result set; for The reflection; The intersection of sets; The intersection is not empty; , They represent opening and closing operations, respectively.

4. The method for matching multi-satellite remote sensing images constrained by coral reef features according to claim 1, characterized in that: In step 3, the geometric features of the spatial extent of the coral reef are described using Hu geometric invariant moments for the reference coral reef image set and the coral reef image set to be matched, respectively, to obtain the reference coral reef geometric feature set and the coral reef geometric feature set to be matched.

5. The multi-satellite remote sensing image matching method constrained by coral reef features according to claim 1, characterized in that: In step 3, grayscale histograms are calculated for the blue, green, red, and near-infrared bands for both the reference and target coral reef image sets to obtain the spectral feature sets of the reference and target coral reefs. The formula for calculating the grayscale histograms is as follows: in, It is the number of grayscale colors after quantization. It is the number of pixels with value k in the grayscale image. It represents the total number of pixels in a grayscale image.

6. The multi-satellite remote sensing image matching method constrained by coral reef features according to claim 1, characterized in that: In step 3, the GIST feature descriptor is used to extract the texture features of the reference coral reef image set and the coral reef image set to be matched, so as to obtain the texture feature set of the reference coral reef and the texture feature set of the coral reef to be matched. When extracting texture features, GIST features are obtained by performing Gabor filtering, block segmentation and mean statistics on the blue light band grayscale image.

7. The method for matching multi-satellite remote sensing images constrained by coral reef features according to claim 1, characterized in that: In step 3, the SIFT local invariant feature descriptor is used to extract local invariant features from the reference coral reef image set and the coral reef image set to be matched. In step 4, local invariant feature point pairs are matched based on the same coral reef.

8. The method for matching multi-satellite remote sensing images constrained by coral reef features according to claim 1, characterized in that: In step 4, the coarse matching of coral reefs with the same name uses an adjusted cosine similarity algorithm to measure the similarity between features, and the formula is as follows: in, Representation of features With features Similarity; and Features and The mean, Represents the first of the range sets of coral reefs to be matched. One characteristic, This represents the feature mean of the set of coral reef ranges to be matched. This indicates the first image in the coral reef image set to be matched. One characteristic, This represents the feature mean of the coral reef image set to be matched. Indicates the feature dimension.

9. The method for matching multi-satellite remote sensing images constrained by coral reef features according to claim 1, characterized in that: In step 4, the calculation formula for the multi-feature similarity measurement criterion is as follows: in, Represents the i-th reference coral reef image Matching the j-th coral reef image Multi-feature similarity between them; It is a symbolic function; , , These represent geometric feature similarity, spectral feature similarity, and texture feature similarity, respectively. , These represent the weights of spectral features and texture features, respectively. and These are the i-th reference coral reef images. and the j-th coral reef image to be matched The maximum frequency of the histogram in the blue light band.

10. The method for matching multi-satellite remote sensing images constrained by coral reef features according to claim 1, characterized in that: In step 4, the geometric transformation model is a homography matrix. The homography matrix is ​​solved using the minimum median method. And screening coral reef corresponding point pairs; in the process of local coral reef fine matching, the geometric transformation model after correction by local invariant feature point pairs is reduced to an affine transformation matrix. The grayscale difference between the reference image and the image to be matched within the matching window is calculated after geometric and radiometric transformations. The image to be matched within the matching window is corrected by translation. The calculation is iterated until the grayscale difference within the matching window is minimized, thus completing the fine matching of the local coral reef.

Images