A high-precision multi-modal remote sensing image automatic matching method and system

Abstract

The application provides a high-precision multi-modal remote sensing image automatic matching method and system, the method comprising: acquiring a reference image and an input image and making a multi-modal remote sensing image training and testing dataset; respectively extracting a pixel-by-pixel feature expression graph of the reference image and the input image; enhancing the structural feature description of the pixel-by-pixel feature expression graph by constructing a multi-branch global attention module, thereby obtaining an attention-enhanced structural feature description graph; constructing a loss function of joint multi-cropping template matching based on a triple loss using the overall and local information of the image; performing network learning training on the multi-modal remote sensing image training and testing dataset, thereby obtaining a multi-modal remote sensing image automatic matching model; and obtaining the position coordinates of the input image on the reference image through the multi-modal remote sensing image automatic matching model. The application can make up for the insufficient structural feature expression of multi-modal images in the traditional matching method, and effectively improve the matching precision of multi-modal remote sensing images.

Description

Technical Field

[0001] This invention relates to the field of satellite image processing technology, and more specifically, to a high-precision automatic matching method and system for multimodal remote sensing images. Background Technology

[0002] Image matching refers to the process of identifying corresponding points between two or more images. This step is a crucial prerequisite for many remote sensing image applications, widely used in information fusion, image stitching, 3D reconstruction, target recognition, ground feature interpretation, and defense equipment. Today, remote sensing platforms are equipped with precision orbital sensors, enabling them to acquire remote sensing images with high geospatial accuracy. Large global geometric transformations (such as rotation and scale changes) can be eliminated through preprocessing. Therefore, most image matching problems can be focused on finite translation estimation. However, due to the influence of different imaging mechanisms, significant nonlinear radiometric differences exist between multimodal remote sensing images (including visible light, infrared, lidar, and congruent aperture radar), making automatic matching of corresponding points still very challenging. To address this issue, further research and improvement of matching algorithms are needed.

[0003] Currently, multimodal remote sensing image matching methods can be mainly divided into two categories: manual computer vision-based methods and deep learning-based methods. Manually crafted image matching methods are typically categorized into feature-based and region-based methods. Feature-based methods primarily detect prominent features in image pairs, commonly including point, line, and area features. Recently, locally invariant features such as Scale Invariant Feature Transform (SIFT), Speed ​​Up Robust Feature (SURF), SAR-SIFT, and PCA-SIFT have also found some application in remote sensing image matching. However, these methods usually require extracting features with high repetition rates between images. For multimodal remote sensing images with significant radiometric differences, the repetition rate of feature extraction is often low, thus limiting their application in automatic multimodal remote sensing image matching. Region-based methods mainly employ template matching strategies, using a certain similarity measure as a criterion to identify corresponding points between images. In this process, the choice of similarity measure is crucial, directly affecting the subsequent matching accuracy. Commonly used similarity measures include the sum of squared gray-level differences, normalized correlation coefficient, and mutual information. However, these similarity measures rely on the similarity of gray levels between images for corresponding point identification. Since the gray level information varies significantly between multimodal remote sensing images, these measures are not well-suited for automatic matching of multimodal remote sensing images. Compared to gray level information, the structural and shape attributes of images exhibit higher similarity. Furthermore, related research utilizes methods such as Histogram of Oriented Phase Congruency (HOPC) and Channel Feature of Oriented Gradient (CFOG) to extract similar structural information from images, effectively resisting nonlinear radiometric differences. Nevertheless, considering the significant differences in complex and diverse multimodal images, the potential of manually crafted features to further improve matching performance is limited.

[0004] Deep learning-based methods directly learn end-to-end similarity metrics for image matching from data. This allows for more effective learning of homogeneous features from highly heterogeneous image pairs, improving image matching performance and making it the mainstream approach in heterogeneous image matching research. In the field of image matching, Siamese networks and Triplet networks have recently become mainstream architectures. Deep learning-based matching methods typically use deep networks to extract common features between images and then utilize similarity metrics between features to achieve high-precision image matching. Recently, deep learning methods based on multimodal image matching, such as DDFN and OSMNet, have outperformed manually constructed descriptors, but they have not shown an overwhelming advantage. Learning-based descriptors are data-driven, but currently, there are not enough training samples for multimodal image registration both domestically and internationally to learn high-quality and widely applicable feature description models. Therefore, how to further extract refined and robust feature descriptions from multimodal images is key to improving the accuracy of existing matching methods. Summary of the Invention

[0005] The present invention aims to provide a high-precision automatic matching method and system for multimodal remote sensing images, so as to extract fine and robust feature descriptions from multimodal images, thereby improving matching accuracy.

[0006] This invention provides a high-precision automatic matching method for multimodal remote sensing images, comprising the following steps:

[0007] Step A: Acquire reference and input images and create a multimodal remote sensing image training and testing dataset;

[0008] Step B: Using a structural feature description strategy, extract pixel-by-pixel feature representation maps of the reference image and the input image respectively;

[0009] Step C: Enhance the structural feature description of the pixel-wise feature representation map by constructing a multi-branch global attention module, thereby obtaining an attention-enhanced structural feature description map;

[0010] Step D: Based on the structural feature description map, construct a loss function for joint multi-cropping template matching using triplet loss based on the overall and local information of the image;

[0011] Step E: Using the loss function of joint multi-cropping template matching, the network is trained on the multimodal remote sensing image training and testing datasets to obtain the multimodal remote sensing image automatic matching model.

[0012] Step F: Obtain the position coordinates of the input image on the reference image through the multimodal remote sensing image automatic matching model.

[0013] Furthermore, in step A, if the reference image and the input image have different resolutions, then the reference image and the input image need to be resampled to the same resolution.

[0014] Furthermore, step B includes the following sub-steps:

[0015] Step B1: For the reference image and the input image, calculate the pixel-by-pixel structural feature descriptor, including the phase consistency orientation histogram (HOPC) and the gradient orientation feature channel (CFOG).

[0016] Step B2: Arrange the pixel-by-pixel structural feature descriptors in the Z direction to form a three-dimensional pixel-by-pixel feature representation map.

[0017] Furthermore, step C includes the following sub-steps:

[0018] Step C1: Use convolutional kernels of different sizes to copy and expand the input pixel-by-pixel feature representation map X into three groups. F 1. F 2. F 3, F 1= F 2= F 3=X to obtain different receptive fields, and the three expanded sets of pixel-wise feature representation maps are merged into a feature map. ,

[0019] Step C2, based on feature maps Extract channel attention maps separately M C ( F ) and spatial attention map M S ( F );

[0020] Step C3: Channel attention map M C ( F ) and spatial attention map M S ( F Expand to the same size and sum, and use sigmoid The function is recalibrated to obtain the final attention map. M ( F );

[0021] Step C4: Transfer the attention map M ( F ) and feature map Element-wise multiplication and remapping yield an attention-enhanced structural feature description map.

[0022] Furthermore, step C includes the following sub-steps:

[0023] Step C1: Use convolutional kernels of different sizes to copy and expand the input pixel-by-pixel feature representation map X into three groups. F 1. F 2. F 3, F 1= F 2= F 3=X to obtain different receptive fields, and the three expanded sets of pixel-wise feature representation maps are merged into a feature map. The calculation formula is:

[0024]

[0025]

[0026] In the formula, i =1,2,3;

[0027] Step C2, based on feature maps Extract channel attention maps separately M C ( F ) and spatial attention map M S ( F ):

[0028] (1) Use the weight module of FCAnet to extract the channel attention map. M C ( F The calculation formula is:

[0029]

[0030] In the formula, These are the attention weights of the corresponding branches. FCAnet is used to learn the weights of the feature channels, which are quantized by multiple spectral components.

[0031]

[0032] In the formula, It is the concat operator, which represents connecting along the dimension of the feature map;

[0033] (2) Spatial attention map is extracted using the spatial attention branch of BAM. M S ( F The calculation formula is:

[0034]

[0035] In the formula, f j Indicates the firstj A convolution operation for a convolution filter, where the superscript indicates the size of the convolution filter; BN This indicates a batch normalization operation;

[0036] Step C3: Channel attention map M C ( F ) and spatial attention map M S ( F Expand to the same size and sum, and use sigmoid The function is recalibrated to obtain the final attention map. M ( F The calculation formula is:

[0037]

[0038] In the formula, sigmoid() represents sigmoid function;

[0039] Step C4: Transfer the attention map M ( F ) and feature map Element-wise multiplication and remapping yield the attention-enhanced structural feature description map, calculated using the following formula:

[0040]

[0041] In the formula, f This indicates the convolution operation of the convolution filter; the superscript indicates the size of the convolution filter. This represents the element-wise multiplication of matrices.

[0042] Furthermore, step D includes the following sub-steps:

[0043] Step D1: After obtaining the attention-enhanced structural feature description maps of a pair of input images and reference images, randomly crop the attention-enhanced structural feature description map of the input image to obtain N sub-template features;

[0044] Step D2: Calculate the similarity scores between the features of each of the n sub-templates and the corresponding search windows on the attention-enhanced structural feature description map of the image;

[0045] Step D3: Sum all similarity scores according to area weights to obtain the final joint multi-cropping similarity score map;

[0046] Step D4: Using the similarity score map of joint multi-cropping, construct a loss function for joint multi-cropping template matching based on triplet loss.

[0047] Furthermore, in step D2, the similarity score is calculated using either cross-correlation (CC) or sum of squared differences (SSD); the calculation formula is as follows:

[0048]

[0049]

[0050] In the formula, D s and D o These represent the two structural feature representations involved in the similarity score calculation. T Representative sub-template features, The sliding position representing the template sub-template feature. v This represents the sliding vector.

[0051] Furthermore, the formula for summing all similarity scores according to area weight in step D3 is as follows:

[0052]

[0053] In the formula, MS The similarity score plot represents the final joint multi-cropping. N The number of sub-template features, S Represents the similarity score; α t Representing the t The area weight coefficient of each sub-template feature.

[0054] Furthermore, the loss function described in step D4 is expressed as:

[0055]

[0056] In the formula, p and n These represent positive and negative samples, respectively. a These are the anchor samples corresponding to the labels. m It is a constant greater than 0.

[0057] The present invention also provides a high-precision multimodal remote sensing image automatic matching system, comprising:

[0058] The acquisition unit is used to acquire the reference image and the input image;

[0059] The preprocessing unit is used to create training and testing datasets for multimodal remote sensing images. If the reference image and the input image have the same resolution, they proceed to the next unit. If the resolutions are different, the reference image and the input image are resampled to the same resolution.

[0060] The feature map description unit is used to extract pixel-wise feature representation maps of the reference image and the input image;

[0061] The feature map enhancement unit is used to construct a multi-branch global attention module to enhance the structural feature description of the pixel-wise feature representation map, thereby obtaining an attention-enhanced structural feature description map.

[0062] The metric learning loss unit is used to construct a joint multi-cropping template matching loss function based on triplet loss, using both global and local image information on the basis of the structural feature description map.

[0063] The training unit is used to train the network on the multimodal remote sensing image training and testing datasets using the loss function of joint multi-cropping template matching to obtain the multimodal remote sensing image automatic matching model.

[0064] The prediction unit obtains the position coordinates of the input image on the reference image through a multimodal remote sensing image automatic matching model.

[0065] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0066] 1. This invention proposes a high-precision automatic matching framework for multimodal remote sensing images. By constructing a matching framework, automatic matching of multimodal remote sensing images can be performed by combining attention-enhanced structural features. The automatic matching framework first extracts structural feature descriptors for each pixel of the image, forming a pixel-by-pixel feature representation map. Next, a multi-branch global attention module is constructed to enhance the structural feature representation map in both channel and spatial dimensions, providing a more refined and robust representation of features compared to traditional structural feature descriptors. Then, a loss function for joint multi-cropping template matching is constructed based on triplet loss, utilizing both global and local image information. Finally, the network is trained on a prepared dataset to obtain an automatic matching model for multimodal remote sensing images. This model is then used to identify the location of the input image within a reference image. The technical solution of this invention can overcome the shortcomings of traditional matching methods in terms of the insufficient refinement of multimodal image structural feature representation, effectively improving the matching accuracy of multimodal remote sensing images such as visible light, infrared, lidar, synthetic aperture radar, and maps.

[0067] 2. The technical solution proposed in this invention is a general technical framework that can be combined with various structural feature descriptors (not limited to HOPC, CFOG, etc.) for image matching. Attached Figure Description

[0068] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0069] Figure 1 This is a flowchart of a high-precision multimodal remote sensing image automatic matching method in an embodiment of the present invention.

[0070] Figure 2 This is a flowchart illustrating the structural features of attention enhancement obtained in an embodiment of the present invention.

[0071] Figure 3 This is a flowchart illustrating the construction of a loss function for joint multi-pruned template matching in an embodiment of the present invention. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0073] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0074] Example

[0075] like Figure 1 As shown in the figure, this embodiment proposes a high-precision automatic matching method for multimodal remote sensing images, including the following steps:

[0076] Step A: Acquire reference images and input images and create a multimodal remote sensing image training and testing dataset; In this embodiment, the image types of the reference images and input images can be any two of the following: visible light images, infrared images, lidar images, synthetic aperture radar images, and maps. In other embodiments, other types of images can also be used to create the dataset, and this invention does not limit this.

[0077] In particular, if the reference image and the input image have different resolutions, the reference image and the input image need to be resampled to the same resolution.

[0078] Step B: Employ a structural feature description strategy to extract pixel-wise feature representation maps for both the reference image and the input image; specifically, this includes the following sub-steps:

[0079] Step B1: For the reference image and the input image, calculate the pixel-by-pixel structural feature descriptors, such as the Histogram of Oriented Phase Congruency (HOPC) and the Channel Feature of Oriented Gradient (CFOG).

[0080] Step B2: Arrange the pixel-by-pixel structural feature descriptors in the Z direction to form a three-dimensional pixel-by-pixel feature representation map.

[0081] Step C: Enhance the structural feature description of the pixel-by-pixel feature representation map by constructing a multi-branch global attention module, thereby obtaining an attention-enhanced structural feature description map; such as Figure 2 As shown, the specific steps include the following:

[0082] Step C1: Use convolutional kernels of different sizes to copy and expand the input pixel-by-pixel feature representation map X into three groups. F 1. F 2. F 3, F 1= F 2= F 3=X to obtain different receptive fields, and the three expanded sets of pixel-wise feature representation maps are merged into a feature map. The calculation formula is:

[0083]

[0084]

[0085] In the formula, i =1,2,3 It is a feature map that contains richer multi-scale structural information.

[0086] Step C2, based on feature maps Extract channel attention maps separately M C ( F ) and spatial attention map M S ( F In this embodiment:

[0087] (1) Use the weight module of FCAnet (Frequency Channel Attention Networks) to extract the channel attention map. M C ( F The calculation formula is:

[0088]

[0089] In the formula, These are the attention weights for the corresponding branches. The recently proposed FCAnet is used to learn the weights of the feature channels, which are quantized by multiple spectral components.

[0090]

[0091] In the formula, It is the concat operator, which represents connecting along the dimension of the feature map;

[0092] (2) Use the spatial attention branch of BAM (Bottleneck Attention Module) to extract the spatial attention map. M S ( F The calculation formula is:

[0093]

[0094] In the formula, f j Indicates the first j A convolution operation for a convolution filter, where the superscript indicates the size of the convolution filter; BN This indicates a batch normalization operation.

[0095] Step C3: Channel attention map M C ( F ) and spatial attention map M S ( F Expand to the same size and sum, and use sigmoid The function is recalibrated to obtain the final attention map. M ( F The calculation formula is:

[0096]

[0097] In the formula, sigmoid() represents sigmoid function.

[0098] Step C4: Transfer the attention map M ( F ) and feature map Element-wise multiplication and remapping yield the attention-enhanced structural feature description map, calculated using the following formula:

[0099]

[0100] In the formula, f This indicates the convolution operation of the convolution filter; the superscript indicates the size of the convolution filter. This represents the element-wise multiplication of matrices.

[0101] Step D: Based on the structural feature description map, construct a joint multi-cropping template matching loss function using triplet loss based on overall and local image information; such as... Figure 3 As shown, the specific steps include the following:

[0102] Step D1: After obtaining the attention-enhanced structural feature description maps of a pair of input images and reference images, randomly crop the attention-enhanced structural feature description map of the input image to obtain N sub-template features;

[0103] Step D2: Calculate the similarity scores between the features of each of the n sub-templates and the corresponding search windows on the attention-enhanced structural feature description map of the image. In this embodiment, the similarity score calculation method can be cross-correlation (CC) or sum of squared differences (SSD). The calculation formula is as follows:

[0104]

[0105]

[0106] In the formula, D s and D o These represent the two structural feature representations involved in the similarity score calculation. T Representative sub-template features, The sliding position representing the template sub-template feature. v This represents the sliding vector.

[0107] Step D3: Accumulate all similarity scores according to area weights to obtain the final joint multi-cropping similarity score map. The calculation formula is as follows:

[0108]

[0109] In the formula, MS The similarity score plot represents the final joint multi-cropping. N The number of sub-template features, S Represents the similarity score; α t Representing the tThe area weight coefficient of each sub-template feature.

[0110] Step D4: Using the similarity score map of joint multi-cropping, construct a loss function for joint multi-cropping template matching based on triplet loss. The loss function is expressed as:

[0111]

[0112] In the formula, p and n These represent positive and negative samples, respectively. a These are the anchor samples corresponding to the labels. m It is a constant greater than 0.

[0113] Step E: Using the loss function of joint multi-cropping template matching, the network is trained on the multimodal remote sensing image training and testing datasets to obtain the multimodal remote sensing image automatic matching model; the network model and training process can be selected and set as needed, and will not be described in detail here.

[0114] Step F: Obtain the position coordinates of the input image on the reference image through the multimodal remote sensing image automatic matching model.

[0115] In another embodiment, a high-precision multimodal remote sensing image automatic matching system is also provided, comprising:

[0116] The acquisition unit is used to acquire the reference image and the input image;

[0117] The preprocessing unit is used to create training and testing datasets for multimodal remote sensing images. If the reference image and the input image have the same resolution, they proceed to the next unit. If the resolutions are different, the reference image and the input image are resampled to the same resolution.

[0118] The feature map description unit is used to extract pixel-wise feature representation maps of the reference image and the input image;

[0119] The feature map enhancement unit is used to construct a multi-branch global attention module to enhance the structural feature description of the pixel-wise feature representation map, thereby obtaining an attention-enhanced structural feature description map.

[0120] The metric learning loss unit is used to construct a joint multi-cropping template matching loss function based on triplet loss, using both global and local image information on the basis of the structural feature description map.

[0121] The training unit is used to train the network on the multimodal remote sensing image training and testing datasets using the loss function of joint multi-cropping template matching to obtain the multimodal remote sensing image automatic matching model.

[0122] The prediction unit obtains the position coordinates of the input image on the reference image through a multimodal remote sensing image automatic matching model.

[0123] The above describes the specific embodiments of the present invention. The present invention proposes a high-precision automatic matching framework for multimodal remote sensing images. By constructing this matching framework, automatic matching of multimodal remote sensing images can be performed by combining attention-enhanced structural features. This automatic matching framework first extracts structural feature descriptors for each pixel of the image, forming a pixel-by-pixel feature representation map. Next, a multi-branch global attention module is constructed to enhance the structural feature representation map in both channel and spatial dimensions, resulting in a more refined and robust representation of features compared to traditional structural feature descriptors. Then, a loss function for joint multi-cropping template matching is constructed based on triplet loss, utilizing both global and local image information. Finally, the network is trained on a prepared dataset to obtain an automatic matching model for multimodal remote sensing images. This model is then used to identify the position of the input image within the reference image. The technical solution of the present invention can overcome the shortcomings of traditional matching methods in terms of the insufficient refinement of multimodal image structural feature representation. Extensive experimental verification shows that the overall matching accuracy is better than one pixel, effectively improving the matching performance of multimodal remote sensing images such as visible light, infrared, lidar, synthetic aperture radar, and maps.

[0124] Furthermore, the technical solution proposed in this invention is a general technical framework that can be combined with various structural feature descriptors (not limited to HOPC, CFOG, etc.) for image matching.

[0125] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A high-precision automatic matching method for multimodal remote sensing images, characterized in that, Includes the following steps: Step A: Acquire reference and input images and create a multimodal remote sensing image training and testing dataset; Step B: Using a structural feature description strategy, extract pixel-by-pixel feature representation maps of the reference image and the input image respectively; Step C: Enhance the structural feature description of the pixel-wise feature representation map by constructing a multi-branch global attention module, thereby obtaining an attention-enhanced structural feature description map; Step D: Based on the structural feature description map, construct a loss function for joint multi-cropping template matching using triplet loss based on the overall and local information of the image; Step E: Using the loss function of joint multi-cropping template matching, the network is trained on the multimodal remote sensing image training and testing datasets to obtain the multimodal remote sensing image automatic matching model. Step F: Obtain the position coordinates of the input image on the reference image through the multimodal remote sensing image automatic matching model; Step C includes the following sub-steps: Step C1: Use convolutional kernels of different sizes to copy and expand the input pixel-by-pixel feature representation map X into three groups. F 1. F 2. F 3, F 1= F 2= F 3=X to obtain different receptive fields, and the three expanded sets of pixel-wise feature representation maps are merged into a feature map. , Step C2, based on feature maps Extract channel attention maps separately M C ( F ) and spatial attention map M S ( F ); step C3. Channel Attention Map M C ( F ) and spatial attention map M S ( F Expand to the same size and sum, and use sigmoid The function is recalibrated to obtain the final attention map. M ( F ); Step C4: Transfer the attention map M ( F ) and feature map Element-wise multiplication and remapping yield an attention-enhanced structural feature description map; Step C includes the following sub-steps: Step C1: Use convolutional kernels of different sizes to copy and expand the input pixel-by-pixel feature representation map X into three groups. F 1. F 2. F 3, F 1= F 2= F 3=X to obtain different receptive fields, and the three expanded sets of pixel-wise feature representation maps are merged into a feature map. The calculation formula is: In the formula, i =1,2,3; Step C2, based on feature maps Extract channel attention maps separately M C ( F ) and spatial attention map M S ( F ): (1) Use the weight module of FCAnet to extract the channel attention map. M C ( F The calculation formula is: In the formula, These are the attention weights of the corresponding branches. FCAnet is used to learn the weights of the feature channels, which are quantized by multiple spectral components. In the formula, It is the concat operator, which represents connecting along the dimension of the feature map; (2) Spatial attention map is extracted using the spatial attention branch of BAM. M S ( F The calculation formula is: In the formula, f j Indicates the first j A convolution operation for a convolution filter, where the superscript indicates the size of the convolution filter; BN This indicates a batch normalization operation; Step C3: Channel attention map M C ( F ) and spatial attention map M S ( F Expand to the same size and sum, and use sigmoid The function is recalibrated to obtain the final attention map. M ( F The calculation formula is: In the formula, sigmoid() represents sigmoid function; Step C4: Transfer the attention map M ( F ) and feature map Element-wise multiplication and remapping yield the attention-enhanced structural feature description map, calculated using the following formula: In the formula, f This indicates the convolution operation of the convolution filter; the superscript indicates the size of the convolution filter. This represents element-wise multiplication of matrices; Step D includes the following sub-steps: Step D1: After obtaining the attention-enhanced structural feature description maps of a pair of input images and reference images, randomly crop the attention-enhanced structural feature description map of the input image to obtain N sub-template features; Step D2: Calculate the similarity scores between the features of each of the n sub-templates and the corresponding search windows on the attention-enhanced structural feature description map of the image; Step D3: Sum all similarity scores according to area weights to obtain the final joint multi-cropping similarity score map; Step D4: Using the similarity score map of joint multi-cropping, construct the loss function of joint multi-cropping template matching based on triplet loss; In step D2, the similarity score is calculated using either cross-correlation (CC) or sum of squared differences (SSD); the calculation formula is as follows: In the formula, D s and D o These represent the two structural feature representations involved in the similarity score calculation. T Representative sub-template features, The sliding position representing the template sub-template feature. v Represents the sliding vector; In step D3, the formula for summing all similarity scores based on area weights is as follows: In the formula, MS The similarity score plot represents the final joint multi-cropping. N The number of sub-template features, S Represents the similarity score; α t Representing the t Area weighting coefficients for individual template features; The loss function described in step D4 is expressed as: In the formula, p and n These represent positive and negative samples, respectively. a These are the anchor samples corresponding to the labels. m It is a constant greater than 0.

2. The high-precision multimodal remote sensing image automatic matching method according to claim 1, characterized in that, In step A, if the reference image and the input image have different resolutions, then the reference image and the input image need to be resampled to the same resolution.

3. The high-precision multimodal remote sensing image automatic matching method according to claim 1, characterized in that, Step B includes the following sub-steps: Step B1: For the reference image and the input image, calculate the pixel-wise structural feature descriptor, including the phase consistency orientation histogram and the gradient orientation feature channel, respectively. Step B2: Arrange the pixel-by-pixel structural feature descriptors in the Z direction to form a three-dimensional pixel-by-pixel feature representation map.

4. A high-precision multimodal remote sensing image automatic matching system, used to execute the high-precision multimodal remote sensing image automatic matching method as described in any one of claims 1-3, characterized in that, include: The acquisition unit is used to acquire the reference image and the input image; The preprocessing unit is used to create training and testing datasets for multimodal remote sensing images. If the reference image and the input image have the same resolution, they proceed to the next unit. If the resolutions are different, the reference image and the input image are resampled to the same resolution. The feature map description unit is used to extract pixel-wise feature representation maps of the reference image and the input image; The feature map enhancement unit is used to construct a multi-branch global attention module to enhance the structural feature description of the pixel-wise feature representation map, thereby obtaining an attention-enhanced structural feature description map. The metric learning loss unit is used to construct a joint multi-cropping template matching loss function based on triplet loss, using both global and local image information on the basis of the structural feature description map. The training unit is used to train the network on the multimodal remote sensing image training and testing datasets using the loss function of joint multi-cropping template matching to obtain the multimodal remote sensing image automatic matching model. The prediction unit obtains the position coordinates of the input image on the reference image through a multimodal remote sensing image automatic matching model.

Images