Methods, apparatus, equipment and media for constructing remote sensing image matching test datasets

Abstract

This application discloses a method, apparatus, device, and medium for constructing a remote sensing image matching test dataset. The method uses two remote sensing images of the same target geographic area taken at different times as image pairs to be matched, inputting each pair into a group of matching models. Each matching model in the group matches each image pair, obtaining the matching results of the same image pair in different matching models. Based on the matching results, a matching evaluation index is calculated for the same image pair, and the matching difficulty level of the corresponding image pair is determined according to the matching evaluation index. The matching difficulty level is used as a classification label to classify the image pairs in the image dataset, generating remote sensing image matching test datasets with different difficulty levels. This achieves efficient construction of remote sensing image matching test datasets and improves the classification accuracy of the test dataset.

Description

Technical Field

[0001] This application relates to the field of data processing and testing technology, and in particular to a method, apparatus, device and medium for constructing a remote sensing image matching test dataset. Background Technology

[0002] In applications such as aircraft navigation, target positioning, and flight path planning, the intelligent matching capability of airborne modules is a crucial underlying factor supporting system perception, decision-making, and control. The quality and coverage of the test dataset directly impact the effectiveness of this capability assessment. Currently, most test datasets are static, single-scene, or constructed from simulation platforms, making it difficult to cover complex situations in real flight environments such as lighting changes, weather interference, dynamic obstacles, and large-scale scene transitions. This leads to significant discrepancies between model evaluation results and actual application performance. To improve the accuracy of model training, it is first necessary to acquire a large number of remote sensing images from different scenes to fit the actual situation. Then, classification labels are generated, and the remote sensing images are automatically classified according to these labels. Currently, the labeling of remote sensing images often combines machine recognition and manual inspection. However, this method is inefficient, and errors inevitably occur during manual inspection, affecting the classification accuracy of remote sensing images and resulting in low accuracy. Summary of the Invention

[0003] This application provides a method, apparatus, device, and medium for constructing a remote sensing image matching test dataset, solving the problems of low efficiency and low accuracy in determining scene classification labels in the prior art. By forming matching image pairs from remote sensing images of the same geographical area taken at different time dimensions and inputting them into different matching models to generate matching results, a matching evaluation index is determined. Finally, the matching difficulty level is determined through the matching evaluation index to form a matching difficulty classification label. The entire process requires no manual intervention, effectively improving the classification efficiency and accuracy of matching difficulty levels. After determining the matching difficulty level of the matching image pairs, the remote sensing images in the remote sensing image matching test dataset are classified into texture regions to generate a remote sensing image matching test dataset with texture region types, so as to facilitate subsequent targeted algorithm testing and analysis for images with different matching difficulty levels.

[0004] In a first aspect, embodiments of this application provide a method for constructing a remote sensing image matching test dataset, including: Remote sensing images of the target geographic area taken at different times are acquired as an image dataset, and two remote sensing images belonging to the same target geographic area in the image dataset are taken as image pairs to be matched. The image pairs to be matched in the image dataset are input one by one into the matching model group. Each matching model in the matching model group matches each image pair to be matched, and obtains the matching results of the same image pair in different matching models. Based on the matching results, a matching evaluation index is calculated for the same pair of images to be matched, and the matching difficulty level of the corresponding pair of images to be matched is determined according to the matching evaluation index. The matching difficulty level is used as the matching difficulty classification label to classify the pairs of images to be matched in the image dataset, generating remote sensing image matching test datasets with different matching difficulty levels.

[0005] In conjunction with the first aspect, in one possible implementation, calculating the matching evaluation index for the same pair of images to be matched based on the matching result includes: The matching point pair ratio of the same image pair to be matched is calculated based on the matching results output by each matching model. If the matching point pair ratio of at least one matching model is not less than the preset ratio threshold, the corresponding image pair to be matched is added to the matchable test set as a matchable image pair. The number of identical matching image pairs in the matching test set is counted as the number of matching models, and the proportion of the number of matching models to the total number of matching models is calculated. The proportion of matching models is used as the first matching evaluation index. Calculate the average number of inliers for the same matchable image pair in the matchable test set, and use the average number of matching points as the second matching evaluation index. Obtain the ground homography matrix of the same image pair to be matched, and determine the target submatrix based on the ground homography matrix of the image pair to be matched; Calculate the rotation angle of the target sub-matrix and convert the rotation angle into an angle value, and obtain the degree of rotation between the same pair of images to be matched as a third matching evaluation index; The square root of the absolute value of the determinant of the target submatrix is ​​calculated as the scaling ratio, and the scaling ratio is determined as the fourth matching evaluation index. The matching evaluation index for the same image pair to be matched is constructed based on the first matching evaluation index, the second matching evaluation index, the third matching evaluation index, and the fourth matching evaluation index.

[0006] In conjunction with the first aspect, in one possible implementation, calculating the proportion of matching point pairs for the same pair of images to be matched based on the matching results includes: The RANSAC algorithm is used to calculate the geometric consistency error of the matching results of the same image pair to be matched from different matching models. Matching point pairs with a geometric consistency error less than a preset threshold are determined as inliers. By using the true homography matrix of the image pair to be matched, the interior points of one remote sensing image in the image pair to be matched are projected onto the coordinate system of the other remote sensing image to obtain the true projection points. Calculate the Euclidean distance between the interior point and the corresponding real projection point, and count the proportion of interior points whose Euclidean distance is within a preset pixel length to the total number of interior points as the matching point pair ratio of the image pairs to be matched.

[0007] In conjunction with the first aspect, in one possible implementation, calculating the rotation angle of the target sub-matrix and converting the rotation angle into an angle value to obtain the degree of rotation between the same pair of images to be matched includes: The target submatrix is ​​decomposed using singular value decomposition, and a pure rotation matrix is ​​constructed based on the decomposed matrix. The rotation angle of the pure rotation matrix is ​​calculated using the arctangent function, and the rotation angle is normalized to an angle value within the range of 0-180 degrees. The angle value represents the degree of rotation between the same matching image pair.

[0008] In conjunction with the first aspect, in one possible implementation, determining the matching difficulty level of the corresponding matching image pair based on the matching evaluation index includes: For the matchable image pairs in the matchable test set, determine whether the third and fourth matching evaluation indicators meet the first preset matchable indicator range. When the third matching evaluation index and the fourth matching evaluation index meet the first preset matchable index range, then based on the first matching evaluation index and the second matching evaluation index, the matchable image pairs corresponding to the matchable test set are evaluated for a first matching difficulty level to obtain the matching difficulty level of the corresponding matchable image pairs.

[0009] In conjunction with the first aspect, in one possible implementation, determining the matching difficulty level of the corresponding matching image pair based on the matching evaluation index further includes: For image pairs to be matched that do not belong to the matchable test set, determine whether the third and fourth matching evaluation indicators meet the range of the second preset matchable indicators. When the third matching evaluation index and the fourth matching evaluation index meet the second preset matchable index range, then based on the third matching evaluation index and the fourth matching evaluation index, the second matching difficulty level evaluation is performed on the image pairs to be matched that have not been included in the matchable test set, and the matching difficulty level of the corresponding image pairs to be matched is obtained.

[0010] In conjunction with the first aspect, in one possible implementation, the method for constructing a remote sensing image matching test dataset further includes: Scene features are extracted from remote sensing images in the remote sensing image matching test dataset using a semantic segmentation network, and the texture region type is determined based on the extracted scene features. The texture region type is used as the scene classification label to annotate the remote sensing images in the remote sensing image matching test dataset, thereby generating a remote sensing image matching test dataset with scene classification labels.

[0011] Secondly, embodiments of this application provide an apparatus for constructing a remote sensing image matching test dataset, comprising: The image data acquisition module is used to acquire remote sensing images of the target geographic area taken at different time dimensions as an image dataset, and to take two remote sensing images belonging to the same target geographic area in the image dataset as image pairs to be matched. The image pair matching processing module is used to input the image pairs to be matched in the image dataset into the matching model group one by one. Each matching model in the matching model group matches each image pair to be matched and obtains the matching results of the same image pair to be matched in different matching models. The matching difficulty level determination module is used to calculate the matching evaluation index of the same image pair to be matched based on the matching result, and determine the matching difficulty level of the corresponding image pair to be matched according to the matching evaluation index. The matching difficulty level classification module is used to classify the pairs of images to be matched in the image dataset by using the matching difficulty level as the matching difficulty classification label, and to generate remote sensing image matching test datasets with different matching difficulty levels.

[0012] Thirdly, embodiments of this application provide an apparatus comprising: a processor; a memory for storing processor-executable instructions; wherein, when the processor executes the executable instructions, it implements the method as described in the first aspect or any possible implementation of the first aspect.

[0013] Fourthly, embodiments of this application provide a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium including storage for storing a computer program or instructions that, when executed, cause the method described in the first aspect or any possible implementation of the first aspect to be implemented.

[0014] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: This application embodiment acquires remote sensing images of the same geographical area taken at different time dimensions as an image dataset, which includes at least one pair of images to be matched. The pairs of images to be matched are input one by one into a matching model group. Each matching model in the matching model group matches each pair of images to be matched, obtaining the matching results of the same pair of images to be matched in different matching models. Based on the matching results, a matching evaluation index for the same pair of images to be matched is calculated, and the matching difficulty level of the corresponding pair of images to be matched is determined according to the matching evaluation index. The matching difficulty level is used as a matching difficulty classification label to automatically classify the pairs of images to be matched in the image dataset, generating remote sensing image matching test datasets with different matching difficulty levels. This achieves efficient construction of remote sensing image matching test datasets and improves the accuracy of test dataset classification. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart illustrating a method for constructing a remote sensing image matching test dataset, provided in an embodiment of this application; Figure 2 A flowchart for calculating the matching evaluation index provided in the embodiments of this application; Figure 3 A flowchart for determining the matching difficulty level is provided for embodiments of this application; Figure 4 Example diagrams of image pairs representing the difficulty level of easy matching provided in embodiments of this application; Figure 5 Example images of image pairs matching difficulty levels provided in the embodiments of this application; Figure 6 Example images of image pairs representing difficulty levels of matching, provided for embodiments of this application; Figure 7 Example images of image pairs representing a very difficult matching level are provided for embodiments of this application; Figure 8 Example images of image pairs representing strongly textured region types provided in embodiments of this application; Figure 9 Example diagrams of image pairs representing texture region types provided in embodiments of this application; Figure 10 Example images of image pairs representing weakly textured region types provided in embodiments of this application; Figure 11 Example images of image pairs representing weakly textured region types provided in embodiments of this application; Figure 12 This is a schematic diagram of a device for constructing a remote sensing image matching test dataset, provided in an embodiment of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0018] The following description of some technologies involved in the embodiments of this application is provided to aid understanding and should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, some descriptions of well-known functions and structures are omitted in the following description.

[0019] This application provides a method for constructing a remote sensing image matching test dataset. By constructing datasets covering different difficulties and scenarios, the comprehensiveness of the test dataset is improved. Specifically, the "difficulty" classification focuses on judging the complexity of remote sensing images (such as the sharpness of image feature points, viewpoint changes, and noise interference), while the "scenario" classification focuses on judging the complexity of the environment (such as texture richness, object density, and environmental dynamics).

[0020] Figure 1 This is a flowchart illustrating a method for constructing a remote sensing image matching test dataset according to an embodiment of this application, including steps S10 to S50. Figure 1 This is merely one execution order shown in the embodiments of this application and does not represent the only execution order for a method of constructing a remote sensing image matching test dataset. Where the final result can be achieved, Figure 1 The steps shown can be performed in parallel or in reverse order.

[0021] S10: Obtain remote sensing images of the target geographic area taken at different time dimensions as an image dataset, and take two remote sensing images belonging to the same target geographic area in the image dataset as image pairs to be matched.

[0022] The target geographic area refers to a specific geographic area obtained based on actual needs, such as Street B in District A of City S.

[0023] Image datasets refer to collections of remote sensing images used to evaluate the effectiveness of intelligent matching capabilities in applications such as aircraft navigation, target localization, and flight path planning. In this embodiment, image datasets include strongly paired image datasets and weakly paired image datasets. Strongly paired image datasets refer to collections of remote sensing images acquired through simulation platforms or high-precision sensors (such as simultaneous acquisition by lidar and high-resolution cameras). These datasets have accurate feature point annotations (such as matching pairs of corresponding points based on geographic information) and a controllable environment (such as a fixed viewing angle), primarily used to evaluate the basic performance of the model under ideal conditions. Weakly paired image datasets refer to collections of remote sensing images acquired through real-world flight tests (such as acquisition by airborne cameras during flight tests). These datasets contain noise data from the real environment (such as motion blur, illumination changes, and sensor errors) and have lower annotation accuracy (such as point pairs obtained through motion methods), primarily used to evaluate the robustness of the model in real-world environments.

[0024] Specifically, remote sensing images of the same geographical area taken at different time dimensions are acquired as an image dataset. Since the subsequent matching model processes data in pairs of images, after obtaining images of the same geographical area taken at different time dimensions, remote sensing images of the same geographical area taken at two different time dimensions need to be selected as image pairs to be matched. Because in this embodiment, the same geographical area may contain remote sensing images taken at multiple time dimensions, the same geographical area in the image dataset may include at least one image pair to be matched.

[0025] Furthermore, when the image dataset in this embodiment stores remote sensing images of multiple geographic areas taken at different time dimensions, it is first necessary to filter out two remote sensing images of the same geographic area at different time dimensions, and then assign an identifier ID to the two remote sensing images. This identifier ID is a unique identifier used to distinguish remote sensing images; one identifier ID corresponds to only one remote sensing image. To reflect that two remote sensing images belong to the same geographic area, the identifier ID in this embodiment uses a combination of geographic area identifier, image pair identifier, and numerical identifier. The geographic area identifier is used to distinguish that the remote sensing images come from different geographic areas, and the image pair identifier is used to identify that the two remote sensing images belong to the same image pair to be matched. A numerical identifier is set for the remote sensing images to ensure the consistency of the two remote sensing images subsequently input into the matching model, i.e., that they are two remote sensing images of the same image pair to be matched.

[0026] For example, remote sensing images of geographic region P in two time dimensions are labeled Pa1 and Pb1, respectively. The remote sensing image corresponding to Pa1 is stored in folder a, and the remote sensing image corresponding to Pb1 is stored in folder b. Remote sensing images of geographic region R in two time dimensions are labeled Ra2 and Pb2, respectively. The remote sensing image corresponding to Ra2 is stored in folder a, and the remote sensing image corresponding to Rb2 is stored in folder b. Remote sensing images of geographic region M in two time dimensions are labeled Ma3 and Mb3, respectively. The remote sensing image corresponding to Ma3 is stored in folder a, and the remote sensing image corresponding to Mb3 is stored in folder b. Folders a and b output two remote sensing images in numerical order to form a pair of images to be matched.

[0027] Understandably, each matching model can output matching results for multiple pairs of images to be matched, and one pair of images to be matched will correspond to matching results output by multiple matching models.

[0028] S20: Input the pairs of images to be matched in the image dataset into the matching model group one by one. Each matching model in the matching model group matches each pair of images to be matched and obtains the matching results of the same pair of images to be matched in different matching models.

[0029] In this embodiment, the matching model group includes, but is not limited to, traditional image matching models, sparse image matching models, semi-dense image matching models, and dense image matching models.

[0030] The traditional image matching model refers to an image matching model designed using conventional methods. This model follows a three-stage pipeline of "detection-description-matching," relying on manual operators such as Scale Invariant Feature Transform (SIFT), Speeded Robust Feature Transform (SURF), and Fast Oriented BRIEF (ORB) to extract sparse keypoints, and performs geometric verification using Euclidean distance and the Random Sample Consensus (RANSAC) algorithm. While this traditional image matching model boasts advantages such as high computational efficiency and interpretability, its performance is limited in challenging scenarios such as weak textures, repetitive patterns, and extreme viewpoint changes.

[0031] Sparse image matching models refer to image matching models built on deep learning neural networks. These models learn highly discriminative descriptors using convolutional neural networks such as SuperPoint, and combine them with image neural networks and Transformer architectures such as SuperGlue or LightGlue to transform the feature correspondence problem into a differentiable allocation problem. This achieves end-to-end optimization of the entire process from feature detection and description to matching, significantly improving the robustness of images under varying lighting and occlusion conditions.

[0032] Semi-dense image matching models refer to semi-dense matching frameworks used for image recognition. These models employ a detector-free strategy, using a coarse-fine architecture to establish dense latent correspondences on downsampled feature maps through self-attention and cross-attention mechanisms, followed by local window refinement to achieve sub-pixel-level localization. LoFTR and ASpanFormer, as representative semi-dense image matching models, effectively solve the matching failure problem in low-texture regions, demonstrating significant advantages in wide-baseline stereo matching and indoor scene reconstruction.

[0033] Dense image matching models refer to image matching models that aim for continuous correspondence field estimation at the full pixel level. These dense image matching models typically use dense optical flow or probabilistic correspondences to explicitly model matching uncertainties through architectures such as PDC-Net, DKM, and RoMa, making them suitable for dense 3D reconstruction and semantic alignment tasks that require pixel-level accuracy.

[0034] Understandably, in this embodiment, the matching results output by each matching model are the position coordinates of the matching point pairs. Each pair of position coordinates is in the form of (Xa, Ya) and (Xb, Yb). The matching point pairs output by different matching models may differ, which is related to the structure of each matching model. Since different matching models have their own advantages and focuses, this embodiment uses different matching models to match the image pairs to be matched, which can improve the generalization and robustness of the matching results.

[0035] S30: Calculate the matching evaluation index for the same pair of images to be matched based on the matching results, and determine the matching difficulty level of the corresponding pair of images to be matched based on the matching evaluation index.

[0036] Among them, the matching evaluation index refers to the index used to evaluate the matching difficulty level of the image pair to be matched based on the matching results.

[0037] Specifically, after obtaining the matching results output by each matching model for the same image pair to be matched, a matching evaluation index for the same image pair to be matched is calculated based on each matching result. Then, the matching difficulty level of the corresponding image pair to be matched is determined by judging the matching evaluation index. In this embodiment, the matching difficulty level includes two levels: a first-level matching difficulty level and a second-level matching difficulty level. The first-level matching difficulty level applies to image pairs to be matched where at least one matching model in the aforementioned matching model group can successfully match them, while the second-level matching difficulty level applies to image pairs to be matched where no matching model in the aforementioned matching model group can successfully match them.

[0038] In this embodiment, the first-level matching difficulty level includes easy matching difficulty level, medium matching difficulty level, and difficult matching difficulty level; the second-level matching difficulty level includes very difficult matching difficulty level.

[0039] Furthermore, the absolute threshold for classifying the first-level and second-level matchmaking difficulty levels is 640. Based on a 480° remote sensing image, other sizes are W... Threshold of remote sensing image of H The absolute threshold is determined.

[0040] S40: Use the matching difficulty level as the classification label for matching difficulty to classify the pairs of images to be matched in the image dataset, and generate remote sensing image matching test datasets with different matching difficulty levels, so as to facilitate subsequent targeted algorithm testing and analysis for pairs of images to be matched with different matching difficulty levels.

[0041] Furthermore, such as Figure 2 As shown, after obtaining the matchable test set, step S30 calculates the matching evaluation index for the same pair of images to be matched based on the matching results, specifically including the following steps: S311: Calculate the matching point pair ratio of the same image pair to be matched based on the matching results output by each matching model. If the matching point pair ratio of at least one matching model is not less than the preset ratio threshold, then add the corresponding image pair to be matched as a matchable image pair to the matchable test set.

[0042] Specifically, after obtaining the matching results of each matching model for the same image pair to be matched, the matching point pair ratio of the same image pair to be matched is calculated based on the matching results output by different matching models. If the matching point pair ratio of a matching model is not less than a preset ratio threshold (such as 50%), the image pair to be matched is considered to be matchable, and the image pair to be matched is added to the matchable test set as a matchable image pair.

[0043] The matching point ratio in this embodiment can be set according to the actual situation. If a higher requirement is required, the matching point ratio can be set higher. If a lower requirement is required, the matching point ratio can be set lower, but it should not be lower than 50%.

[0044] Further, the specific steps for calculating the matching point pair ratio of the same image pair to be matched based on the matching results include: 1) Calculating the geometric consistency error of the matching results of the same image pair to be matched from different matching models using the RANSAC algorithm, and determining the matching point pairs with a geometric consistency error less than a preset threshold as inliers; 2) Projecting the inliers of one remote sensing image in the image pair to the coordinate system of the other remote sensing image using the ground homography matrix of the image pair to be matched, obtaining the true projection points; 3) Calculating the Euclidean distance between the inliers and the corresponding true projection points, and statistically analyzing the proportion of inliers whose Euclidean distance is within a preset pixel length to the total number of inliers as the matching point pair ratio of the matched image pair. Here, the preset pixel length is the product of the preset pixel size and the corresponding resolution.

[0045] S312: Count the number of the same matching image pairs in the matching test set as the number of matching models, and calculate the proportion of the number of matching models to the total number of matching models. Use the proportion of matching models as the first matching evaluation index.

[0046] S313: Calculate the average number of inliers for the same matchable image pair in the matchable test set, and use the average number of matching points as the second matching evaluation index.

[0047] S314: Obtain the ground homography matrix of the same image pair to be matched, and determine the target submatrix based on the ground homography matrix of the image pair to be matched.

[0048] Specifically, after obtaining the ground homography matrix of the same image pair to be matched, the target submatrix is ​​determined in the ground homography matrix (assuming that the ground homography matrix H is a 3×3 matrix, then the target submatrix A is the first 2×2 submatrix in the 3×3 matrix).

[0049] S315: Calculate the rotation angle of the target submatrix and convert the rotation angle into an angle value. Obtain the degree of rotation between the same pair of images to be matched as the third matching evaluation index.

[0050] Specifically, after determining the target submatrix, singular value decomposition (SVD) is performed (the SVD calculation formula is: The objective submatrix A of the linear transformation is decomposed into three independent matrices: a left-right orthogonal matrix and a singular value matrix. Then, a pure rotation matrix R is constructed based on the product of the two left-right orthogonal matrices U and V (i.e., ...). The rotation angle of the pure rotation matrix is ​​calculated using the arctangent function atan2; finally, the rotation angle is normalized to an angle value in the range of 0-180 degrees to represent the relative rotation degree between the image pairs to be matched.

[0051] S316: Calculate the square root of the absolute value of the determinant of the target submatrix as the scaling ratio, and determine the scaling ratio as the fourth matching evaluation index.

[0052] The scale ratio refers to the value used to determine the scaling degree of each remote sensing image in the image pair to be matched.

[0053] Specifically, a scale ratio greater than 1 indicates that remote sensing image 2 in the image pair to be matched is magnified relative to remote sensing image 1, less than 1 indicates that remote sensing image 2 in the image pair to be matched is shrunk relative to remote sensing image 1, and equal to 1 indicates that the scale of remote sensing image 2 in the image pair to be matched has not changed relative to remote sensing image 1, reflecting the overall scaling degree of the image pair to be matched.

[0054] S317: Matching evaluation metrics are formed based on the first matching evaluation metric, the second matching evaluation metric, the third matching evaluation metric, and the fourth matching evaluation metric to constitute the same image pair to be matched.

[0055] Furthermore, to facilitate the determination of the matching difficulty level of image pairs, this embodiment divides the image pairs to be matched into matchable image pairs that can be added to the matching test set and image pairs that cannot be added to the matchable test set. Matchable image pairs refer to image pairs that are determined to have a matching possibility according to the above matching model, while image pairs not added to the matchable test set refer to image pairs that are determined to have no matching possibility according to the above matching model. For matchable image pairs, the matching difficulty level is determined using a first matching difficulty level evaluation method; for image pairs that are not matchable image pairs, the matching difficulty level is determined using a second matching difficulty level evaluation method, such as... Figure 3 As shown, in step S30, the matching difficulty level of the corresponding image pair to be matched is determined according to the matching evaluation index, which specifically includes the following steps: S321: For the matchable image pairs in the matchable test set, determine whether the third matching evaluation index and the fourth matching evaluation index meet the first preset matchable index range.

[0056] S322: When the third matching evaluation index and the fourth matching evaluation index meet the first preset matchable index range, then based on the first matching evaluation index and the second matching evaluation index, the first matching difficulty level evaluation is performed on the matchable image pairs corresponding to the matchable test set to obtain the matching difficulty level of the corresponding matchable image pairs.

[0057] The first preset matchable index range refers to the index range used to determine whether the matching difficulty level of matchable image pairs in the matchable test set can be determined. The first preset matchable index range includes a first preset rotation angle range and a first preset scale ratio range.

[0058] Specifically, for matchable image pairs, the third matching evaluation metric satisfies the first preset rotation angle range (e.g., the rotation angle is within a certain range). And the fourth matching evaluation index meets the first preset scale ratio range (e.g., the scale ratio is within the range of the first preset scale ratio). The matching difficulty level is categorized into Level 1. In this embodiment, Level 1 matching difficulty includes: easy matching difficulty, medium matching difficulty, and difficult matching difficulty.

[0059] Furthermore, after determining that the matchable image pairs belong to the first-level matching difficulty level based on the third and fourth matching evaluation indicators, the specific matching difficulty level, medium matching difficulty level, or difficult matching difficulty level of the matchable image pairs is determined based on the first and second matching evaluation indicators.

[0060] The process for determining the difficulty level of matching is as follows: the first matching evaluation indicator (the proportion of matchable models, i.e., the proportion of matchable models to the total number of matched models) is... Furthermore, the second matching evaluation metric for sparse image matching models, the average number of matching points (the average number of interior points that are "matchable" for the image pair across all sparse image matching models), is... Or, the second matching evaluation metric for traditional image matching models, semi-dense image matching models, and dense image matching models is the average number of matching points. It is sufficient to satisfy the second matching evaluation metric of a matching model.

[0061] The specific process for determining the difficulty level of matching is as follows: The first matching evaluation indicator is the proportion of models that can be matched. Furthermore, the second matching evaluation metric of the sparse image matching model, the average number of matching points, is in Or, the second matching evaluation metric for traditional image matching models, semi-dense image matching models, and dense image matching models is the average number of matching points. It is sufficient to satisfy the second matching evaluation metric of a matching model; and the proportion of models that can be matched by the first matching evaluation metric is within a certain range. Furthermore, the second matching evaluation metric of the sparse image matching model, the average number of matching points, is in Or, the second matching evaluation metric for traditional image matching models, semi-dense image matching models, and dense image matching models is the average number of matching points. It is sufficient to satisfy the second matching evaluation metric of a matching model.

[0062] The process for determining the difficulty level of matching is as follows: The first matching evaluation metric is the proportion of models that can be matched. Furthermore, the second matching evaluation metric of the sparse image matching model, the average number of matching points, is in Or, the second matching evaluation metric for traditional image matching models, semi-dense image matching models, and dense image matching models is the average number of matching points. It is sufficient to satisfy the second matching evaluation metric of a matching model.

[0063] S323: For image pairs to be matched that do not belong to the matchable test set, determine whether the third matching evaluation index and the fourth matching evaluation index meet the range of the second preset matchable index.

[0064] S324: When the third matching evaluation index and the fourth matching evaluation index meet the second preset matchable index range, then based on the third matching evaluation index and the fourth matching evaluation index, the second matching difficulty level evaluation is performed on the image pairs to be matched that have not been included in the matchable test set, and the matching difficulty level of the corresponding image pairs to be matched is obtained.

[0065] The second preset matchable index range refers to the index range used to determine whether the difficulty level of matching can be determined for matching image pairs that do not belong to the matchable test set. The second preset matchable index range includes the second preset rotation angle range and the second preset scale ratio range.

[0066] Specifically, when the third matching evaluation index meets the second preset rotation angle range (e.g., the rotation angle is within the range of the second preset rotation angle range), And the fourth matching evaluation index meets the second preset scale ratio range (e.g., the scale ratio is within the range of 0.5%). If the image pairs to be matched are not included in the matchable test set, then a second matching difficulty level evaluation is performed to obtain the matching difficulty level of each image pair to be matched.

[0067] The second matching difficulty level evaluation in this embodiment includes: very difficult matching difficulty level.

[0068] To visually represent the results obtained after performing steps S10-S40 in this embodiment, as follows: Figure 4 As shown, this represents the matchable image pairs according to their difficulty level; such as... Figure 5 As shown, this represents image pairs that match difficulty levels, such as... Figure 6 As shown, these represent the matchable image pairs according to their difficulty level; for example... Figure 7 As shown, the matching image pairs represent the difficulty level of "very difficult to match". Furthermore, after automatically classifying the image pairs to be matched according to the matching difficulty level to obtain remote sensing image matching test datasets of different difficulty levels, this embodiment also provides a method for constructing a remote sensing image matching test dataset, which specifically includes the following steps: S50 uses a semantic segmentation network to extract scene features from remote sensing images in a remote sensing image matching test dataset, and determines the texture region type based on the extracted scene features.

[0069] Specifically, this embodiment uses semantic segmentation networks (including but not limited to DeepLabV3+ and SAM) to evaluate the texture complexity of each remote sensing image in the remote sensing image matching test dataset, thereby classifying the remote sensing images into three types: strong texture, medium texture, and weak texture. Strong texture types (such as urban centers) typically exhibit complex spatial structures, characterized by high-density building clusters and intricate road networks, with high texture contrast and object density. Medium texture types (such as rural areas) typically exhibit moderate texture structure and object density. Weak texture types (such as deserts and open water areas) have relatively simple structures, monotonous surface texture, lack of prominent objects, and low grayscale contrast. Texture region classification enables the rational allocation of computational resources, effectively improving overall processing efficiency while ensuring processing quality.

[0070] This embodiment uses a semantic segmentation network to extract scene features from remote sensing images in a remote sensing image matching test dataset, and determines the texture region type based on the extracted scene features. Specifically, it includes the following steps: Step 1: Semantic Segmentation Processing A semantic segmentation network is used to perform pixel-level semantic segmentation on each remote sensing image in the remote sensing image matching test dataset. This semantic segmentation network employs a sliding window strategy to process the remote sensing images. (pixels), window size set to The pixels are stride-sized to the same size as the window to avoid overlap. The semantic segmentation network outputs a segmentation mask of the same size as the remote sensing images in the remote sensing image matching test dataset, where each pixel is assigned to one of eight predefined categories. The predefined categories involved in this embodiment include: background, bare land, grass, road, tree, water, cropland, and building.

[0071] Step 2: Calculation of Category Ratios Based on the segmentation mask, the pixel proportion of each predefined category in the remote sensing image is calculated. The specific formula is:

[0072] in, Indicates the first The proportion of each predefined category. Indicates the first The number of pixels in each predefined category. This represents the total number of pixels in the remote sensing images in the remote sensing image matching test dataset. The final result is a vector composed of eight scale values, representing the distribution of each predefined category in the remote sensing image.

[0073] Furthermore, after determining the category proportions, two key features are extracted from the corresponding category proportion vectors as scene features for scene classification. Among them, Scene Feature 1: Number of Valid Categories ( ): The number of categories whose statistical proportion exceeds a preset threshold (set to 1%), reflecting the significant semantic category diversity in remote sensing images.

[0074] Scene Feature 2: Maximum Category Ratio ( ): Identify the category with the highest proportion and its corresponding proportion value, reflecting the dominant features of the remote sensing image.

[0075] After determining the scene features, the texture region type is determined based on the extracted scene features. In this embodiment, the texture region types include weak texture, medium texture, and strong texture. Weak texture determination satisfies the following condition: if the number of valid categories If the texture is weak, the remote sensing image is classified as weak texture. These types of remote sensing images typically exhibit simple or minimalist structural features, such as large areas of farmland or water.

[0076] Texture determination satisfies either of the following conditions: 1) If the number of valid categories is... ;2) If the number of valid categories and the largest category proportion This means that a single category dominates. These types of remote sensing images contain structures of moderate complexity, such as suburban landscapes or mixed farmland.

[0077] Strong texture determination satisfies any of the following conditions: 1) If the number of valid categories is... and the largest category proportion 2) If the largest category is building, it is directly classified as strong texture. This type of remote sensing image contains complex spatial structures, such as city centers.

[0078] S60: Use texture region type as scene classification label to annotate remote sensing images in the remote sensing image matching test dataset, and generate a remote sensing image matching test dataset with scene classification labels to reflect the real and complex real environment.

[0079] To visually represent the results obtained after executing steps S50-S60 in this embodiment, as follows: Figure 8 As shown, this represents a highly textured region – a city; for example… Figure 9 As shown, this represents a medium-texture region - rural; as Figure 10 As shown, this represents a weakly textured region – the ocean; as... Figure 11 As shown, this represents a region with weak textures – farmland and wasteland.

[0080] In this embodiment, remote sensing images of the same geographical area taken at different times are acquired to form image pairs to be matched. Multiple matching models in a matching model group are used to match each image pair, obtaining matching results from different models. Then, a matching evaluation index is calculated based on the matching results for the same image pair, and the matching difficulty level of the corresponding image pair is determined according to the matching evaluation index, thereby achieving automatic classification of image pairs with different matching difficulty levels. Simultaneously, semantic segmentation and scene feature analysis methods are used to perform semantic scene classification on the image data in the remote sensing image matching test dataset, thereby achieving efficient construction of the test dataset system and improving classification accuracy.

[0081] While this application provides method operation steps as shown in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-inventive labor. The order of steps listed in this embodiment is merely one possible execution order among many and does not represent the only execution order. In actual device or client product execution, the method can be executed sequentially according to this embodiment or the accompanying drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0082] like Figure 12 As shown in the illustration, this application also provides an apparatus for constructing a remote sensing image matching test dataset. The apparatus includes: The image data acquisition module 10 is used to acquire remote sensing images of the target geographic area taken at different time dimensions as an image dataset, and to take two remote sensing images belonging to the same target geographic area in the image dataset as a pair of images to be matched. The image pair matching processing module 20 is used to input the image pairs to be matched in the image dataset into the matching model group one by one. Each matching model in the matching model group matches each image pair to be matched and obtains the matching results of the same image pair to be matched in different matching models. The matching difficulty level determination module 30 is used to calculate the matching evaluation index of the same image pair to be matched based on the matching result, and determine the matching difficulty level of the corresponding image pair to be matched based on the matching evaluation index. The matching difficulty level classification module 40 is used to classify the image pairs to be matched in the image dataset by using the matching difficulty level as the matching difficulty classification label, and to generate remote sensing image matching test datasets with different matching difficulty levels.

[0083] Some modules in the apparatus described in this application can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0084] The apparatus or module described in the above embodiments can be implemented by a computer chip or physical entity, or by a product with a certain function. For ease of description, the above apparatus is described by dividing it into various modules according to their functions. When implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware. Of course, a module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.

[0085] The methods, apparatus, or modules described in this application can be implemented in a computer-readable program code manner. The controller can be implemented in any suitable manner, such as a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of a memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code manner, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included within it for implementing various functions can also be considered as structures within the hardware component. Alternatively, the device used to implement various functions can be viewed as either a software module that implements the method or a structure within a hardware component.

[0086] This application also provides an apparatus, the apparatus comprising: a processor; a memory for storing processor-executable instructions; wherein, when the processor executes the executable instructions, it implements the method described in this application.

[0087] This application also provides a non-volatile computer-readable storage medium storing a computer program or instructions thereon, which, when executed, enables the method described in this application embodiment to be implemented.

[0088] Furthermore, in the various embodiments of the present invention, each functional module can be integrated into a processing module, or each module can exist independently, or two or more modules can be integrated into a single module.

[0089] The aforementioned storage media include, but are not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), Cache, Hard Disk Drive (HDD), or Memory Card. The memory can be used to store computer program instructions.

[0090] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, or it can be embodied in the process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0091] The various embodiments described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. All or part of this application can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, mobile communication terminals, multiprocessor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0092] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of this application.

Claims

1. A method for constructing a remote sensing image matching test dataset, characterized in that, include: Remote sensing images of the target geographic area taken at different times are acquired as an image dataset, and two remote sensing images belonging to the same target geographic area in the image dataset are taken as image pairs to be matched. The image pairs to be matched in the image dataset are input one by one into the matching model group. Each matching model in the matching model group matches each image pair to be matched, and obtains the matching results of the same image pair in different matching models. Based on the matching results, a matching evaluation index is calculated for the same pair of images to be matched, and the matching difficulty level of the corresponding pair of images to be matched is determined according to the matching evaluation index. The matching difficulty level is used as the matching difficulty classification label to classify the pairs of images to be matched in the image dataset, generating remote sensing image matching test datasets with different matching difficulty levels.

2. The method for constructing a remote sensing image matching test dataset according to claim 1, characterized in that, The calculation of the matching evaluation index for the same pair of images to be matched based on the matching results includes: The matching point pair ratio of the same image pair to be matched is calculated based on the matching results output by each matching model. If the matching point pair ratio of at least one matching model is not less than the preset ratio threshold, the corresponding image pair to be matched is added to the matchable test set as a matchable image pair. The number of identical matching image pairs in the matching test set is counted as the number of matching models, and the proportion of the number of matching models to the total number of matching models is calculated. The proportion of matching models is used as the first matching evaluation index. Calculate the average number of inliers for the same matchable image pair in the matchable test set, and use the average number of matching points as the second matching evaluation index. Obtain the ground homography matrix of the same image pair to be matched, and determine the target submatrix based on the ground homography matrix of the image pair to be matched; Calculate the rotation angle of the target sub-matrix and convert the rotation angle into an angle value, and obtain the degree of rotation between the same pair of images to be matched as a third matching evaluation index; The square root of the absolute value of the determinant of the target submatrix is ​​calculated as the scaling ratio, and the scaling ratio is determined as the fourth matching evaluation index. The matching evaluation index for the same image pair to be matched is constructed based on the first matching evaluation index, the second matching evaluation index, the third matching evaluation index, and the fourth matching evaluation index.

3. The method for constructing a remote sensing image matching test dataset according to claim 2, characterized in that, The calculation of the proportion of matching point pairs for the same image pair to be matched based on the matching results includes: The RANSAC algorithm is used to calculate the geometric consistency error of the matching results of the same image pair to be matched from different matching models. Matching point pairs with a geometric consistency error less than a preset threshold are determined as inliers. By using the true homography matrix of the image pair to be matched, the interior points of one remote sensing image in the image pair to be matched are projected onto the coordinate system of the other remote sensing image to obtain the true projection points. Calculate the Euclidean distance between the interior point and the corresponding real projection point, and count the proportion of interior points whose Euclidean distance is within a preset pixel length to the total number of interior points as the matching point pair ratio of the image pairs to be matched.

4. The method for constructing a remote sensing image matching test dataset according to claim 2, characterized in that, The step of calculating the rotation angle of the target sub-matrix and converting the rotation angle into an angle value to obtain the degree of rotation between the same pair of images to be matched includes: The target submatrix is ​​decomposed using singular value decomposition, and a pure rotation matrix is ​​constructed based on the decomposed matrix. The rotation angle of the pure rotation matrix is ​​calculated using the arctangent function, and the rotation angle is normalized to an angle value within the range of 0-180 degrees. The angle value represents the degree of rotation between the same matching image pair.

5. The method for constructing a remote sensing image matching test dataset according to claim 2, characterized in that, The step of determining the matching difficulty level of the corresponding matching image pair based on the matching evaluation index includes: For the matchable image pairs in the matchable test set, determine whether the third and fourth matching evaluation indicators meet the first preset matchable indicator range. When the third matching evaluation index and the fourth matching evaluation index meet the first preset matchable index range, then based on the first matching evaluation index and the second matching evaluation index, the matchable image pairs corresponding to the matchable test set are evaluated for a first matching difficulty level to obtain the matching difficulty level of the corresponding matchable image pairs.

6. The method for constructing a remote sensing image matching test dataset according to claim 5, characterized in that, The step of determining the matching difficulty level of the corresponding matching image pair based on the matching evaluation index further includes: For image pairs to be matched that do not belong to the matchable test set, determine whether the third and fourth matching evaluation indicators meet the range of the second preset matchable indicators. When the third matching evaluation index and the fourth matching evaluation index meet the second preset matchable index range, then based on the third matching evaluation index and the fourth matching evaluation index, the second matching difficulty level evaluation is performed on the image pairs to be matched that have not been included in the matchable test set, and the matching difficulty level of the corresponding image pairs to be matched is obtained.

7. The method for constructing a remote sensing image matching test dataset according to claim 1, characterized in that, The method for constructing the remote sensing image matching test dataset also includes: Scene features are extracted from remote sensing images in the remote sensing image matching test dataset using a semantic segmentation network, and the texture region type is determined based on the extracted scene features. The texture region type is used as the scene classification label to annotate the remote sensing images in the remote sensing image matching test dataset, thereby generating a remote sensing image matching test dataset with scene classification labels.

8. A device for constructing a remote sensing image matching test dataset, characterized in that, include: The image data acquisition module is used to acquire remote sensing images of the target geographic area taken at different time dimensions as an image dataset, and to take two remote sensing images belonging to the same target geographic area in the image dataset as image pairs to be matched. The image pair matching processing module is used to input the image pairs to be matched in the image dataset into the matching model group one by one. Each matching model in the matching model group matches each image pair to be matched and obtains the matching results of the same image pair to be matched in different matching models. The matching difficulty level determination module is used to calculate the matching evaluation index of the same image pair to be matched based on the matching result, and determine the matching difficulty level of the corresponding image pair to be matched according to the matching evaluation index. The matching difficulty level classification module is used to classify the image pairs to be matched in the image dataset by using the matching difficulty level as the matching difficulty classification label, and generate remote sensing image matching test datasets with different matching difficulty levels.

9. An apparatus for performing a method for constructing a remote sensing image matching test dataset, characterized in that, include: processor; Memory used to store processor-executable instructions; When the processor executes the executable instructions, it implements the method as described in any one of claims 1 to 7.

10. A non-volatile computer-readable storage medium, characterized in that, Includes storage of computer programs or instructions that, when executed, cause the method as described in any one of claims 1 to 7 to be implemented.

Images