A method for detecting mosaic misalignment in remote sensing images

Abstract

This invention relates to the field of remote sensing image processing technology, specifically to a method for detecting mosaic misalignment in remote sensing images. The method includes: training: acquiring a dataset and constructing a training sample set; pre-constructing a classification model based on an improved classification network, and training the classification model using a two-stage training method and the training sample set until a target classification model is obtained; inference: acquiring the mosaic lines and orthophotos of a region of the image to be detected, extracting the regions on both sides containing the mosaic line features, and cropping them according to a preset size to obtain several image blocks; inputting all image blocks as input images into the target classification model, and outputting a misalignment detection result describing whether a misalignment exists in the image to be detected. This invention extracts image regions on both sides of the mosaic lines by cropping image blocks along the mosaic lines, and automatically determines whether a misalignment exists using an improved classification model. The final detection result is output in vectorized form for subsequent misalignment repair or quality assessment.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and specifically to a method for detecting mosaic misalignment in remote sensing images. Background Technology

[0002] Traditional mosaicking misalignment detection techniques mainly include three methods: gradient-based, corner-based, and spectral phase shift estimation-based. However, these three traditional methods generally suffer from insufficient detection accuracy, poor adaptability to complex scenes, and low efficiency, making it difficult to meet the practical needs of high-precision, large-scale remote sensing image mosaicking misalignment detection.

[0003] To address this, researchers have further incorporated deep learning technology into mosaicking misalignment detection, automatically extracting deep features from images on both sides of the mosaicking seam and learning complex misalignment patterns to achieve accurate and efficient detection. For example, 1) Pan et al.'s paper, "Blind Evaluation of Remote Sensing Image Mosaic Quality Based on Weak Supervision," published in the *Journal of Remote Sensing*, designed a dual-branch network to extract and learn features of color differences and ground feature misalignments on both sides of the mosaicking seam, employing a two-stage weakly supervised learning strategy to solve the problem of high annotation costs for remote sensing images. Furthermore, this method constructed a large-scale training set using simulation data and calibrated the model using real-world annotation data, attempting to achieve accurate evaluation of complex mosaicking problems even with limited data. However, this method has two shortcomings: first, the model's generalization ability is weak due to data limitations; second, the model struggles to capture detailed features in complex scenes, thus affecting detection accuracy. 2) Chinese patent CN119048520A, entitled "Detection Method for Misalignment Points in Image Mosaic Based on Satellite Imagery and VGG Model Fine-tuning," primarily targets misalignment phenomena caused by geometric errors, feature offsets, and discontinuous seams during satellite image mosaicking. The method uses VGG as the feature extraction network, but its feature extraction and representation capabilities are relatively limited. Especially when processing high-resolution, complex features, it struggles to capture detailed information in the areas on both sides of the mosaic line, affecting the accuracy and robustness of the detection.

[0004] Besides the methods mentioned above, researchers have also employed different technical approaches, such as deep learning, human collaboration, and optimization algorithms, to address issues like color difference detection, mosaicking editing, and mosaicking line optimization in image mosaicking. However, none of these methods are suitable for the automated identification and detection of mosaicking misalignments. For example, 1) Chinese patent CN119360052A, entitled "Automatic Detection Method for Color Difference in Large-Area Remote Sensing Image Mosaic," enhances the model's global representation of image color difference features by introducing a global response normalization (GRN) mechanism, enabling it to automatically and efficiently detect and label color difference regions on both sides of the mosaicking line. However, this method is not suitable for mosaicking misalignment detection because the problem of mosaicking misalignment is essentially a mismatch in the geometric position of the image. The ConvNeXt enhanced by the global response normalization mechanism focuses on extracting global color information from the image and lacks sufficient sensitivity to local geometric misalignments and textural differences, making it difficult to effectively capture and identify geometric misalignment phenomena in images. 2) Chinese patent CN118886868A, entitled "Method, System, Device and Medium for Collaborative Color Mosaic Editing of Orthophotos," achieves efficient manual detection and correction of color differences during large-scale orthophoto mosaicking through reasonable task division, personnel management and collaborative operation. However, this technology relies entirely on manual visual judgment and does not involve automatic algorithm detection, therefore it cannot be used for mosaicking misalignment detection, which requires automated identification of local geometric misalignments. 3) Chinese patent CN111311622A, entitled "Optimization Selection Method for Multi-Image Mosaic Net Based on Ant Colony Algorithm," automatically determines the optimal mosaicking line position by analyzing the grayscale differences and texture features of the images to achieve seamless image stitching. However, this technology does not involve the identification or detection of geometric misalignments that may occur during the mosaicking process, therefore it is not suitable for mosaicking misalignment detection tasks.

[0005] There are also some image stitching and defect detection methods in the industrial field, but each has certain limitations and is not suitable for automated detection of mosaic misalignment in remote sensing images. For example: 1) Chinese patent CN114022450A, entitled "A Method for Judging the Stitching Effect of Vehicle Panoramic Surround View Testing," identifies gaps, misalignments, ghosting, and losses through preset detection rules. However, this method is only applicable to clearly defined rules in vehicle scenarios and is difficult to adapt to complex misalignment situations in remote sensing images. 2) Chinese patent CN116309314A, entitled "A Method and System for Calculating the Quality of Vehicle Panoramic Surround View Stitched Images Based on Statistics," detects misalignment through the segmentation and clustering fitting of checkerboard regions. However, this method relies on manually placing special checkerboard auxiliary equipment, limiting its versatility. 3) Chinese patent CN116486178A, entitled "A Defect Detection Method, Device, Electronic Equipment, and Storage Medium," determines the defect area by analyzing the distance from pixel features to the feature center point in the image to be detected. This method is only suitable for industrially processed parts with relatively fixed structures and well-defined features. 4) The Chinese patent with publication number CN118691954A, entitled "Method for analyzing the stitching quality of 360° panoramic images of automobiles based on machine vision", analyzes the changes in stitching quality through real-time driving speed, but mainly focuses on the stability of stitching quality with speed, and is not applicable to the detection of geometric misalignment in static remote sensing images.

[0006] To address the aforementioned issues, those skilled in the art urgently need to propose a novel method for detecting mosaic misalignments. Summary of the Invention

[0007] Therefore, the technical problem to be solved by the present invention is to overcome the defects existing in the prior art, thereby providing a method for detecting misalignment in remote sensing image mosaic.

[0008] A method for detecting mosaic misalignment in remote sensing images includes: Training: Acquire the dataset and construct the training sample set; pre-build the classification model based on the improved classification network, and train the classification model using a two-stage training method and the training sample set until the target classification model is obtained after training; Inference: Obtain the mosaic lines and orthophotos of a certain region of the image to be detected, extract the two sides containing the mosaic line features, and crop them according to the preset size to obtain several image blocks; All image patches are input into the target classification model, and the output is a misalignment detection result describing whether misalignment exists in the image to be detected.

[0009] Preferably, the classification model is trained using a two-stage training method and a training sample set, specifically including: The first stage of training involves randomly sampling negative samples from the dataset to ensure a 1:1 ratio of positive to negative samples during the first stage of training, and then augmenting the data using data augmentation methods. The classification model is then trained using the training sample set obtained from the processed dataset, and further optimized using a loss function and a training optimizer to obtain the first model. The second stage of training involves obtaining the model weights of the first model as initial weights, then retraining the classification model using all positive and negative samples in the dataset. During the training process, data augmentation methods, loss functions, and training optimizers are also used, along with training samples obtained from the dataset, to finally obtain the target classification model.

[0010] The preferred and improved classification network is: using ConvNeXt as the base network, further introducing a Fourier edge detection module that synchronously processes the model input image and four pre-built edge preprocessing modules connected to the Fourier edge detection module; Furthermore, an edge attention module connected to the corresponding edge preprocessing module is added at the input of each of the four stages of the ConvNeXt backbone network.

[0011] Preferably, the network structure of the edge preprocessing module is a sequentially connected convolution-batch normalization-activation-pooling.

[0012] Preferably, the network structure of the edge attention module includes two parts: the first part is an edge feature-convolution-batch normalization-convolution-activation connected in sequence; The second part is the intermediate feature; Furthermore, the intermediate features are multiplied by the activation of the first part to obtain a product, and the intermediate features are then summed with the product.

[0013] Preferably, acquiring the dataset and constructing the training sample set specifically includes: The misaligned image regions and the repaired, non-misaligned image regions are extracted from the pre-acquired orthophotos, and then cropped, labeled, and saved to a pre-established empty set to obtain the dataset. Remove images from the dataset whose labels do not correspond to whether a misalignment has occurred; Negative samples in the dataset are randomly sampled to control the ratio of positive to negative samples; images that are misaligned in the dataset are considered positive samples, while images that are not misaligned are considered negative samples. The image sizes in the dataset are normalized, and images exceeding the size threshold are removed, ultimately yielding the dataset required by the classification model.

[0014] Preferably, negative samples in the dataset are randomly sampled to control the ratio of positive to negative samples in the dataset to be 1:100.

[0015] The technical solution of this invention has the following advantages: This invention uses ConvNeXt as the basic structure of the classification model, but it lacks sensitivity to local geometric misalignments and textural differences in remote sensing images. Therefore, this invention designs an edge attention module to enhance the classification model's sensitivity to texture details and edge features, thereby significantly improving the accuracy of misalignment detection. Simultaneously, to alleviate the problem of imbalanced positive and negative samples, a two-stage training method is designed. This method provides an efficient solution for mosaic quality control in DOM production, significantly improving the automation level and application value of large-scale remote sensing image data processing. Attached Figure Description

[0016] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the overall network structure of the classification model of the present invention; Figure 2 This is a schematic diagram of important nodes in the training and inference phases of Embodiment 2 of the present invention; Figure 3 This is an example diagram of the detection results in Embodiment 2 of the present invention. Detailed Implementation

[0018] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0020] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0021] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0022] Example 1 Before detailing the technical solution of this embodiment, relevant technical concepts will be introduced to facilitate understanding. Digital orthophoto map (DOM) is a geometrically corrected remote sensing image product. By eliminating the effects of terrain undulations, sensor tilt, perspective distortion, and other factors, it precisely positions each pixel in the image to its actual location on the ground, ensuring the image has a uniform scale and true geographic coordinates. This allows DOM to not only possess rich image details but also be directly used in Geographic Information Systems (GIS) for analysis and mapping. DOM is widely used in fields such as land resource surveys, agricultural monitoring, urban planning, environmental protection, traffic management, and disaster emergency response, providing crucial spatial information support and possessing extremely high social and economic value.

[0023] Mosaic technology is an indispensable part of the DOM (Image Atlas) production process. Its main task is to stitch multiple remote sensing images together geographically into a continuous, fully covering image atlas. In large-area DOM production, mosaic technology can significantly improve image integration efficiency. However, due to issues such as geometric correction errors in remote sensing images, the influence of terrain undulations, and registration errors, image misalignment can easily occur on both sides of the mosaic lines. Mosaic misalignment not only compromises the geometric accuracy of the images but also affects the visual effect, thereby weakening the application value of the DOM and creating difficulties for users in data analysis and decision-making.

[0024] Traditional mosaicking misalignment detection techniques mainly include three methods: gradient-based, corner-based, and spectral phase shift estimation-based. Gradient-based detection algorithms determine misalignment regions by calculating image gradient changes. However, in complex terrain scenes, gradient changes are easily affected by texture similarity and noise interference, leading to inaccurate detection results. Corner-based detection algorithms rely on the matching relationships of salient feature points in the image, such as corners, to determine misalignment. However, when the texture of the images on both sides of the mosaic line is not obvious or the terrain features are sparse, the stability and robustness of this method are significantly reduced. Furthermore, spectral phase shift estimation algorithms estimate mosaicking errors by analyzing the frequency domain features of the image, typically using changes in phase difference in the frequency domain to determine misalignment regions. However, this method is highly dependent on the completeness of spectral information and the regularity of frequency distribution. When the image texture is complex or the terrain features vary greatly, the frequency domain features may not have obvious periodicity, making it difficult to accurately estimate the phase difference and thus affecting detection accuracy. In addition, this method has high requirements for the symmetry of the images on both sides of the mosaic line, making it difficult to adapt to scenes with large terrain undulations or uneven distribution of terrain features.

[0025] To address the aforementioned issues, this embodiment proposes a mosaic misalignment detection method. This method extracts image regions on both sides of the mosaic lines by cutting out silhouette blocks along the mosaic lines, and automatically determines whether misalignment exists using an improved classification model. The final detection result is output in vectorized form for subsequent misalignment repair or quality assessment. A remote sensing image mosaic misalignment detection method includes: Training: Acquire the dataset and construct the training sample set; pre-build the classification model based on the improved classification network, and train the classification model using a two-stage training method and the training sample set until the target classification model is obtained after training; Inference: Obtain the mosaic lines and orthophotos of a certain region of the image to be detected, extract the two sides containing the mosaic line features, and crop them according to the preset size to obtain several image blocks; All image patches are input into the target classification model, and the output is a misalignment detection result describing whether misalignment exists in the image to be detected.

[0026] Specifically: Obtaining the dataset and constructing the training sample set specifically includes: The misaligned image regions and the repaired, non-misaligned image regions are extracted from the pre-acquired orthophotos, and then cropped, labeled, and saved to a pre-established empty set to obtain the dataset. Remove images from the dataset whose labels do not correspond to whether a misalignment has occurred; Negative samples in the dataset are randomly sampled to control the ratio of positive to negative samples in the dataset; images that are misaligned in the dataset are positive samples, and images that are not misaligned are negative samples.

[0027] The image sizes in the dataset are normalized, and images exceeding the size threshold are removed, ultimately yielding the dataset required by the classification model.

[0028] Negative samples in the dataset are randomly sampled to control the ratio of positive to negative samples in the dataset to be 1:100.

[0029] The classification model is trained using a two-stage training method and a training sample set, specifically including: The first stage of training involves randomly sampling negative samples from the dataset to ensure a 1:1 ratio of positive to negative samples during the first stage of training, and then augmenting the data using data augmentation methods. The classification model is then trained using the training sample set obtained from the processed dataset, and further optimized using a loss function and a training optimizer to obtain the first model. The second stage of training involves obtaining the model weights of the first model as initial weights, then retraining the classification model using all positive and negative samples in the dataset. During the training process, data augmentation methods, loss functions, and training optimizers are also used, along with training samples obtained from the dataset, to finally obtain the target classification model.

[0030] The improved classification network is as follows: ConvNeXt is used as the base network, and a Fourier edge detection module that synchronously processes the input image of the model and four pre-built edge preprocessing modules connected to the Fourier edge detection module are further introduced. An edge attention module, connected to the corresponding edge preprocessing module, is added at the input of each of the four stages of the ConvNeXt backbone network.

[0031] The network structure of the edge preprocessing module is a sequentially connected convolution-batch normalization-activation-pooling.

[0032] The network structure of the edge attention module consists of two parts: the first part is a sequentially connected edge feature-convolution-batch normalization-convolution-activation; The second part is the intermediate feature; Furthermore, the intermediate features are multiplied by the activation of the first part to obtain a product, and the intermediate features are then summed with the product.

[0033] Specifically: the network structure of the classification model is shown in the attached figure. Figure 1As shown, the classification model takes an 8-bit remote sensing image with a size of 224×224×3 as input. It extracts features step-by-step through a series of modules to ultimately determine whether misalignment exists in the input image. First, these input images are processed by a Fourier edge detection module to extract high-frequency edge features. The Fourier edge detection module converts the input image into a grayscale image, performs a two-dimensional Fast Fourier Transform (FFT) to obtain its frequency domain representation, and then applies a high-pass filter to mask low-frequency components, retaining only the image's edge information. Finally, the edge feature map is restored through an inverse Fourier transform, and after normalization, a single-channel edge feature map with a size of [224, 224, 1] is output.

[0034] The first part of the backbone network is convolution and layer normalization. The input image is convolved with a 4×4 kernel and a stride of 4 to achieve downsampling, while simultaneously adjusting the number of channels. This step transforms the input image feature map from [224,224,3] to [56,56,96]. Meanwhile, the edge feature map, after passing through the first-stage edge preprocessing module, is aligned with the image feature map, and its width and height are adjusted to match the input image feature map. Figure 1 The edge preprocessing module performs convolution and pooling operations on the edge feature map to further encode and enhance feature representation capabilities and achieve multi-scale matching. The image feature map and the edge feature map are fused through the edge attention module. The edge attention module performs two convolution operations on the edge features to generate an attention map, multiplies the attention weights element-wise with the image features, and then adds them back to the original image feature map, thereby strengthening the feature representation of the edge region. The size of the fused feature map remains [56, 56, 96].

[0035] In the four stages of the backbone network, each stage consists of several ConvNeXt modules. The core of each ConvNeXt module includes depthwise separable convolutions for extracting spatial features, channel normalization for stabilizing feature distribution, and point convolutions to enhance global feature representation across the channel dimension. The input feature map size in the first stage is [56, 56, 96], which remains unchanged after processing by three ConvNeXt modules. Subsequently, the image feature map is downsampled using convolutions with a stride of 2, entering the next stage where the image feature map size becomes [28, 28, 192]. Simultaneously, the edge feature map is adjusted to the same width and height through corresponding edge preprocessing modules. After downsampling in each stage, the image feature map and edge feature map are fused using an edge attention module to enhance the feature representation of edge regions. Similarly, the image feature map sizes in subsequent stages become [14, 14, 384] and [7, 7, 768], respectively. Through multiple convolutions, attention fusions, and downsampling operations in each stage, multi-scale features with rich semantic information are finally extracted. In the final stage, the classification model performs global average pooling (GAP) on the image feature map to reduce its dimensionality to [1, 768], and outputs the classification result through a fully connected layer.

[0036] The ConvNeXt module consists of a series of interconnected deep convolutions, layer normalization, convolutions, activations, convolutions, layer scaling, and regularizations; and the final output of the ConvNeXt module is the sum of the input image feature map from the deep convolutions and the regularization output from the ConvNeXt module.

[0037] Example 2 Based on Example 1, this example further discloses the following detailed technical content: The two-stage training and reasoning methods for classification are attached. Figure 2 As shown, the training phase can be divided into five main stages: orthophotos, mosaicking, and manual quality vector detection; manual review; dataset construction; data augmentation; and two-stage model training. The inference phase can be divided into four main nodes: orthophoto and mosaic lines of a certain region; data segmentation; model determination of whether misalignment exists; and misalignment vector generation. The specific two-stage model training steps are as follows: First, the dataset needs to be constructed. Manual quality control vectors are used to identify problematic areas in the orthophotos. Simultaneously, mosaic line vectors are used to stitch and process these areas to generate high-quality orthophotos. Misaligned image regions and repaired, non-misaligned image regions are extracted from the orthophotos, and then cropped and saved separately. Positive samples labeled as misaligned are assigned a label of 1; negative samples labeled as non-misaligned are assigned a label of 0. After sample collection, the images and labels are manually checked for matching, and samples that do not meet the correspondence are removed from the dataset. Since the number of negative samples is significantly greater than positive samples, negative samples are randomly sampled to reduce redundancy, ultimately controlling the positive-to-negative sample ratio in the dataset to approximately 1:100. Subsequently, the image sizes in the dataset are normalized. For images with a width or height less than 224, resample the shorter side to 224 and adjust the longer side proportionally to the corresponding size; for images with a width or height between 500 and 1000, resample the longer side to 500 and adjust the shorter side proportionally to the corresponding size; for images with a width or height exceeding 1000, remove them from the dataset.

[0038] Next, the first stage of training for the classification network is performed. Since the ratio of positive to negative samples in the dataset is approximately 1:100, indicating a significant class imbalance, negative samples are randomly sampled again during the first stage of training to ensure the ratio of positive to negative samples is adjusted to approximately 1:1. Simultaneously, to improve the model's generalization ability, data augmentation methods are introduced, including random horizontal flipping and random scaling / cropping. The AdamW optimizer is selected, and the cross-entropy loss function is used.

[0039] Finally, the second stage of model training is performed. After loading the model weights from the first stage of training as initial weights, the model is retrained using all positive and negative samples. During training, data augmentation methods such as random horizontal flipping and random scaling / cropping are still introduced to improve the model's robustness. The optimizer and loss function remain consistent with the first stage: the AdamW optimizer and the cross-entropy loss function.

[0040] This embodiment constructs the dataset according to the method described above. Specifically, 60,000 positive samples and 16 million negative samples were initially collected. After data normalization and negative sample resampling, a standardized dataset containing 57,000 positive samples and 4.96 million negative samples was finally formed. In the first stage of training, the number of positive samples remained at 57,000, while negative samples were randomly sampled, selecting 72,000 negative samples. The dataset was divided into training and test sets in an 8:2 ratio. In the second stage of training, all positive and negative samples were used, and the training and test sets were also divided in an 8:2 ratio. To improve the generalization ability of the model, data augmentation methods were used in both stages of training, including random horizontal flipping and random scaling and pruning. The AdamW training optimizer was selected, the cross-entropy loss function was used, and the maximum number of training epochs was set to 100.

[0041] The reasoning steps are as follows: First, the mosaic line region is cropped. The input consists of a mosaic line and an orthophoto of a given region. Using the mosaic line as the center, the region is expanded to the left and right by a specified width to extract the regions containing the mosaic line features. These regions are then cropped into 224×224 image patches. To avoid missing crucial information at the cropping boundaries, a 20-30 pixel overlap is set between adjacent image patches during cropping. The cropped image patches are saved as TIFF files to prepare for classification model inference.

[0042] Then, inference and misalignment determination are performed. All cropped image patches are used as input to the classification model, and are fed into the pre-trained classification model in batches. The classification model first uses a Fourier edge detection module to extract edge features of each image, and then combines a dynamic edge attention module in the backbone network to extract multi-scale image features of the input image layer by layer. Through the final classification layer of the classification model, it is determined whether each image patch has mosaic misalignment. For misalignment determination, the classification model returns a binary classification result, representing "misalignment exists" and "no misalignment", and generates a corresponding judgment label for each image patch.

[0043] Finally, misaligned regions are merged and vectors are generated. After obtaining the inference results of all image patches, the areas of image patches marked as "misaligned" are extracted, and the vectors of these misaligned regions are merged based on the geographic coordinates of the image patches to generate a complete vectorized result representing all misaligned areas within the survey area. Ultimately, the vectors of these misaligned regions are saved for subsequent mosaic line repair or quality assessment.

[0044] To verify the effectiveness of the classification model designed in Example 1, several different types of models were compared under the same experimental settings. Table 1 shows the classification performance of different types of models on the test set, including the classification model designed in this example, the reference model ConvNeXt, and the ResNet-18 and VGG models used in related studies.

[0045] Table 1. Classification performance of different models on the test set

[0046] To verify the effectiveness of the two-stage training strategy designed in this embodiment, the classification performance of the classification model under the two-stage training strategy and the one-stage training strategy was compared, as shown in Table 2.

[0047] Table 2. Classification performance of different training methods for classification models

[0048] To verify the effectiveness of the method proposed in this embodiment, it was applied to the quality inspection process of an orthophoto production project. Examples of some misalignments detected are shown below. Figure 3 As shown.

[0049] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for detecting mosaic misalignment in remote sensing images, characterized in that, include: Training: Acquire the dataset and construct the training sample set; A classification model is pre-built based on the improved classification network, and the classification model is trained using a two-stage training method and a training sample set until the target classification model is obtained after training. Inference: Obtain the mosaic lines and orthophotos of a certain region of the image to be detected, extract the two sides containing the mosaic line features, and crop them according to the preset size to obtain several image blocks; All image patches are input into the target classification model, and the output is a misalignment detection result describing whether there is a misalignment in the image to be detected. The improved classification network is as follows: ConvNeXt is used as the base network, and a Fourier edge detection module that synchronously processes the input image of the model and four pre-built edge preprocessing modules connected to the Fourier edge detection module are further introduced. Furthermore, an edge attention module connected to the corresponding edge preprocessing module is added at the input of each of the four stages of the ConvNeXt backbone network; The network structure of the edge attention module consists of two parts: the first part is a sequentially connected edge feature-convolution-batch normalization-convolution-activation; The second part is the intermediate feature; Furthermore, the intermediate features are multiplied by the activation of the first part to obtain a product, and the intermediate features are then summed with the product.

2. The remote sensing image mosaic misalignment detection method according to claim 1, characterized in that, The classification model is trained using a two-stage training method and a training sample set, specifically including: The first stage of training involves randomly sampling negative samples from the dataset to ensure a 1:1 ratio of positive to negative samples during the first stage of training, and then augmenting the data using data augmentation methods. The classification model is then trained using the training sample set obtained from the processed dataset, and further optimized using a loss function and a training optimizer to obtain the first model. The second stage of training involves obtaining the model weights of the first model as initial weights, then retraining the classification model using all positive and negative samples in the dataset. During the training process, data augmentation methods, loss functions, and training optimizers are also used, along with training samples obtained from the dataset, to finally obtain the target classification model.

3. The method for detecting misalignment in remote sensing image mosaicking according to claim 1, characterized in that, The network structure of the edge preprocessing module is a sequentially connected convolution-batch normalization-activation-pooling.

4. The method for detecting misalignment in remote sensing image mosaicking according to claim 1, characterized in that, Obtaining the dataset and constructing the training sample set specifically includes: The misaligned image regions and the repaired, non-misaligned image regions are extracted from the pre-acquired orthophotos, and then cropped, labeled, and saved to a pre-established empty set to obtain the dataset. Remove images from the dataset whose labels do not correspond to whether a misalignment has occurred; Negative samples in the dataset are randomly sampled to control the ratio of positive to negative samples; images that are misaligned in the dataset are considered positive samples, while images that are not misaligned are considered negative samples. The image sizes in the dataset are normalized, and images exceeding the size threshold are removed, ultimately yielding the dataset required by the classification model.

5. The remote sensing image mosaic misalignment detection method according to claim 4, characterized in that, Negative samples in the dataset are randomly sampled to control the ratio of positive to negative samples in the dataset to be 1:100.

Images