Remote sensing image geometric distortion detection method combining spatial domain and frequency domain feature learning

Abstract

This invention discloses a method for detecting geometric distortion in remote sensing images based on joint spatial and frequency domain feature learning. The method involves acquiring the original remote sensing image and the original DEM (Digital Elevation Model). A modified DEM is generated by overlaying human errors onto the original DEM, and the original remote sensing image is orthorectified to generate a geometrically distorted remote sensing image. Pixel-level labels are generated based on the modified location information. Frequency domain features are extracted from the original remote sensing image using a Gaussian bandpass filter to extract frequency domain features at different scales and reconstruct a single-frequency band feature image. This image is then fused with the spatial domain bands of the original remote sensing image to generate a fused image. A semantic segmentation network is constructed and adapted. A geometric distortion detection model is trained using the fused image and pixel-level labels. The remote sensing image to be detected is processed in the same way and input into the model, outputting the geometric distortion region detection results. This invention eliminates the need for manual annotation and external reference data, enabling high-precision and high-efficiency batch detection of geometric distortion in remote sensing images.

Description

Technical Field

[0001] This invention relates to the fields of remote sensing image processing and computer vision technology, specifically to a method for detecting geometric distortion in remote sensing images by jointly learning spatial and frequency domain features. Background Technology

[0002] Remote sensing imagery is a core data source for geographic information acquisition, widely used in fields such as land surveys, urban planning, disaster emergency monitoring, resource exploration, and national defense. Its geometric accuracy directly determines the reliability and application value of geographic information products. Geometric distortion is one of the most common and significant quality problems in remote sensing image production. It refers to the systematic or local deviation between the relative position, shape, size, and orientation of ground features in the image and the actual geographic spatial features, caused by a combination of factors including sensor platform attitude fluctuations, terrain undulations, atmospheric refraction effects, Earth's rotation, and projection transformation errors. To ensure the effectiveness of subsequent applications of remote sensing imagery, geometric distortion detection has become an indispensable and crucial step in the factory quality inspection of remote sensing imagery, verification of orthorectification effects, and quality control processes for geographic information products.

[0003] Currently, the technologies used in the industry for detecting geometric distortion in remote sensing images can be mainly divided into three categories: manual visual interpretation methods, traditional mathematical model-based recognition methods, and deep learning-based intelligent recognition methods.

[0004] The manual visual interpretation method is the earliest developed and most widely used traditional detection method. It involves technicians with professional geographical knowledge comparing remote sensing images with high-precision reference geographic data frame by frame to analyze abnormal deviations in the outline and position of ground features to determine distorted areas. This method is simple in principle and does not require complex technical support, but it has inherent defects such as strong subjectivity and difficulty in unifying detection standards. Moreover, the detection efficiency is extremely low. Manual detection of a single large-size high-resolution remote sensing image often takes several hours or even days, resulting in high labor costs. It cannot meet the demand for batch detection of massive images brought about by the rapid development of the current satellite remote sensing and UAV remote sensing industries.

[0005] Traditional geometric distortion identification methods based on mathematical models mainly rely on photogrammetry principles and digital image processing technology. They describe and fit the distortion patterns in images by establishing various geometric transformation models. Some methods also require the use of external auxiliary data such as ground control points and digital elevation models (DEMs) for correction and detection. These methods can achieve certain detection results in specific scenarios and with known distortion types. However, they rely too much on the quality and distribution density of external auxiliary data. When the number of control points is insufficient, their distribution is uneven, or the DEM accuracy is low, the detection accuracy will drop significantly. At the same time, a large number of model parameters need to be manually adjusted according to the image sensor type, resolution, and terrain conditions. They have poor generalization ability and are difficult to promote and apply across scenarios and sensors.

[0006] In recent years, with the breakthroughs in deep learning technology in the field of computer vision, deep learning-based methods for detecting geometric distortion in remote sensing images have gradually become a research hotspot. These methods, through end-to-end feature learning, can automatically extract complex nonlinear distortion features from images, showing improvements in both detection accuracy and automation compared to traditional methods. However, the performance of deep learning methods is highly dependent on large-scale, high-quality pixel-level labeled training samples. The labeling of geometric distortion in remote sensing images is highly specialized and complex, requiring professionals to perform pixel-by-pixel manual labeling using high-precision georeferenced data. The labeling cost is several times or even tens of times that of ordinary image semantic segmentation tasks. Furthermore, the number of geometric distortion samples in real-world scenarios is scarce and their types are diverse, resulting in a lack of publicly available large-scale standardized geometric distortion labeling datasets in the industry.

[0007] Prior art 1: CN118365611A, a method, apparatus and electronic equipment for quantitative quality inspection of internal geometric distortion of remote sensing images;

[0008] Existing technology 2: CN114429448B, Method for detecting internal geometric distortion of images in uncontrolled adjustment of large-area optical satellite networks;

[0009] Prior art 3: CN121458533A, Distortion sensing method and device based on super-resolution quality of remote sensing images.

[0010] Existing technologies 1-3 cannot overcome the above-mentioned technical shortcomings. Therefore, there is an urgent need in this field for a new method for detecting geometric distortion in remote sensing images to meet the pressing need for batch quality inspection of massive amounts of remote sensing images. Summary of the Invention

[0011] To alleviate or partially alleviate the above-mentioned technical problems, the solution of the present invention is as follows:

[0012] A method for detecting geometric distortion in remote sensing images by jointly learning spatial and frequency domain features includes the following steps:

[0013] Step S1: Acquire the original remote sensing image and the corresponding original digital elevation model (DEM), and superimpose human error onto the original DEM to generate a modified DEM. Generate pixel-level labels based on the location information modified by superimposing the human error. Perform orthorectification on the original remote sensing image using the modified DEM to generate a geometrically distorted remote sensing image, and the geometrically distorted remote sensing image spatially corresponds to the pixel-level labels.

[0014] Step S2: Extract frequency domain features from the original remote sensing image: Extract frequency domain features at different scales using a Gaussian bandpass filter with a preset frequency range, and reconstruct each frequency domain feature into a single-frequency band feature image; perform channel fusion between the single-frequency band feature image and the spatial domain band of the original remote sensing image to generate a fused image containing spatial domain information and frequency domain information.

[0015] Step S3: Construct a semantic segmentation network: Configure the number of channels in the input layer of the semantic segmentation network to be the same as the number of bands in the fused image, and use a frequency band independent normalization strategy to normalize the bands in the fused image that belong to the frequency domain features and the bands that belong to the spatial domain features respectively; train the semantic segmentation network using a sample set composed of the fused image and the pixel-level labels to obtain a geometric distortion detection model;

[0016] Step S4: Perform the same frequency domain feature extraction and channel fusion operations as in Step S2 on the remote sensing image to be detected, generate the fused image to be detected, input the fused image to be detected into the geometric distortion detection model, and output the detection results of the geometric distortion region in the remote sensing image to be detected.

[0017] In a preferred embodiment, the types of human error superimposed in step S1 include one or more of the following: overall offset, random noise, local bulge, local depression, and topographic fault.

[0018] In a preferred embodiment, the Gaussian bandpass filter with a preset frequency range in step S2 corresponds to five frequency ranges: low frequency, low-mid frequency, mid frequency, mid-high frequency, and high frequency.

[0019] The frequency range of the low-frequency Gaussian bandpass filter is 0.001-0.01 cycles per pixel;

[0020] The frequency range of the low-to-mid frequency Gaussian bandpass filter is 0.01-0.05 cycles per pixel;

[0021] The frequency range of the intermediate frequency Gaussian bandpass filter is 0.05-0.15 cycles per pixel;

[0022] The frequency range of the mid-to-high frequency Gaussian bandpass filter is 0.15-0.3 cycles per pixel;

[0023] The frequency range of the high-frequency Gaussian bandpass filter is 0.3-0.5 cycles per pixel.

[0024] In a preferred embodiment, the fused image generated in step S2 is an 8-band fused image, wherein the 8 bands are as follows:

[0025] The R-band, G-band, and B-band of the original remote sensing image; and,

[0026] The five single-band feature images, reconstructed from the features of five frequency bands (low frequency, low-mid frequency, mid frequency, mid-high frequency, and high frequency), correspond to the respective wavebands.

[0027] In a preferred embodiment, the frequency band independent normalization strategy in step S3 is specifically as follows:

[0028] For each band corresponding to a single-band feature image in the fused image, the pixel mean and standard deviation are calculated on the entire training set and Z-score normalization is performed independently.

[0029] The three spatial domain bands in the fused image—R-band, G-band, and B-band—are normalized using the standard mean and standard deviation of the ImageNet dataset.

[0030] In a preferred embodiment, when constructing the sample set in step S3, a sliding window is used to simultaneously crop the fused image and the corresponding pixel-level labels, and the cropped samples are screened to discard pure background samples with a distorted pixel ratio of less than a preset threshold.

[0031] In one preferred embodiment, data augmentation is performed on the selected samples before training the semantic segmentation network;

[0032] The data augmentation operation includes one or more of random rotation, random scaling, random color jitter, and adding Gaussian noise, and the data augmentation operation is applied to both the fused image sample and the corresponding label sample.

[0033] In a preferred embodiment, after outputting the geometric distortion region detection result in step S4, the method further includes:

[0034] The raster results output by the geometric distortion detection model are vectorized to obtain the geometric distortion vector range.

[0035] In one preferred embodiment, after obtaining the range of the geometric distortion vector, a post-processing step is further included:

[0036] Based on a preset area threshold, regions within the geometric distortion vector range whose area is smaller than the preset area threshold are filtered out, and voids within the geometric distortion vector range whose internal area is smaller than the preset area threshold are filled.

[0037] In a preferred embodiment, step S3 uses "DeepLabV3+" as the basic architecture of the semantic segmentation network.

[0038] The technical solution of this invention has one or more of the following beneficial technical effects:

[0039] (1) By introducing controllable errors into the original DEM and combining them with orthorectification, controllable generation of geometrically distorted remote sensing images and automatic acquisition of pixel-level labels can be achieved, solving the problem of sample scarcity. Large-scale training sets can be constructed without manual annotation, greatly reducing the model training cost.

[0040] (2) Frequency domain features corresponding to different distortion types are extracted by five sets of multi-scale Gaussian bandpass filters and fused with the original spatial domain RGB image, which significantly enhances the model’s ability to perceive various geometric distortions and effectively improves the accuracy, recall and boundary integrity of distortion detection.

[0041] (3) By adapting the input layer of the semantic segmentation network and adopting the frequency band independent normalization strategy, referenceless geometric distortion detection can be achieved. It does not rely on external data such as ground control points, has strong generalization ability, and can meet the application needs of batch quality inspection of massive remote sensing images.

[0042] Furthermore, other beneficial effects of the present invention will be mentioned in the specific embodiments. Attached Figure Description

[0043] Figure 1 A schematic diagram of the overall process of the remote sensing image geometric distortion detection method based on joint spatial domain and frequency domain feature learning provided in an embodiment of the present invention;

[0044] Figure 2 Example image of the original remote sensing image provided in the embodiment of the present invention;

[0045] Figure 3 This is an example diagram of the original digital elevation model provided in an embodiment of the present invention;

[0046] Figure 4 Example image of a remote sensing image with reorthorectified geometric distortion provided in an embodiment of the present invention;

[0047] Figure 5 This is a modified DEM example image provided in an embodiment of the present invention;

[0048] Figure 6 This is an example diagram of DEM modification information recording provided in an embodiment of the present invention;

[0049] Figure 7 Examples of five single-band feature images (low frequency, mid-low frequency, mid frequency, mid-high frequency, and high frequency) provided in embodiments of the present invention;

[0050] Figure 8 An example image of an 8-band fused image and its layer attribute information provided in an embodiment of the present invention;

[0051] Figure 9 This invention provides examples of geometrically distorted remote sensing images and corresponding automatically generated label instances.

[0052] Figure 10 This is an example image of the cropped and fused image and its corresponding label in an embodiment of the present invention;

[0053] Figure 11 These are examples of various data augmentation effects used in embodiments of the present invention;

[0054] Figure 12 This is an example image of the remote sensing image to be detected and the preliminary detection results in an embodiment of the present invention;

[0055] Figure 13 This diagram illustrates the core steps of the remote sensing image geometric distortion detection method based on combined spatial and frequency domain feature learning of this invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0057] To facilitate a clear description of the technical solutions in the embodiments of the present invention, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, but are only used to distinguish different technical features.

[0058] The term "human error" refers to errors intentionally superimposed on the original digital elevation model (DEM) to simulate real geometric distortions. These errors include overall offset, random noise, local bulges, local depressions, and topographic faults. By controlling the type, location, and magnitude of human errors, geometrically distorted remote sensing images can be generated in a controllable manner.

[0059] The term "geometrically distorted remote sensing image" refers to a remote sensing image that has undergone orthorectification of an original distortion-free remote sensing image using a modified digital elevation model, thus artificially introducing geometric distortion. This image is used to pair with automatically generated pixel-level labels to form training samples.

[0060] The term "single-band feature image" refers to a spatial domain image obtained by filtering the original remote sensing image through a Gaussian bandpass filter within a specific frequency range and then reconstructing it using inverse Fourier transform. This image contains only the geometric distortion feature information of the corresponding frequency band.

[0061] The term "fusion image to be detected" refers to a fusion image (in one embodiment, an 8-band fusion image) generated after performing frequency domain feature extraction and channel fusion operations on the remote sensing image to be detected. This image serves as input to a trained geometric distortion detection model for inferring and outputting distorted regions.

[0062] This invention discloses a remote sensing image geometric distortion detection method based on joint spatial and frequency domain feature learning. This method artificially simulates geometric distortion to automatically generate pixel-level high-precision labels, solving the problems of difficulty in obtaining real distortion samples and high labeling costs. At the same time, this method deeply fuses the spatial domain information of the original remote sensing image with the frequency domain features extracted by multi-band filtering to construct multi-band input data, thereby enhancing the deep learning network's ability to perceive various geometric distortion features, and ultimately achieving high-precision and highly automated remote sensing image geometric distortion detection.

[0063] Figure 1 This is a schematic diagram illustrating the overall process of the remote sensing image geometric distortion detection method based on joint spatial and frequency domain feature learning provided in an embodiment of the present invention. Figure 1As shown, the process begins with artificial simulation and automatic annotation of image geometric distortion. A modified DEM is generated by introducing controllable errors into the original DEM. The modified DEM is then used to orthorectify the original remote sensing image, resulting in a geometrically distorted remote sensing image. Simultaneously, the modification location information of the DEM is automatically recorded for subsequent pixel-level label generation. Next, frequency domain features are extracted from the original remote sensing image. A two-dimensional Fourier transform converts the image from the spatial domain to the frequency domain. Five pre-designed Gaussian bandpass filters are used to separate components of different frequencies. Finally, an inverse Fourier transform reconstructs five single-band images. Feature images are generated; then, five single-band feature images are fused with the original RGB image to generate an 8-band fused image containing spatial and frequency domain information; the fused image and corresponding labels are then cropped into standard samples and data augmentation is performed to construct an image geometric distortion sample set; the semantic segmentation network is then adapted in terms of input layer and normalization strategy, and a geometric distortion detection model is trained using the sample set; finally, the remote sensing image to be detected is processed into a fused image to be detected, i.e., the 8-band fused image is input into the trained model, and the final geometric distortion region is obtained after inference and result post-processing.

[0064] In one specific embodiment of the present invention, a distortion-free high-resolution remote sensing image is selected as the original remote sensing image. Its spatial resolution is 0.5 meters, its size is 2250×2101 pixels, and it includes three bands: red (R), green (G), and blue (B). At the same time, an original DEM with the same geographical area as the image is acquired, and the grid spacing of the original DEM is consistent with the image resolution. Figure 2 An example image of the original remote sensing image provided in an embodiment of the present invention. Figure 3 An example diagram of the original digital elevation model provided in an embodiment of the present invention.

[0065] First, the image geometric distortion generation and automatic label generation steps are performed. Various types of errors are manually defined on the original DEM, including overall offset, random noise, local bulges, local depressions, and topographic faults. In this embodiment, two typical errors are mainly created: local bulges and local depressions. A modified DEM is generated, and the location information of the modified DEM is recorded. This information will be directly used for subsequent automatic label generation. The original remote sensing image is then orthorectified using the modified DEM. Due to the manually introduced errors in the DEM, geometric distortions corresponding to the DEM errors will occur at the corresponding locations during orthorectification, resulting in a remote sensing image containing a large amount of geometric distortion.

[0066] Figure 4 An example image of a remote sensing image with reorthorectified geometric distortion provided in an embodiment of the present invention. Figure 5 This is a modified DEM example image provided in an embodiment of the present invention. Figure 6This is an example diagram of DEM modification information recording provided in an embodiment of the present invention.

[0067] The core principle of orthorectification is to eliminate the image projection difference caused by terrain undulations based on the terrain elevation information provided by the DEM. Therefore, the accuracy of the DEM directly determines the geometric accuracy of the orthorectification result. By manually controlling the error position and magnitude of the DEM, the position, type and degree of the generated geometric distortion can be precisely controlled, and the controllable generation of geometrically distorted remote sensing images can be achieved.

[0068] Then, a multi-band Gaussian bandpass filter was designed. A two-dimensional coordinate grid of the same size as the original remote sensing image was created. The Euclidean distance from each pixel to the center point of the coordinate grid was calculated, and the distance values ​​were normalized to the range of 0 to 0.5 to obtain a normalized frequency coordinate matrix. Based on this normalized frequency coordinate matrix, five groups of non-overlapping Gaussian bandpass filters with increasing center frequencies were constructed. Each group of filters corresponds to a specific frequency range and a typical type of geometric distortion. The specific parameter settings are as follows:

[0069] The low-frequency filter has a frequency range of 0.001–0.01 cycles per pixel and is used to capture global translation and rotation distortions in images.

[0070] The low-to-mid frequency filter has a frequency range of 0.01–0.05 cycles per pixel and is used to capture distortions caused by mesoscale terrain undulations.

[0071] The intermediate frequency filter has a frequency range of 0.05–0.15 cycles per pixel and is used to capture systematic distortions caused by the sensor's own characteristics.

[0072] The mid-to-high frequency filter has a frequency range of 0.15–0.3 cycles per pixel and is used to capture geometric distortions in local areas.

[0073] The high-frequency filter has a frequency range of 0.3–0.5 cycles per pixel and is used to capture high-frequency noise and edge distortion in images.

[0074] Then, the frequency band components are extracted and single-band feature images are reconstructed. The original remote sensing image is processed into a grayscale image, and a two-dimensional Fourier transform is performed on the grayscale image to convert the image from the spatial domain to the frequency domain, resulting in a spectrum image. Subsequently, the zero-frequency component of the spectrum image is moved to the center of the spectrum to complete the spectrum centering process, which facilitates the application of subsequent filters. The centered spectrum image is then multiplied pixel-by-pixel with the above five sets of Gaussian bandpass filters to filter out irrelevant spectral information outside the corresponding frequency band, thereby separating five independent frequency domain feature components.

[0075] For each frequency domain feature component, an inverse centering operation is first performed to move the zero-frequency component back to its original position. Then, an inverse Fourier transform is performed on the inverse Fourier transform component to convert it from the frequency domain back to the spatial domain. The imaginary part in the inverse Fourier transform result is discarded, and only the real part containing the main image information is retained. Finally, all real part data is normalized to obtain 5 reconstructed single-band feature images.

[0076] Figure 7 Examples of five single-band feature images (low frequency, low-mid frequency, mid frequency, mid-high frequency, and high frequency) provided in embodiments of the present invention are shown. Figure 7 Part (a) is a low-frequency feature map; Figure 7 Part (b) is a mid-to-low frequency feature map; Figure 7 Part (c) is the mid-frequency characteristic map; Figure 7 Part (d) is a mid-to-high frequency feature map; Figure 7 Part (e) is a high-frequency feature map.

[0077] Then, the spatial domain and frequency domain feature fusion step is performed. The five single-band feature images obtained above are taken as five independent bands and stitched together with the R, G, and B bands of the original remote sensing image in a specified order. The band order is as follows: R band, G band, B band, low-frequency feature map band, mid-low frequency feature map band, mid-frequency feature map band, mid-high frequency feature map band, and high-frequency feature map band. Finally, a fused image containing eight bands is generated.

[0078] Figure 8 This is an example image of an 8-band fused image and its layer attribute information provided in an embodiment of the present invention. Figure 8 Part (a) is an 8-band fused image; Figure 8 Part (b) shows the layer attribute information of the 8-band fused image. The "Raster Information" column of the attributes in the figure clearly marks the key parameters of the image: the number of columns and rows is 2250×2101, which is completely consistent with the size of the original remote sensing image; the number of bands is 8, corresponding to the original red (R), green (G), and blue (B) spatial domain bands, as well as the low frequency, mid-low frequency, mid frequency, mid-high frequency, and high frequency frequency characteristic bands; the pixel size is 1×1, the uncompressed size is 36.07MB, the storage format is TIFF, the source type is general, and the pixel type is 8-bit unsigned integer.

[0079] This multi-band fusion method can effectively combine spatial domain information such as the shape, texture, and color of ground features contained in the original image with geometric distortion features of different scales extracted in the frequency domain, providing richer feature inputs for subsequent deep learning models and improving the model's ability to recognize various geometric distortions.

[0080] Next, image geometric distortion labels are created. Using the location information of DEM modifications recorded in the previous steps, pixel-level labels corresponding to the geometrically distorted remote sensing images are automatically generated. The pixel attribute value corresponding to the DEM-modified area in the pixel-level label is assigned a value of 1, indicating that the area is a geometrically distorted region. The pixel attribute value of the remaining background areas is assigned a value of 0, indicating that the area is a normal region. The generated label vector data is rasterized into single-band pixel-level labels that are strictly aligned with the fused image and have the same resolution, ensuring a one-to-one correspondence between the pixel-level labels and every pixel in the fused image.

[0081] Figure 9 This is an example of a geometrically distorted remote sensing image and its corresponding automatically generated label. Figure 9 Part (a) is a remote sensing image with geometric distortion; Figure 9 Part (b) is the automatically generated label corresponding to the geometrically distorted remote sensing image.

[0082] This label generation method is entirely based on DEM modification records and is completed automatically without any manual annotation. It not only significantly reduces the labor cost of label production, but also ensures the pixel-level accuracy of the labels, solving the problems of low accuracy and poor consistency of traditional manual annotation.

[0083] Then, an image geometric distortion sample set was created. The 8-band fused image and the corresponding single-band pixel-level label were simultaneously cropped in a sliding window manner. The size of the cropping window was 512×512 pixels, and the sliding step size was set to 128 pixels. By setting an overlapping sliding step size, the loss of image edge information can be avoided, while increasing the number of samples.

[0084] In one embodiment, after cropping, all samples are screened, and pure background samples with a distorted pixel ratio of less than 2% are discarded to ensure the balance of training samples and avoid the problem of the model being biased towards predicting the background during training.

[0085] Subsequently, data augmentation operations are performed on the selected samples to improve the model's generalization ability. Data augmentation methods include random rotation, random scaling, and random color jitter. In this embodiment, the specific data augmentation methods include horizontal flipping, vertical flipping, random rotation ±15 degrees, random color jitter, and adding Gaussian noise. All data augmentation operations are applied simultaneously to the fused image samples and the corresponding label samples to ensure the consistency between the images and the labels.

[0086] Figure 10 This is an example image of the cropped and fused image and its corresponding label in an embodiment of the present invention. Figure 10 Part (a) is the cropped and merged image; Figure 10 Part (b) is the label corresponding to the cropped and merged image.

[0087] Figure 11 These are examples of various data augmentation effects used in embodiments of the present invention. Figure 11 Part (a) is an example diagram of the vertical flipping data enhancement effect; Figure 11 Part (b) is an example of the color jitter data enhancement effect; Figure 11 Part (c) shows an example of Gaussian noise data enhancement effect; Figure 11 Part (d) is an example diagram of the data augmentation effect of horizontal flipping; Figure 11 Part (e) is an example diagram of random rotation data augmentation effect.

[0088] Then, the image geometric distortion region detection model is trained. The model training process is as follows:

[0089] Two key modifications were made to address the 8-band input and multi-band feature characteristics of this invention. The first modification was an input layer adaptation. The semantic segmentation network was modified to adapt the input layer by changing the original number of input channels to 8 to accommodate the input requirements of 8-band fused images. The second modification was a frequency-band independent normalization strategy. Since the dynamic range of pixel values ​​in single-band feature images varies significantly across different frequency bands, a uniform normalization method would result in the loss of effective features in some frequency bands. Therefore, the pixel mean and standard deviation of each of the five frequency domain feature images were calculated on the entire training set, and then Z-score normalization was performed independently for each frequency band image. For the original R, G, and B bands, the standard mean and standard deviation from the ImageNet dataset were used for normalization. Finally, the image geometric distortion sample set was input into the modified network for training, resulting in a trained geometric distortion detection model.

[0090] This embodiment uses DeepLabV3+ as the basic network architecture. First, the number of input channels in the first convolutional layer of the DeepLabV3+ network is changed from the default 3 to 8. Then, the pixel mean and standard deviation of the five frequency domain feature images are calculated on the entire training set. Z-score normalization is then performed independently on the images of each frequency band. For the original R, G, and B bands, the standard mean and standard deviation of the ImageNet dataset are used for normalization. During model training, the Adaptive Moment Estimation (Adam) optimizer is used with an initial learning rate of 0.0001 and a batch size of 16. The Sørensen-Dice coefficient (Dice coefficient) is monitored on the validation set as an evaluation metric for model performance. When the Dice coefficient on the validation set no longer increases after 10 consecutive training epochs, an early stopping mechanism is triggered to stop training, ultimately resulting in a trained geometric distortion detection model.

[0091] The image geometric distortion region inference step is performed. For the remote sensing image to be detected, the same process as described above is followed first to extract frequency domain features and fuse spatial and frequency domain features to generate a corresponding 8-band fused image. The generated 8-band fused image is then input into the trained geometric distortion detection model. The model extracts multi-scale features through an encoder and restores the resolution of the feature map through a decoder, finally outputting a pixel-level distortion raster result with the same size as the input image. Regions with a pixel value of 1 represent detected geometric distortion regions, and regions with a pixel value of 0 represent normal regions. The raster result output by the model is then vectorized to obtain the preliminary geometric distortion vector range. Figure 12 This is an example image of the remote sensing image to be detected and the preliminary detection results in this embodiment. Figure 12 Part (a) is the remote sensing image to be detected; Figure 12 Part (b) is an example of the preliminary detection results corresponding to the remote sensing image to be detected.

[0092] Finally, the post-processing steps involve a two-step optimization of the initially obtained geometric distortion vector range to improve the accuracy and completeness of the detection results. The first step is filtering for extremely small areas. An area threshold is set to filter out extremely small areas within the geometric distortion vector range that are smaller than this threshold. These areas are typically noise generated by false detections in the model. The second step is filling in the tiny voids within the distorted areas to ensure the integrity of the region's outline, ultimately outputting accurate geometric distortion detection results. In this embodiment, the area threshold is set to 400 m². This threshold can be flexibly adjusted according to the accuracy requirements of the actual application scenario to fill in tiny voids within the geometric distortion vector region with an area smaller than 400 m².

[0093] Figure 13 This diagram illustrates the core steps of the remote sensing image geometric distortion detection method based on combined spatial and frequency domain feature learning, as described in this invention. Figure 13 As shown in the figure, the four core steps of the present invention are clearly illustrated: Step S1 is the automatic generation of geometrically distorted remote sensing images and pixel-level labels; Step S2 is the extraction and fusion of spatial and frequency domain features; Step S3 is the construction and training of the geometric distortion detection model; and Step S4 is the geometric distortion inference and result output of the remote sensing image to be detected.

[0094] This invention achieves controllable generation of geometrically distorted remote sensing images and automatic acquisition of pixel-level labels by combining DEM error introduction with orthorectification, completely solving the core problem of the lack of high-quality training samples in the field of geometric distortion detection of remote sensing images. At the same time, by extracting multi-band frequency domain features and fusing spatial domain features, it significantly improves the deep learning model's ability to perceive geometric distortions of different scales and types. Compared with existing technologies, it has higher detection accuracy, recall rate and boundary integrity, and does not rely on external control points and reference data. It has strong generalization ability and can meet the urgent need for batch quality inspection of massive amounts of remote sensing images.

[0095] To better illustrate the present invention, numerous specific details have been set forth in the detailed embodiments described above. Those skilled in the art should understand that the present invention can be practiced even without certain specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art have not been described in detail in order to highlight the spirit of the present invention.

[0096] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting geometric distortion in remote sensing images based on joint spatial and frequency domain feature learning, characterized in that, Includes the following steps: Step S1: Acquire the original remote sensing image and the corresponding original digital elevation model (DEM), and superimpose human error onto the original DEM to generate a modified DEM. Generate pixel-level labels based on the location information modified by superimposing the human error. Perform orthorectification on the original remote sensing image using the modified DEM to generate a geometrically distorted remote sensing image, and the geometrically distorted remote sensing image spatially corresponds to the pixel-level labels. Step S2: Extract frequency domain features from the original remote sensing image: Extract frequency domain features at different scales using a Gaussian bandpass filter with a preset frequency range, and reconstruct each frequency domain feature into a single-frequency band feature image; perform channel fusion between the single-frequency band feature image and the spatial domain band of the original remote sensing image to generate a fused image containing spatial domain information and frequency domain information. Step S3: Construct a semantic segmentation network: Configure the number of channels in the input layer of the semantic segmentation network to be the same as the number of bands in the fused image, and use a frequency band independent normalization strategy to normalize the bands in the fused image that belong to the frequency domain features and the bands that belong to the spatial domain features respectively; train the semantic segmentation network using a sample set composed of the fused image and the pixel-level labels to obtain a geometric distortion detection model; Step S4: Perform the same frequency domain feature extraction and channel fusion operations as in Step S2 on the remote sensing image to be detected, generate the fused image to be detected, input the fused image to be detected into the geometric distortion detection model, and output the detection results of the geometric distortion region in the remote sensing image to be detected.

2. The remote sensing image geometric distortion detection method based on joint spatial domain and frequency domain feature learning according to claim 1, characterized in that: The types of human error superimposed in step S1 include one or more of the following: overall offset, random noise, local bulge, local depression, and topographic fault.

3. The remote sensing image geometric distortion detection method based on joint spatial domain and frequency domain feature learning according to claim 1, characterized in that, The Gaussian bandpass filter with preset frequency range in step S2 corresponds to five frequency ranges: low frequency, low-mid frequency, mid frequency, mid-high frequency, and high frequency. The frequency range of the low-frequency Gaussian bandpass filter is 0.001-0.01 cycles per pixel; The frequency range of the low-to-mid frequency Gaussian bandpass filter is 0.01-0.05 cycles per pixel; The frequency range of the intermediate frequency Gaussian bandpass filter is 0.05-0.15 cycles per pixel; The frequency range of the mid-to-high frequency Gaussian bandpass filter is 0.15-0.3 cycles per pixel; The frequency range of the high-frequency Gaussian bandpass filter is 0.3-0.5 cycles per pixel.

4. The remote sensing image geometric distortion detection method based on joint spatial domain and frequency domain feature learning according to claim 3, characterized in that, The fused image generated in step S2 is an 8-band fused image, and the 8 bands are as follows: The R-band, G-band, and B-band of the original remote sensing image; and, The five single-band feature images, reconstructed from the features of five frequency bands (low frequency, low-mid frequency, mid frequency, mid-high frequency, and high frequency), correspond to the respective wavebands.

5. The remote sensing image geometric distortion detection method based on joint spatial domain and frequency domain feature learning according to claim 1, characterized in that, The frequency band independent normalization strategy in step S3 is as follows: For each band corresponding to a single-band feature image in the fused image, the pixel mean and standard deviation are calculated on the entire training set and Z-score normalization is performed independently. The three spatial domain bands in the fused image—R-band, G-band, and B-band—are normalized using the standard mean and standard deviation of the ImageNet dataset.

6. The remote sensing image geometric distortion detection method based on joint spatial domain and frequency domain feature learning according to claim 1, characterized in that: In step S3, when constructing the sample set, a sliding window method is used to simultaneously crop the fused image and the corresponding pixel-level labels, and the cropped samples are screened to discard pure background samples with a distorted pixel ratio of less than a preset threshold.

7. The remote sensing image geometric distortion detection method based on joint spatial domain and frequency domain feature learning according to claim 6, characterized in that: Before training the semantic segmentation network, data augmentation is performed on the selected samples; The data augmentation operation includes one or more of random rotation, random scaling, random color jitter, and adding Gaussian noise, and the data augmentation operation is applied to both the fused image sample and the corresponding label sample.

8. The remote sensing image geometric distortion detection method based on joint spatial domain and frequency domain feature learning according to claim 1, characterized in that: After outputting the geometric distortion region detection results in step S4, the method further includes: The raster results output by the geometric distortion detection model are vectorized to obtain the geometric distortion vector range.

9. The remote sensing image geometric distortion detection method based on joint spatial domain and frequency domain feature learning according to claim 8, characterized in that, After obtaining the range of the geometric distortion vector, post-processing steps are also included: Based on a preset area threshold, regions within the geometric distortion vector range whose area is smaller than the preset area threshold are filtered out, and voids within the geometric distortion vector range whose internal area is smaller than the preset area threshold are filled.

10. The remote sensing image geometric distortion detection method based on joint spatial domain and frequency domain feature learning according to any one of claims 1 to 9, characterized in that: In step S3, "DeepLabV3+" is used as the basic architecture of the semantic segmentation network.

Images