Satellite-borne TDI CCD high-frequency tremor distortion correction method based on progressive learning

Abstract

The invention relates to the technical field of remote sensing satellite image data processing, in particular to a satellite-borne TDI CCD high-frequency tremor distortion correction method based on progressive learning, and the method comprises the following steps: S1, constructing a distortion data set with tremor levels decreasing in sequence: generating the distortion data set with tremor levels decreasing in sequence through a tremor simulation method for training of a neural network; s2, designing a generative adversarial network and a progressive learning strategy: designing the generative adversarial network, starting training from a high-distortion image by using the progressive learning strategy, and gradually introducing a low-distortion image; and S3, distortion correction of the large-scale remote sensing image: carrying out local processing and line-by-line correction on the large-scale distorted remote sensing image according to the characteristic of line-to-line consistency of the TDI camera, and finally reconstructing a complete corrected image. The method has the advantages of being high in precision, automatic and suitable for large-scale remote sensing image processing, and the quality and the application value of the satellite remote sensing image can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing satellite image data processing, and particularly to a method for correcting high-frequency tremor distortion of on-board TDI CCD based on progressive learning. Background Art

[0002] Satellite platform tremor refers to the perturbations caused by factors such as attitude adjustment, pointing control, solar panel adjustment, and periodic movement of on-board moving parts during the on-orbit operation of the satellite, resulting in a small flutter response of the satellite. These tremor perturbations will cause slight displacement of the camera, thereby destroying the synchronization between the photo-generated charge packet and the image movement, affecting the clarity and geometric accuracy of the image. In the TDI CCD imaging system, the impact of tremor on the image quality is particularly significant. TDI CCD extends the exposure time through multi-stage time integration to improve the light flux, sensitivity, and signal-to-noise ratio, thereby improving the performance of the imaging system. However, the tiny displacement caused by tremor may lead to image shift, making the image blurred during the imaging process and seriously affecting the geometric accuracy of high-resolution remote sensing images.

[0003] Currently, the detection and compensation of tremor distortion mainly rely on two methods: one is based on the parallax imaging system, which detects the tremor effect by analyzing the overlapping area of multi-spectral bands or matching adjacent CCD images. However, this method requires high-precision feature extraction and image matching and is difficult to apply to single full-color images. The other method is to use high-frequency attitude sensors, such as angular velocity and angular displacement sensors, to capture the tiny jitter information of the satellite to assist in compensation and image correction. However, these methods usually rely on high-precision sensors and are limited by the satellite hardware configuration. Traditional physical model methods often face problems such as complex parameters and poor adaptability when dealing with high-frequency tremor distortion and are difficult to efficiently correct tiny tremor distortion. In recent years, although deep learning has made certain progress in the field of image deblurring, most methods focus on the correction of images with large distortions, and relatively few studies have been conducted on the correction of tiny distortions. Therefore, it is difficult for the existing technology to achieve high-precision tremor distortion correction. Summary of the Invention

[0004] The present invention provides a method for correcting high-frequency tremor distortion of on-board TDI CCD based on progressive learning, which can effectively suppress the influence brought by satellite platform tremor and can effectively correct the tiny geometric distortion caused by high-frequency tremor to remote sensing images. The present invention has high precision, automation, and broad application prospects.

[0005] The method for correcting high-frequency tremor distortion of on-board TDI CCD based on progressive learning includes the following steps:

[0006] S1, constructing a distorted dataset with decreasing tremor levels: generating a distorted dataset with decreasing tremor levels through a tremor simulation method for neural network training;

[0007] S2, Generative Adversarial Network Design and Progressive Learning Strategy: Design a generative adversarial network whose generator includes a distortion network and a detail enhancement network. Using a progressive learning strategy based on a hierarchical distortion dataset, training begins with highly distorted images and gradually introduces low-distortion images to improve the neural network's ability to perceive and correct small distortions.

[0008] S3, large-scale remote sensing image distortion correction: Based on the inter-row consistency characteristics of the TDI camera, large-scale remote sensing images are locally processed and corrected row by row, and finally a complete corrected image is reconstructed.

[0009] Optionally, the tectonic tremor level dataset in S1 includes:

[0010] S11, collect original images: collect high-quality reference images with significant geometric features from remote sensing image datasets such as DOTA-v1.5 and VEDAI, and natural image datasets such as ImageNet, COCO, and Places2;

[0011] S12, generating distorted images with decreasing tremor levels: applying different degrees of tremor distortion to the reference image, defining tremor levels according to amplitude and amplitude attenuation factor, and generating distorted images with decreasing tremor levels;

[0012] S13, generating a graded distortion dataset: pairing distorted images of different tremor levels with reference images to form a graded distortion image dataset for training, evaluation, and testing;

[0013] S14, Dataset division: Use the natural image dataset as the training set and the remote sensing image dataset as the validation set and test set.

[0014] Optionally, the amplitude attenuation factor is expressed as:

[0015] α=|sin c(Nτf)|;

[0016] Where α is the amplitude attenuation factor.

[0017] Optionally, the distorted image dataset is expressed as:

[0018]

[0019] Among them, L J (i) is the distorted image of tremor level i, and A is the amplitude.

[0020] Optionally, the generative adversarial network design and progressive learning strategy in S2 include:

[0021] S21, Generator Design: The generator consists of a distortion network and a detail enhancement network. The distortion network processes geometric distortion, while the detail enhancement network improves the clarity and details of the distorted image.

[0022] S22, Discriminator Design: The discriminator follows the DCGAN structure and is used to evaluate the authenticity of the distorted image processed by the generator and output the probability of whether the image is a real image;

[0023] S23, progressive learning strategy: Using a progressive learning method, start training from high-distortion images and gradually introduce low-distortion images to improve the neural network's ability to perceive and correct small distortions.

[0024] Optionally, the large-scale remote sensing image distortion correction in S3 includes:

[0025] S31, image block processing: dividing the remote sensing image into image blocks evenly along the row direction;

[0026] S32, selecting a reference block and extracting a distortion field: In each row of image blocks, select the image block with the richest geometric features as the reference block, extract the distortion field information of the reference block, and extend the extracted distortion field information to the entire row through an extension strategy;

[0027] S33, geometric distortion correction: using the distortion field information extracted from the reference block, perform geometric distortion correction on the entire row of image blocks;

[0028] S34, detail enhancement and optimization: After completing geometric distortion correction, the correction results are optimized through the detail enhancement network to restore image details;

[0029] S35, stitching and reconstructing the complete image: After completing the selection of reference blocks and extraction of distortion fields, geometric distortion correction, detail enhancement and optimization line by line, all image blocks are stitched together to reconstruct the complete corrected image.

[0030] Optionally, the size of the image block is 256×256 pixels.

[0031] Beneficial effects of the present invention:

[0032] This paper uses tremor simulation to generate distortion datasets with decreasing tremor levels. Combining this with a generative adversarial network and a progressive learning strategy, it gradually improves the network's ability to correct for minor distortions. When processing large-scale remote sensing imagery, this paper uses a strategy of local processing and row-by-row correction to not only improve processing efficiency but also significantly enhance image accuracy.

[0033] The present invention, through the method of image block division and line-by-line correction, can meticulously process each small image block, ensuring the precise correction and detail restoration of large-scale remote sensing images. By reasonably dividing image blocks, selecting reference blocks with rich features, and extracting and expanding their distortion field information, it follows the inter-line consistency principle of the TDI imaging system and ensures the consistency of the distortion correction effect. This method has the advantages of high precision, automation, and is applicable to the processing of large-scale remote sensing images, which helps to improve the quality and application value of satellite remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only those of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0035] Figure 1 Schematic diagram of the correction method process for the embodiment of the present invention;

[0036] Figure 2 Schematic diagram of the generation of a distortion data set with gradually decreasing tremor levels for the embodiment of the present invention;

[0037] Figure 3 Schematic diagram of the structure and correction process of the generator for the embodiment of the present invention;

[0038] Figure 4 Schematic diagram of the progressive learning strategy for the embodiment of the present invention;

[0039] Figure 5 Schematic diagram of the correction effect of images with gradually decreasing tremor distortion levels for the embodiment of the present invention;

[0040] Figure 6 Schematic diagram of the correction principle of large-scale real distortion remote sensing images for the embodiment of the present invention;

[0041] Figure 7 Original image of large-scale real distortion remote sensing images for the embodiment of the present invention;

[0042] Figure 8 Correction result of large-scale real distortion remote sensing images for the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments. For some well-known technologies, those skilled in the art can also adopt other alternative methods for implementation; moreover, the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0044] It should be noted that in the specification, the mention of "an embodiment", "embodiment", "exemplary embodiment", "some embodiments", etc. indicates that the described embodiment may include specific features, structures or characteristics, but not necessarily every embodiment includes such specific features, structures or characteristics. Additionally, when combining embodiments to describe specific features, structures or characteristics, implementing such features, structures or characteristics in combination with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the relevant art.

[0045] Generally, terms can be understood at least in part from their use in the context. For example, at least in part depending on the context, the term "one or more" used herein can be used to describe any feature, structure or characteristic in a singular sense, or can be used to describe a combination of features, structures or characteristics in a plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey a set of exclusive factors, but instead, at least in part depending on the context, allowing for the existence of other factors that may not be explicitly described.

[0046] As Figures 1-8 shown, the spaceborne TDI CCD high-frequency tremor distortion correction method based on progressive learning includes the following steps:

[0047] I. Construct datasets with different tremor levels through the method of tremor simulation:

[0048] 1. Collect original images: Collect high-quality images containing significant geometric information as reference images from remote sensing image datasets such as DOTA-v1.0, DOTA-v1.5, and VEDAI, and large natural image datasets such as ImageNet, COCO, and Places2.

[0049] 2. Produce distorted images with gradually decreasing tremor levels: First, we introduce the tremor level L J to describe different degrees of tremor distortion. It is defined as the ratio of the amplitude attenuation factor to the amplitude. The amplitude attenuation factor represents the reduction of the error caused by jitter in multiple integration stages of the TDI imaging system. The expression of the amplitude attenuation factor is as follows:

[0050] α = |sinc(Nτf)|;

[0051] With the primary goal of correcting minor geometric distortions, we focused on simulating distorted images with low blur levels when constructing the dataset, so as to fully reflect the geometric distortions caused by amplitude. Since geometric distortions are mainly affected by amplitude, we defined a tremor level based on amplitude A and amplitude attenuation factor α. The dataset with tremor level i is defined as follows:

[0052]

[0053] First, high-quality reference images rich in geometric features are selected from the reference dataset; then, different degrees of tremor distortion are applied to the reference images by adjusting the tremor simulation parameters to generate multiple groups of distorted images with gradually decreasing tremor levels; finally, these distorted images and the corresponding reference images are paired for use in the training, evaluation, and testing of subsequent correction models. Through this method, we can construct a dataset highly matching the actual scenario by simulation means without a dedicated distorted dataset, ensuring that the trained model can effectively correct tremor distortion in the real distorted scenario. In addition, the reason for choosing natural images as the training set is that they usually have higher image quality, and compared with remote sensing images, natural images usually have richer geometric features and texture information, which helps the model better learn the distortion patterns and correction rules during the training process. On the other hand, we use remote sensing images for the validation set and the test set because the ultimate goal of this invention is to correct distorted remote sensing images. Using remote sensing images as the validation set aims to verify whether the network trained with natural images can effectively correct the distortions in remote sensing images. This process helps us adjust the model parameters in a timely manner during the training stage to ensure its good generalization ability. Remote sensing images are used as the test set for the final evaluation of the model performance. By comparing the performance of different models in correcting remote sensing images, we can determine which models perform best in terms of correction accuracy, detail preservation, etc. and apply them to the correction of real distorted remote sensing images. The generation process of the simulated distorted dataset with different tremor levels is as Figure 2 shown.

[0054] II. Design of Generative Adversarial Network and Progressive Learning Strategy:

[0055] The generator of the network adopts a dual-network structure composed of a distortion network and a detail enhancement network. Among them, the distortion network plays a decisive role in dealing with geometric distortions and is the key component for eliminating image distortions; while the detail enhancement network effectively improves the clarity and detail performance of the image. The two complement each other, and the absence of any component will affect the correction effect. The structure and correction process of its generator are as Figure 3As shown. The discriminator is designed following the discriminator structure concept of DCGAN and is used to output the probability of the authenticity of the image. Deep learning has advantages in dealing with complex distortions, but its feature extraction ability in the early training stage is limited, making it difficult to capture pixel-level subtle distortions, thus affecting the correction accuracy. If the model is directly allowed to learn and correct these images with low-level distortions, it may be difficult for the model to effectively capture these small changes, thereby affecting the final correction effect. For this reason, the present invention proposes a progressive learning method, adopting a training strategy from high distortion to low distortion, to guide the model to gradually adapt and improve its perception and correction ability for subtle geometric distortions. In the initial stage of training, a dataset composed of high-distortion images is used for training, enabling the model to initially establish the distortion correction ability. Subsequently, a dataset composed of low-distortion images is gradually introduced for further training, gradually enhancing its ability to identify and correct smaller geometric distortions, and finally achieving precise correction of minute distortions. This training method not only enables the network to master the ability to correct minute geometric distortions from easy to difficult, but also effectively alleviates the training difficulties when directly training to correct slightly distorted images. The progressive learning strategy is as Figure 4 shown. Figure 5 shows the correction results of real distorted remote sensing images with different distortion degrees, Figure 5 and the distortion degree of each image in

[0056] decreases sequentially. However, our method can still effectively correct them.

[0057] When dealing with large-scale high-resolution remote sensing images, the interline signal synchronous accumulation characteristic of the TDI imaging system ensures the consistency of the changes in each row of images in the push-broom direction. This characteristic provides an effective means for the geometric distortion recognition and correction of large-scale high-resolution remote sensing images. Based on this, the present invention proposes a local processing strategy to improve the processing efficiency and correction accuracy through the methods of fine segmentation and row-by-row correction. First, the image is evenly divided into small image blocks of 256×256 pixels along the row direction to conduct a detailed analysis of local geometric distortions; subsequently, the image block with the richest geometric features is selected as the reference block in each row of image blocks, and its distortion field information is extracted and its distortion field is expanded to the entire row through an expansion strategy to maintain interline consistency. Through the above operations, the local distortion field of this row can be obtained. Then, with the help of this local distortion field, geometric distortion correction is realized for all the image blocks in this row, and then the correction result is further optimized through a detail enhancement network to restore the image details. After completing the above operations row by row, all the image blocks are stitched together to reconstruct the complete corrected image. The correction principle is as Figure 6 shown. Figure 7 shows the original image of a large-scale real distorted remote sensing image, Figure 8 and

[0058] The present invention covers any alternatives, modifications, equivalent methods, and solutions that are within the spirit and scope of the present invention. For the public to have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, and those skilled in the art can fully understand the present invention without the description of these details. Additionally, well-known methods, processes, procedures, components, and circuits are not described in detail to avoid unnecessary confusion with the essence of the present invention.

[0059] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, several improvements and refinements can be made without departing from the principles of the present invention, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A method for correcting high-frequency tremor distortion of spaceborne TDI CCD based on progressive learning, characterized in that The following steps are involved: S1, constructing a distortion dataset with decreasing tremor levels: generating a distortion dataset with decreasing tremor levels for neural network training by using a tremor simulation method; S2, Generative Adversarial Network Design and Progressive Learning Strategy: Design a generative adversarial network whose generator includes a distortion network and a detail enhancement network. Using a progressive learning strategy based on a hierarchical distortion dataset, training begins with highly distorted images and gradually introduces low-distortion images to improve the neural network's ability to perceive and correct small distortions. S3, large-scale remote sensing image distortion correction: Based on the inter-row consistency characteristics of the TDI camera, large-scale remote sensing images are locally processed and corrected row by row, and finally a complete corrected image is reconstructed.

2. The method for on-orbit TDI CCD high-frequency tremor distortion correction based on progressive learning according to claim 1, wherein The tectonic tremor level dataset in S1 includes: S11, collect original images: collect high-quality reference images with significant geometric features from DOTA-v1.5, VEDAI remote sensing image datasets and ImageNet, COCO, and Places2 natural image datasets; S12, generating distorted images with decreasing tremor levels: applying different degrees of tremor distortion to the reference image, defining tremor levels according to amplitude and amplitude attenuation factor, and generating distorted images with decreasing tremor levels; S13, generating a graded distortion dataset: pairing distorted images of different tremor levels with reference images to form a graded distortion image dataset for training, evaluation, and testing; S14, Dataset division: Use the natural image dataset as the training set and the remote sensing image dataset as the validation set and test set.

3. The on-orbit TDI CCD high-frequency tremor distortion correction method based on progressive learning according to claim 2, wherein The amplitude attenuation factor is expressed as: α=|sin c(Nτf)|; Where α is the amplitude attenuation factor.

4. The on-orbit TDI CCD high-frequency tremor distortion correction method based on progressive learning according to claim 3, wherein The distorted image dataset is represented as: Among them, L J (i) is the distorted image of tremor level i, and A is the amplitude.

5. The on-orbit TDI CCD high-frequency tremor distortion correction method based on progressive learning according to claim 4, characterized in that, The generative adversarial network design and progressive learning strategy in S2 include: S21, Generator Design: The generator consists of a distortion network and a detail enhancement network. The distortion network processes geometric distortion, while the detail enhancement network improves the clarity and details of the distorted image. S22, Discriminator Design: The discriminator follows the DCGAN structure and is used to evaluate the authenticity of the distorted image processed by the generator and output the probability of whether the image is a real image; S23, progressive learning strategy: Using a progressive learning method, start training from high-distortion images and gradually introduce low-distortion images to improve the neural network's ability to perceive and correct small distortions.

6. The on-orbit TDI CCD high-frequency tremor distortion correction method based on progressive learning according to claim 5, characterized in that The large-scale remote sensing image distortion correction in S3 includes: S31, image block processing: the remote sensing image is evenly divided into image blocks along the row direction; S32, selecting a reference block and extracting a distortion field: In each row of image blocks, select the image block with the richest geometric features as the reference block, extract the distortion field information of the reference block, and extend the extracted distortion field information to the entire row through an extension strategy; S33, geometric distortion correction: using the distortion field information extracted from the reference block, perform geometric distortion correction on the entire row of image blocks; S34, detail enhancement and optimization: After completing geometric distortion correction, the correction results are optimized through the detail enhancement network to restore image details; S35, Stitching and reconstructing the complete image: After completing the selection of reference blocks, extraction of distortion fields, geometric distortion correction, detail enhancement and optimization line by line, all image patches are stitched to reconstruct the complete corrected image.

7. The method for correcting high-frequency tremor distortion of spaceborne TDI CCD based on progressive learning according to claim 6, characterized in that The size of the said image patch is 256×256 pixels.

Images