Satellite-borne TDI CCD high-frequency jitter distortion correction method based on progressive learning

Abstract

The present application relates to the technical field of remote sensing satellite image data processing, and specifically relates to a spaceborne TDI CCD high-frequency tremor distortion correction method based on progressive learning, comprising the following steps: S1, constructing a distortion data set with gradually decreasing tremor levels: generating a distortion data set with gradually decreasing tremor levels by a tremor simulation method for neural network training; S2, generating an adversarial network design and a progressive learning strategy: designing an adversarial network, using a progressive learning strategy, starting training from a high-distortion image, and gradually introducing a low-distortion image; S3, large-scale remote sensing image distortion correction: according to the characteristics of the inter-row consistency of the TDI camera, locally processing and row-by-row correcting large-scale distortion remote sensing images, and finally reconstructing a complete corrected image. The present application has the advantages of high precision, automation and suitability for large-scale remote sensing image processing, and helps to improve the quality and application value of satellite remote sensing images.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing satellite image data processing, and particularly relates to a spaceborne TDI CCD high-frequency tremor distortion correction method based on progressive learning. BACKGROUND

[0002] Satellite platform tremor refers to the small amplitude vibration response of a satellite caused by disturbances such as attitude adjustment, pointing control, solar panel adjustment, and periodic movement of on-board moving parts during on-orbit operation of the satellite. These tremor disturbances can cause slight displacement of the camera, thereby destroying the synchronization between photo-generated charge packets and image motion, affecting the clarity and geometric accuracy of the image. In a TDI CCD imaging system, the influence of tremor on image quality is particularly significant. TDI CCD extends the exposure time through multi-stage time integration, improves the light flux, sensitivity, and signal-to-noise ratio, thereby improving the performance of the imaging system. However, the slight displacement caused by tremor can cause image motion, resulting in image blurring during imaging, which seriously affects the geometric accuracy of high-resolution remote sensing images.

[0003] Currently, tremor distortion detection and compensation mainly rely on two methods: one is based on a parallax imaging system, which detects tremor effects by analyzing the overlapping area of multiple spectral bands or matching adjacent CCD images. However, this method requires high-precision feature extraction and image matching, and is difficult to apply to single panchromatic images. The other method is to use high-frequency attitude sensors such as angular velocity and angular displacement sensors to capture satellite micro-tremor information to assist compensation and image correction. However, these methods usually rely on high-precision sensors and are limited by satellite hardware configuration. Traditional physical model methods often face problems such as complex parameters, poor adaptability, and difficulty in efficiently correcting micro-tremor distortion. In recent years, although deep learning has made some progress in image deblurring, most methods focus on correcting large distortion images, and relatively little research has been done on correcting micro-distortion. Therefore, existing technology cannot achieve high-precision tremor distortion correction. SUMMARY

[0004] The present application provides a spaceborne TDI CCD high-frequency tremor distortion correction method based on progressive learning, which can effectively suppress the influence of satellite platform tremor and effectively correct the micro-geometric distortion caused by high-frequency tremor on remote sensing images. The present application has high precision, automation, and wide application prospects.

[0005] The spaceborne TDI CCD high-frequency tremor distortion correction method based on progressive learning comprises the following steps:

[0006] S1, constructing a distortion dataset with sequentially decreasing tremor levels: generating a distortion dataset with sequentially decreasing tremor levels by a tremor simulation method for training the neural network;

[0007] S2, generating a generative adversarial network design and a progressive learning strategy: designing a generative adversarial network, the generator of which includes a distortion network and a detail enhancement network, and using a progressive learning strategy based on the hierarchical distortion dataset to train from high-distortion images and gradually introduce low-distortion images to improve the neural network's perception and correction ability for minor distortions;

[0008] S3, large-scale remote sensing image distortion correction: according to the characteristics of TDI camera interline consistency, performing local processing and row-by-row correction on large-scale remote sensing images, and finally reconstructing a complete corrected image.

[0009] Optionally, the S1 includes:

[0010] S11, collecting original images: collecting high-quality reference images containing significant geometric features from DOTA-v1.5, VEDAI, and other remote sensing image datasets, and ImageNet, COCO, Places2, and other natural image datasets;

[0011] S12, making distortion images with sequentially decreasing tremor levels: applying different degrees of tremor distortion to the reference images, defining tremor levels according to amplitude and amplitude attenuation factor, and generating distortion images with sequentially decreasing tremor levels;

[0012] S13, generating a hierarchical distortion dataset: pairing distortion images of different tremor levels with reference images to form a hierarchical distortion image dataset for training, evaluation, and testing;

[0013] S14, dataset division: using natural image datasets as training sets, and remote sensing image datasets as validation and test sets.

[0014] Optionally, the amplitude attenuation factor is represented as:

[0015] α=|sin c(Nτf)|;

[0016] Wherein, α is the amplitude attenuation factor.

[0017] Optionally, the distortion image dataset is represented as:

[0018]

[0019] Wherein, L J (i) is the distortion image of tremor level i, and A is the amplitude.

[0020] Optionally, the design of the generative adversarial network in S2 includes:

[0021] S21, generator design: the generator is composed of a distortion network and a detail enhancement network, the geometric distortion is processed through the distortion network, and the clarity and details of the distorted image are improved through the detail enhancement network;

[0022] S22, discriminator design: the discriminator follows the DCGAN structure, and is used to evaluate the authenticity of the distorted image processed by the generator, and outputs the probability of whether the image is a real image;

[0023] S23, progressive learning strategy: the progressive learning method is adopted, the high-distortion image is trained first, and the low-distortion image is gradually introduced to improve the perception and correction ability of the neural network to the micro-distortion.

[0024] Optionally, the large-scale remote sensing image distortion correction in S3 includes:

[0025] S31, image block processing: the remote sensing image is uniformly divided into image blocks along the row direction;

[0026] S32, selecting a reference block and extracting a distortion field: in each row of image blocks, the image block with the most abundant geometric features is selected as the reference block, the distortion field information of the reference block is extracted, and the extracted distortion field information is expanded to the entire row through an expansion strategy;

[0027] S33, geometric distortion correction: the distortion field information extracted from the reference block is used to correct the geometric distortion of the image blocks in the entire row;

[0028] S34, detail enhancement and optimization: after completing the geometric distortion correction, the correction result is optimized through the detail enhancement network to restore the image details;

[0029] S35, splicing and reconstructing the complete image: after completing the selection of the reference block and the extraction of the distortion field, the geometric distortion correction, the detail enhancement and optimization, all image blocks are spliced to reconstruct the complete corrected image.

[0030] Optionally, the size of the image block is 256*256 pixels.

[0031] The beneficial effects of the present application are:

[0032] The present application generates a distortion data set with decreasing tremor levels in turn through tremor simulation, combines generative adversarial networks and progressive learning strategies, and gradually improves the correction ability of the network to micro-distortion. When processing large-scale remote sensing images, the present application uses local processing and row-by-row correction strategies, which not only improves the processing efficiency, but also greatly improves the accuracy of the image.

[0033] The present application can process each small image block in detail by the image block and line-by-line correction method, ensure the accurate correction and detail restoration of large-scale remote sensing images, and follow the interline consistency principle of the TDI imaging system and ensure the consistency of the distortion correction effect by reasonably dividing the image blocks, selecting the reference blocks with rich features and extracting and expanding the distortion field information, so that the method has the advantages of high precision, automation and applicability to large-scale remote sensing image processing, and helps to improve the quality and application value of satellite remote sensing images. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the present application or prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only a part of the present application, and other drawings can also be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0035] Figure 1 The correction method flowchart of the embodiment of the present application;

[0036] Figure 2 The generation diagram of the tremor distortion data set with gradually decreasing tremor levels of the embodiment of the present application;

[0037] Figure 3 The structure and correction process diagram of the generator of the embodiment of the present application;

[0038] Figure 4 The progressive learning strategy diagram of the embodiment of the present application;

[0039] Figure 5 The correction effect diagram of the image with gradually decreasing tremor distortion levels of the embodiment of the present application;

[0040] Figure 6 The large-scale real distortion remote sensing image correction principle diagram of the embodiment of the present application;

[0041] Figure 7 The large-scale real distortion remote sensing image of the embodiment of the present application;

[0042] Figure 8 The large-scale real distortion remote sensing image correction result of the embodiment of the present application. DETAILED DESCRIPTION

[0043] The application will be described in detail below with reference to the drawings and specific embodiments. It should be noted here that in order to make the embodiments more detailed, the following embodiments are the best, preferred embodiments, and other alternative ways can also be implemented by those skilled in the art for some known technologies; and the drawings are only used to describe the embodiments more specifically, and are not intended to specifically limit the application.

[0044] It should be noted that in the specification, "one embodiment", "embodiment", "exemplary embodiment", "some embodiments" and the like indicate that the described embodiment can include a specific feature, structure or property, but not necessarily every embodiment includes this specific feature, structure or property. In addition, when a specific feature, structure or property is described in combination with an embodiment, it should be within the knowledge of those skilled in the related art to implement such a feature, structure or property in combination with other embodiments, whether or not it is explicitly described.

[0045] Generally, the terms can be understood at least in part from the context of their use. For example, depending on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular or can be used to describe combinations of features, structures, or characteristics, whether large or small, whether related or unrelated to each other. In addition, the term "based on" can be understood as not necessarily intended to convey a set of exclusive factors, but can instead, at least in part, depend on the context, allowing the presence of other factors not necessarily explicitly described.

[0046] As shown in Figures 1-8 The progressive learning-based on-board TDI CCD high-frequency jitter distortion correction method includes the following steps:

[0047] I. Constructing data sets of different jitter levels by jitter simulation method:

[0048] 1. Collecting original images: Collect high-quality images containing significant geometric information from DOTA-v1.0, DOTA-v1.5, and VEDAI remote sensing image datasets and ImageNet, COCO, and Places2 large natural image datasets as reference images.

[0049] 2. Make distortion images with decreasing jitter levels: First, we introduce the jitter level L J to describe different degrees of jitter distortion. Its definition is the ratio of the amplitude attenuation factor to the amplitude. The amplitude attenuation factor represents the reduction of error caused by jitter in multiple integration stages of the TDI imaging system, and the amplitude attenuation factor expression is as follows:

[0050] α=|sin c(Nτf)|;

[0051] To mainly correct slight geometric distortion, we focus on simulating images with low blur degree when constructing the dataset, so as to fully reflect the geometric distortion caused by amplitude. Since the geometric distortion is mainly affected by the amplitude, we define the tremor level based on the amplitude A and the amplitude attenuation factor a. The dataset with tremor level i is defined as follows:

[0052]

[0053] First, we select high-quality and rich-geometry reference images from the reference dataset; then, we apply different degrees of tremor distortion to the reference images by adjusting the tremor simulation parameters, generating multiple sets of distorted images with decreasing tremor levels; finally, we pair these distorted images with the corresponding reference images for subsequent training, evaluation and testing of the correction model. By this method, we can construct a highly matched dataset with actual scenes using simulation means without a special distortion dataset, ensuring that the trained model can effectively correct the tremor distortion in real distortion scenarios. In addition, the reason for choosing natural images as the training set is that they usually have higher image quality and more rich geometric features and texture information than remote sensing images, which helps the model learn the distortion pattern and correction rule better during training. On the other hand, we use remote sensing images for the validation set and test set because the ultimate goal of this invention is to correct distorted remote sensing images. Using remote sensing images as the validation set aims to verify whether the network trained using natural images can effectively correct the distortion in remote sensing images. This process helps us adjust the model parameters in the training stage to ensure good generalization ability. Remote sensing images are used as the test set for final model performance evaluation. By comparing the performance of different models in correcting remote sensing images, we can determine which models perform best in terms of correction accuracy and detail preservation, and apply them to the correction of real distorted remote sensing images. The process of generating simulated distortion datasets with different tremor levels is shown in Figure 2 .

[0054] II. Design of generative adversarial network and progressive learning strategy

[0055] The generator of the network adopts a double-network structure composed of a distortion network and a detail enhancement network. The distortion network plays a decisive role in handling geometric distortion and is the key component for eliminating image distortion; while the detail enhancement network effectively improves the clarity and detail performance of the image. Both components complement each other, and the absence of either component will affect the correction effect. The structure of the generator and the correction process are shown in Figure 3The discriminator design of the discriminator follows the discriminator structure concept of DCGAN, which is used to output the probability of image authenticity. Deep learning has advantages in processing complex distortion, but its feature extraction capability is limited in the early training stage, and it is difficult to capture subtle pixel-level distortion, thereby affecting the correction accuracy. If the model is directly allowed to learn and correct these low-level distortion images, it may be difficult for the model to effectively capture these minor changes, thereby affecting the final correction effect. To this end, the present application proposes a progressive learning method, which adopts a training strategy from high distortion to low distortion, and guides the model to gradually adapt and improve its perception and correction ability for subtle geometric distortion. In the initial stage of training, the dataset composed of high distortion images is used for training, so that the model can initially establish the distortion correction capability, and then gradually introduce the dataset composed of low distortion images for further training, gradually enhance its ability to identify and correct small geometric distortion, and finally achieve accurate correction of minor distortion. This training method not only enables the network to master the correction ability of minor geometric distortion from easy to difficult, but also effectively alleviates the training difficulty when directly training to correct slightly distorted images. The progressive learning strategy is as shown in Figure 4 . Figure 5 The correction results of real distortion remote sensing images with different distortion levels are shown in FIGS. Figure 5 The distortion level of each image in FIGS.

[0056] Three, large-scale high-resolution remote sensing image distortion correction:

[0057] In processing large-scale high-resolution remote sensing images, the interline signal synchronization accumulation characteristics of the TDI imaging system ensure the consistency of the changes of each row of images in the push-broom direction, which provides an effective means for geometric distortion identification and correction of large-scale high-resolution remote sensing images. Based on this, the present application proposes a local processing strategy, which improves the processing efficiency and correction accuracy through fine segmentation and row-by-row correction. First, the image is uniformly divided into small image blocks of 256x256 pixels along the row direction, so as to analyze the local geometric distortion in detail; then, the image block with the most abundant geometric features is selected as the reference block in each row of image blocks, and the distortion field information is extracted and expanded to the whole row through the expansion strategy to maintain the consistency between rows. Through the above operation, the local distortion field of this row can be obtained, and then the geometric distortion correction of all image blocks in this row is realized by means of the local distortion field, and then the correction result is further optimized by the detail enhancement network to restore the image details. After the above operations are completed row by row, all image blocks are spliced to reconstruct the complete corrected image. The correction principle is as shown in Figure 6 . Figure 7 The original image of a large-scale real distortion remote sensing image is shown in FIGS. Figure 8 The correction result is shown in FIGS.

[0058] The present application encompasses any alternatives, modifications, equivalent methods and solutions made to the essence and scope of the present application. In order to make the public have a thorough understanding of the present application, specific details are described in the following preferred embodiments of the present application, and the present application can also be fully understood without the description of these details to those skilled in the art. In addition, in order to avoid unnecessary confusion to the essence of the present application, well-known methods, processes, procedures, elements and circuits, etc. are not described in detail.

[0059] The above is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can also be made, which should be considered as the protection scope of the present application.

Claims

1. A method for correcting high-frequency jitter distortion of a space-borne TDI CCD based on progressive learning, characterized in that, The method comprises the following steps: S1, constructing a distortion data set with decreasing tremor levels: generating a distortion data set with decreasing tremor levels by a tremor simulation method for training a neural network; S2, generating a generative adversarial network design and a progressive learning strategy: designing a generative adversarial network, the generator of which comprises a distortion network and a detail enhancement network, and using a progressive learning strategy based on the hierarchical distortion data set to start training from high-distortion images and gradually introduce low-distortion images to improve the perception and correction ability of the neural network for slight distortion; S3, large-scale remote sensing image distortion correction: according to the characteristics of the TDI camera interline consistency, the large-scale remote sensing image is processed locally and corrected row by row, and finally the complete corrected image is reconstructed.

2. The method of claim 1, wherein the method is a method of correcting high frequency jitter distortion of a space-borne TDI CCD based on progressive learning. The construction of the tremor level data set in S1 comprises: S11, collecting original images: collecting reference images of high quality and containing significant geometric features from DOTA-v1.5, VEDAI remote sensing image data set and ImageNet, COCO, Places2 natural image data set; S12, making distortion images with decreasing tremor levels: applying different degrees of tremor distortion to the reference images, defining tremor levels according to amplitude and amplitude attenuation factor, and generating distortion images with decreasing tremor levels; S13, generating a hierarchical distortion data set: pairing distortion images of different tremor levels with reference images to form a hierarchical distortion image data set for training, evaluation and testing; S14, data set division: using natural image data set as training set, remote sensing image data set as verification set and test set.

3. The method of claim 2, wherein the method is a method of on-board TDI CCD high frequency jitter distortion correction based on progressive learning. The amplitude attenuation factor is represented as: α=|sin c(Nτf)|; Wherein, α is the amplitude attenuation factor.

4. The method of claim 3, wherein the method is a space-borne TDI CCD high frequency jitter distortion correction method based on progressive learning. The distortion image data set is represented as: where L J (i) Distorted image for tremor grade i, A is amplitude.

5. The method of claim 4, wherein the method is a space-borne TDI CCD high frequency jitter distortion correction method based on progressive learning. The generative adversarial network design and progressive learning strategy in S2 comprises: S21, generator design: the generator is composed of a distortion network and a detail enhancement network, which processes geometric distortion through the distortion network and improves the clarity and details of the distortion image through the detail enhancement network; S22, discriminator design: the discriminator follows the DCGAN structure and is used to evaluate the authenticity of the distortion image processed by the generator, outputting the probability of the image being a real image; S23, progressive learning strategy: using a progressive learning method, starting training from high-distortion images and gradually introducing low-distortion images to improve the perception and correction ability of the neural network for slight distortion.

6. The method of claim 5, wherein the method is a space-borne TDI CCD high frequency jitter distortion correction method based on progressive learning. The large-scale remote sensing image distortion correction in S3 comprises: S31, image block processing: dividing the remote sensing image into image blocks uniformly along the row direction; S32, selecting reference blocks and extracting distortion fields: in each row of image blocks, selecting the image block with the most abundant geometric features as the reference block, extracting the distortion field information of the reference block, and expanding the extracted distortion field information to the entire row through an expansion strategy; S33, geometric distortion correction: using the distortion field information extracted from the reference block to correct the geometric distortion of the entire row of image blocks; S34, detail enhancement and optimization: after completing the geometric distortion correction, the correction result is optimized through the detail enhancement network to restore the image details; S35, splicing and reconstructing a complete image: after selecting reference blocks and extracting distortion fields, geometric distortion correction, detail enhancement and optimization are completed row by row, all image blocks are spliced to reconstruct a complete corrected image.

7. The on-board TDI CCD high-frequency jitter distortion correction method based on progressive learning according to claim 6, characterized in that, The size of the image block is 256*256 pixels.

Images