A TDI-CCD image deartifacting method, device, equipment, medium and product

Abstract

The application discloses a TDI-CCD image deartifact method and device, equipment, medium and product, and relates to the technical field of image processing. The method comprises the following steps: constructing a deep deblurring network; the deep deblurring network comprises a Wiener filter module and a generative adversarial network, and the Wiener filter module is arranged at the input end of a generator of the generative adversarial network; the Wiener filter module is used for filtering a frequency domain image; the deep deblurring network is trained, and the trained deep deblurring network is used as a deartifact model; during the training of the deep deblurring network, a point spread function and a regularization parameter of the Wiener filter module are used as learnable network parameters; and an image with row smear obtained from a TDI-CCD camera is processed by using the deartifact model to obtain a deartifact image. The application can effectively remove image artifacts and improve the definition of a TDI-CCD image.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, device, medium and product for removing artifacts from TDI-CCD images. Background Technology

[0002] Unlike traditional charge-coupled devices (CCDs), time-delayed integration (TDI) CCDs are linear array photodetectors arranged along a direction on the image sensor, typically perpendicular to the direction of motion. As the subject moves in front of the image sensor, the TDI-CCD array gradually accumulates image information in stages on the sensor. Over time, each pixel accumulates light signals from the same location, which are then passed to neighboring pixels, ultimately forming the entire image.

[0003] While TDI-CCDs offer significant advantages in high-quality imaging of moving scenes, their delay integration capability can introduce a series of errors due to velocity mismatch. The primary contributing factor is the mismatch between the relative speed of the TDI-CCD and the object, and the line switching speed of the TDI-CCD. This error causes inconsistencies in the object information captured by each line of the TDI-CCD, ultimately resulting in artifacts or blurring in the image. Therefore, further repair of image artifacts produced by TDI-CCDs in this situation is necessary to improve image quality.

[0004] Currently, to reduce image distortion caused by velocity mismatch errors, engineers mainly employ three solutions: stabilizing the camera or target, reducing exposure time, and post-processing. Stabilizing the camera or target is achieved by reducing the relative speed between the camera and the subject. This method minimizes the interference caused by velocity mismatch errors, but it also limits the application scenarios of TDI-CCDs, making it impossible to achieve high-quality imaging of high-speed moving objects or high line frequency conditions. Reducing exposure time shortens the exposure time to reduce the image capture time window, thereby reducing the impact of velocity mismatch errors. However, shorter exposure times may lead to a lower image signal-to-noise ratio, making the image harder to distinguish. Post-processing techniques use digital image processing to repair artifacts or blur in the image. This is a low-cost approach, but it still requires additional computational resources and algorithms.

[0005] Traditional image deblurring methods are mostly based on the premise that the blur kernel is known or that prerequisite conditions are estimated or assumed, which has considerable limitations. Summary of the Invention

[0006] The purpose of this application is to provide a method, apparatus, device, medium, and product for removing artifacts from TDI-CCD images, which can effectively remove image artifacts and improve the clarity of TDI-CCD images.

[0007] To achieve the above objectives, this application provides the following solution:

[0008] In a first aspect, this application provides a method for removing artifacts from TDI-CCD images, the method comprising:

[0009] A deep deblurring network is constructed; the deep deblurring network includes a Wiener filtering module and a generative adversarial network, the Wiener filtering module is set at the input end of the generator of the generative adversarial network; the Wiener filtering module is used to filter the frequency domain image;

[0010] The deep deblurring network is trained and used as an artifact removal model. During the training of the deep deblurring network, the point spread function and regularization parameters of the Wiener filter module are used as learnable network parameters.

[0011] The image with line trailing artifacts acquired from the TDI-CCD camera is processed using the artifact removal model to obtain an artifact-free image.

[0012] Optionally, the deep deblurring network is trained, and the trained deep deblurring network is used as an artifact removal model, specifically including:

[0013] Freeze the generative adversarial network and pre-train the Wiener filtering module to obtain a pre-trained deep deblurring network.

[0014] The pre-trained deep deblurring network is jointly trained using a training set, and the trained deep deblurring network is used as the artifact removal model. Each sample data in the training set includes a pair of sample images and label data. The sample images are TDI-CCD blurred images, and the label data are TDI-CCD clear images.

[0015] Optionally, the pre-trained deep deblurring network is jointly trained using the training set, and the trained deep deblurring network is used as the artifact removal model, specifically including:

[0016] For each sample image, a two-dimensional fast Fourier transform is performed on the sample image to convert the spatial domain sample image into a frequency domain image; the sample image is a TDI-CCD sample image with line trailing.

[0017] Input the frequency domain image into the Wiener filtering module to obtain the filtered frequency domain image;

[0018] The filtered frequency domain image is subjected to a two-dimensional inverse fast Fourier transform to obtain a filtered spatial domain sample image.

[0019] The filtered spatial domain sample image is input into the generative adversarial network. The generator outputs a motion-blurred sample image, and the discriminator judges the generator's output as real or fake.

[0020] Optionally, the frequency domain image is input into the Wiener filtering module to obtain a filtered frequency domain image, specifically including: the filtered frequency domain image is represented as follows:

[0021]

[0022] in, Let F(u,v) be the filtered frequency domain image, G(u,v) be the Wiener filter module, α be the regularization coefficient, H(u,v) be the point spread function spectrum of the Wiener filter module, and * denote the complex conjugate operation. |H(u,v)| 2 The energy spectrum H(u,v)H * (u,v), where u is the spatial frequency domain component in the horizontal direction and v is the spatial frequency domain component in the vertical direction.

[0023] Optionally, the point spread function spectrum of the Wiener filter module is expressed as:

[0024] H(u,v) = FFT{h(x,y)};

[0025]

[0026] Where h(x,y) is the point spread function, x is the horizontal coordinate in the spatial domain, y is the vertical coordinate in the spatial domain, FFT{} is the two-dimensional fast Fourier transform, and L is the number of trailing lines.

[0027] Optionally, the generative adversarial network is a DeblurGAN-v2 network, in which the generator is a dual-scale relative discriminator, which includes a global discriminator and a local discriminator;

[0028] When the pre-trained deep deblurring network is jointly trained using the training set, both the global discriminator and the local discriminator use the RaLSGAN loss function, and the total loss value is the sum of the RaLSGAN loss values ​​of the global discriminator and the local discriminator.

[0029] Secondly, this application provides a TDI-CCD image artifact removal apparatus, which applies the aforementioned TDI-CCD image artifact removal method, and the TDI-CCD image artifact removal apparatus includes:

[0030] A deep deblurring network construction module is used to construct a deep deblurring network; the deep deblurring network includes a Wiener filtering module and a generative adversarial network, the Wiener filtering module is set at the input end of the generator of the generative adversarial network; the Wiener filtering module is used to filter the frequency domain image;

[0031] A deep deblurring network training module is used to train the deep deblurring network and use the trained deep deblurring network as an artifact removal model. During the training of the deep deblurring network, the point spread function and regularization parameters of the Wiener filter module are used as learnable network parameters.

[0032] The artifact removal model application module is used to process images with line trailing shadows acquired from a TDI-CCD camera using the artifact removal model to obtain an artifact-removed image.

[0033] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the TDI-CCD image artifact removal method described in any one of the above.

[0034] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the TDI-CCD image artifact removal method described above.

[0035] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the TDI-CCD image artifact removal method described above.

[0036] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0037] This application provides a method, apparatus, device, medium, and product for removing artifacts from TDI-CCD images. The deep deblurring network includes a Wiener filtering module and a generative adversarial network (GAN). The Wiener filtering module is located at the input of the generator in the GAN. The Wiener filtering module filters the frequency domain image, quickly suppressing linear motion blur in known directions and improving the initial deblurring effect. Then, the generator in the GAN further restores the remaining details, generating a more natural and clear image. This reduces artifact generation while maintaining the deblurring effect, improving the clarity and visual effect of the TDI-CCD image. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 A flowchart illustrating a TDI-CCD image artifact removal method according to an embodiment of this application;

[0040] Figure 2 A detailed flowchart illustrating a TDI-CCD image artifact removal method provided in an embodiment of this application;

[0041] Figure 3 This is a schematic diagram of the working environment of a TDI-CCD camera provided in an embodiment of this application;

[0042] Figure 4 This is a schematic diagram of the functional modules of a TDI-CCD image artifact removal device provided in an embodiment of this application;

[0043] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0044] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] In one exemplary embodiment, such as Figures 1-2 As shown, this application discloses a TDI-CCD image artifact removal method, which includes steps 101-103.

[0047] Step 101: Construct a deep deblurring network; the deep deblurring network includes a Wiener filtering module and a generative adversarial network, the Wiener filtering module is set at the input of the generator of the generative adversarial network; the Wiener filtering module is used to filter the frequency domain image.

[0048] Step 102: Train the deep deblurring network and use the trained deep deblurring network as the artifact removal model; during the training of the deep deblurring network, the point spread function and regularization parameters of the Wiener filter module are used as learnable network parameters.

[0049] Step 103: The image with line trailing shadow acquired from the TDI-CCD camera is processed using the artifact removal model to obtain an artifact-removed image.

[0050] The Wiener filtering module is a frequency domain Wiener filtering module.

[0051] This application adds a Wiener filter module to the input of the generator of a generative adversarial network, and uses the point spread function (PSF) and regularization parameters of the Wiener filter module as learnable network parameters, thereby learning the image deblurring operator and physical model parameters simultaneously in end-to-end training.

[0052] The core idea of ​​this application includes: the generative adversarial network is the DeblurGAN-v2 network, and a trainable Wiener filter module is introduced at the front end of the DeblurGAN-v2 generator to perform frequency domain preprocessing on the input blurred image and to suppress linear motion blur by using physical priors.

[0053] The PSF and regularization parameter α of the Wiener filter module are embedded as learnable parameters of the network. The PSF is optimized through backpropagation during training based on the initial geometry settings of TDI-CCD imaging (approximately rectangular impulse response).

[0054] An end-to-end joint training framework was designed, with the generator adopting the DeblurGAN-v2 feature pyramid structure and the discriminator employing adversarial training, to recover image details in a way that combines perception and data-driven approaches.

[0055] The loss function combines pixel reconstruction loss and adversarial loss to guide the network output to be consistent with the real clear image. At the same time, the network parameters of the Wiener filtering module adapt to different degrees of ghosting during training.

[0056] In an exemplary embodiment, step 102 specifically includes steps 201-202.

[0057] Step 201: Freeze the generative adversarial network and pre-train the Wiener filtering module to obtain the pre-trained deep deblurring network.

[0058] Step 202: Jointly train the pre-trained deep deblurring network using the training set, and use the trained deep deblurring network as the artifact removal model; each sample data in the training set includes a pair of sample images and label data, the sample images are TDI-CCD blurred images, and the label data are TDI-CCD clear images.

[0059] A blurred TDI-CCD image is an image with line trailing artifacts acquired by a TDI-CCD camera. The operating state of the TDI-CCD camera is as follows: Figure 3 As shown, the electric conveyor belt 3 provides relative motion, and the TDI-CCD camera 1 is mounted above the electric conveyor belt 3 via the profile support frame 2.

[0060] Specifically, this application uses a precision-controlled electric conveyor belt system with a TDI-CCD line scan camera as the imaging device. The TDI-CCD camera 1 is fixed to the profile support frame 2, and a relative motion state is formed between it and the moving target object on the electric conveyor belt 3, thus acquiring a blurred TDI-CCD image with line trailing. The clear TDI-CCD image is the image of the target object captured when the electric conveyor belt 3 is not moving.

[0061] In one exemplary embodiment, step 202 specifically includes:

[0062] Step 301: For each sample image, a two-dimensional Fast Fourier Transform (FFT) is performed to convert the spatial domain sample image to a frequency domain image; the sample image is a TDI-CCD sample image with line trailing. The transformed frequency domain image is F(u,v). This transformation makes the vertical linear trailing correspond to a regular amplitude spectrum in the frequency domain, which is convenient for subsequent filtering processing.

[0063] Step 302: Input the frequency domain image into the Wiener filtering module. The Wiener filtering module weights the frequency domain image to obtain the filtered frequency domain image.

[0064] Step 303: Perform a two-dimensional inverse fast Fourier transform on the filtered frequency domain image to obtain a filtered spatial domain sample image.

[0065] The filtered frequency domain image The image is transformed back to the spatial domain by a two-dimensional inverse Fourier transform (IFFT) to obtain the preliminary deblurred image. At this point, the line blur of the input image is significantly reduced, but some blur and noise still remain.

[0066]

[0067] Step 304: Input the filtered spatial domain sample image into the generative adversarial network. The generator outputs the motion-blurred sample image, and the discriminator judges the generator's output as real or fake.

[0068] In an exemplary embodiment, the parameter initialization of the point spread function of the Wiener filter module includes:

[0069] (1) The point spread function is specifically a two-dimensional point spread function. In the spatial domain, the two-dimensional point spread function h(x,y) is defined, which is zero only in the vertical (row) direction, and its initial shape is a rectangular pulse:

[0070]

[0071] Where h(x,y) is the point spread function, x is the horizontal coordinate in the spatial domain, y is the vertical coordinate in the spatial domain, and L is the number of trailing lines. The two-dimensional point spread function utilizes the linear geometric characteristics of the TDI-CCD trailing effect, which helps stabilize the initial training.

[0072] (2) Frequency domain transformation: Perform a two-dimensional fast Fourier transform on h(x,y) to obtain:

[0073] H(u,v) = FFT{h(x,y)};

[0074] Where h(x,y) is the point spread function, x is the horizontal coordinate in the spatial domain, y is the vertical coordinate in the spatial domain, and FFT{} is the two-dimensional fast Fourier transform. The FFT length is consistent with the input image, and zero padding is used to ensure size matching. During network training, H(u,v) is stored as nn.Parameter in the form of real part plus imaginary part, and is updated during network training.

[0075] In an exemplary embodiment, the regularization coefficient α is modeled and initialized.

[0076] (1) The physical meaning of α.

[0077] The power spectral ratio of the approximate α signal to the noise is K = S N / S XThe estimate is used to balance the deconvolution gain and noise amplification in the frequency domain.

[0078] (2) Parameterization of α

[0079] α is defined as a single positive scalar. Furthermore, considering the signal-to-noise ratio differences in different frequency bands, α is parameterized as a frequency spectrum with the same dimension as H(u,v), which is also stored as nn.Parameter and updated during network training.

[0080] The scalar α is initialized to 0.05-0.1 to handle images captured with high noise (high ISO of TDI-CCD).

[0081] In one exemplary embodiment, a Wiener filtering module G(u,v) is constructed.

[0082]

[0083] During network training, the Adam optimizer is used to update H(u,v) and α simultaneously. A small learning rate (1 / 5 of the generator's rate) is set for the complex parameters to prevent excessive oscillations. α is frozen in the early stages of training to limit its range and ensure a positive value, followed by open training to enhance adaptability.

[0084] In an exemplary embodiment, a trainable Wiener filtering module is applied to the frequency domain image F(u,v) for weighting. The filtered frequency domain image in step 302 is represented as:

[0085]

[0086] in, Let F(u,v) be the filtered frequency domain image, G(u,v) be the Wiener filter module, α be the regularization coefficient, and H(u,v) be the weight function of the Wiener filter module in the frequency domain, i.e., the point spread function spectrum of the Wiener filter module. * indicates complex conjugate operation, |H(u,v)| 2 The energy spectrum H(u,v)H * (u,v), |H(u,v)| 2 The response intensity of this frequency component is measured, where u is the spatial frequency domain component in the horizontal direction and v is the spatial frequency domain component in the vertical direction. Initially, H(u,v) is set according to the line blur characteristics of the TDI-CCD image (the PSF is initialized as a rectangular function whose length is proportional to the number of sensor rows or the speed ratio), and α can take a small positive value. During training, the filter is updated through the backpropagation algorithm to adapt to different degrees of blur. This step uses a known physical model to perform linear deconvolution on the image, removing most of the directional blur and noise effects in advance.

[0087] In an exemplary embodiment, step 304 inputs the filtered spatial domain sample image into the generative adversarial network for joint training.

[0088] (1) Generator.

[0089] The DeblurGAN-v2 generator employs a Relativistic GAN framework and incorporates a Feature Pyramid Network (FPN) into the deblurring task to achieve feature fusion and recovery across multiple scales. In this application, the generator's input is no longer the original blurred image, but rather the pre-filtered image output from the Wiener filtering module. The steps are as follows:

[0090] 1) Feature extraction.

[0091] Multi-scale feature maps {C1,C2,…,Cn} are extracted through a series of convolutional and downsampling layers in the backbone network (Inception-ResNet-v2).

[0092] 2) Top-down fusion.

[0093] Using the FPN structure, upsampling branches are inserted from top to bottom starting from the deepest layer and added to the corresponding lateral connections (shallow features after 1×1 convolutions) to generate a set of fused features {Pn, Pn-1, ..., P1}, where Pn, Pn-1, and P1 are the nth, (n-1th), and 1st fused features, respectively. FPN enables the generator to simultaneously focus on large-scale structure and fine-grained texture.

[0094] 3) Decoding and reconstruction.

[0095] The fused multi-scale features are gradually restored to the input resolution through several deconvolution or interpolation + convolution modules, and the generator image is finally output.

[0096] (2) Discriminator.

[0097] The generator in the DeblurGAN-v2 network described in this application is a dual-scale relative discriminator, which includes a global discriminator and a local discriminator. The global discriminator is used to determine the realism of the entire image, while the local discriminator (70×70 pixels) is used to determine the local texture quality of image patches. When jointly training the pre-trained deep deblurring network using the training set, both the global discriminator and the local discriminator employ the RaLSGAN loss function, causing the discriminator output to represent the relative probability that "the real image is more real than the fake image." The total loss value is the sum of the RaLSGAN loss values ​​of the global discriminator and the local discriminator (local patch discriminator).

[0098] The discriminator implementation steps are as follows:

[0099] 1) Input pair construction.

[0100] For each pre-filtered result (the sample image in the spatial domain after filtering) Generator output Paired with the original, clear image (TDI-CCD clear image) I, two types of input pairs are formed: and

[0101] 2) Forward propagation.

[0102] The input pair is fed into the global discriminator D. G Output and And simultaneously fed into the local discriminator D P Output and A global discriminator ensures structural consistency, while a local discriminator enhances texture details.

[0103] 3) Relative counter-loss.

[0104] For the global discriminator, calculate the following RaLSGAN loss.

[0105]

[0106] in, E represents the loss of the global discriminator.

[0107] Loss of local discriminator Similarly The calculation.

[0108] 4) Optimize the discriminator.

[0109] right Calculate the gradient and update the discriminator parameters to enhance the discriminator's sensitivity to subtle differences in detail, thereby prompting the generator to focus on detail repair.

[0110] (4) Construction and optimization of loss function.

[0111] The generator is trained using the following hybrid loss:

[0112] L G =λ adv L adv +λ MSE L MSE +λ perc L perc .

[0113] Among them, L advTo combat the loss, a relative RaLSGAN approach is adopted, with multi-scale discriminators in parallel constraints, to improve visual realism and artifact removal capabilities. MSE , λ perc and λ adv All are weighting coefficients.

[0114]

[0115] After that, L MSE Represents pixel reconstruction loss, ensuring the proximity of the generated image to the real, clear image in pixel space, and guaranteeing high peak signal-to-noise ratio (PSNR) and structural similarity (SSIM).

[0116]

[0117] L perc Representing perceptual loss, based on features extracted from the VGG19 network at the conv3×3 layer, it enhances high-level semantic consistency and detail quality.

[0118]

[0119] φ represents the feature map extracted by the pre-trained VGG-19 network at the conv3×3 layer. First, ... I and I are fed into the VGG-19 network to obtain high-dimensional feature maps. Then, the error of the feature maps is calculated to emphasize the recovery of the image's structure and texture information.

[0120] Regarding the weighting coefficient settings, λ is set. MSE =50, λ perc =0.01, λ adv =1.

[0121] (5) Joint training.

[0122] The aforementioned deep deblurring network was trained end-to-end using gradient descent (Adam optimizer) on a large number of paired blurred TDI-CCD images and corresponding sharp images. During training, the parameters (PSF spectrum and α) of the Wiener filter module and the weights of the DeblurGAN-v2 network were updated together. Through iterative optimization, the Wiener filter module and the generator cooperated to achieve the best deblurring effect in the final output image.

[0123] The training process is divided into a pre-training phase (Epochs 1-5) and a joint training phase (Epochs 6-N). In the pre-training phase, only the PSF and α and L in the Wiener filter module are optimized. MSEFreeze other parameters of the generator and discriminator. During the joint training phase, optimize the above L end-to-end. G With discriminator loss Use the Adam optimizer.

[0124] Experiments have shown that combining the traditional Wiener filtering module with a deep adversarial network can significantly reduce artifacts and improve the deblurring quality of TDI-CCD ghosting.

[0125] This application achieves efficient deblurring of line trailing artifacts in TDI-CCD images. The method combines the advantages of physical imaging models and deep learning models: the Wiener filtering module can quickly suppress linear trailing artifacts in known directions, improving the initial deblurring effect; the subsequent GAN generator further restores remaining details, generating a more natural and clear image. Compared with traditional methods, this application reduces artifact generation while maintaining the deblurring effect, improving image reconstruction quality and visual effect.

[0126] Based on the same inventive concept, this application also provides a TDI-CCD image artifact removal apparatus for implementing the TDI-CCD image artifact removal method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations of one or more TDI-CCD image artifact removal apparatus embodiments provided below can be found in the limitations of the TDI-CCD image artifact removal method described above, and will not be repeated here.

[0127] In one exemplary embodiment, such as Figure 4 As shown, a TDI-CCD image artifact removal device is provided. The TDI-CCD image artifact removal device applies the aforementioned TDI-CCD image artifact removal method. The TDI-CCD image artifact removal device includes:

[0128] A deep deblurring network construction module is used to construct a deep deblurring network; the deep deblurring network includes a Wiener filtering module and a generative adversarial network, the Wiener filtering module is set at the input end of the generator of the generative adversarial network; the Wiener filtering module is used to filter the frequency domain image.

[0129] A deep deblurring network training module is used to train the deep deblurring network and use the trained deep deblurring network as an artifact removal model. During the training of the deep deblurring network, the point spread function and regularization parameters of the Wiener filter module are used as learnable network parameters.

[0130] The artifact removal model application module is used to process images with line trailing shadows acquired from a TDI-CCD camera using the artifact removal model to obtain an artifact-removed image.

[0131] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 5 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores TDI-CCD image artifact removal data. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a TDI-CCD image artifact removal method.

[0132] Those skilled in the art will understand that Figure 5 The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0133] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0134] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0135] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0136] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0137] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units, etc., and are not limited to these.

[0138] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0139] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for removing artifacts from TDI-CCD images, characterized in that, The TDI-CCD image artifact removal method includes: A deep deblurring network is constructed; the deep deblurring network includes a Wiener filtering module and a generative adversarial network, the Wiener filtering module is set at the input end of the generator of the generative adversarial network; the Wiener filtering module is used to filter the frequency domain image; The deep deblurring network is trained and used as an artifact removal model. During the training of the deep deblurring network, the point spread function and regularization parameters of the Wiener filter module are used as learnable network parameters. The image with line ghosting acquired from the TDI-CCD camera is processed using the aforementioned artifact removal model to obtain an artifact-free image; Training the deep deblurring network and using the trained deep deblurring network as an artifact removal model specifically includes: Freeze the generative adversarial network and pre-train the Wiener filtering module to obtain a pre-trained deep deblurring network. The pre-trained deep deblurring network is jointly trained using a training set, and the trained deep deblurring network is used as an artifact removal model. Each sample data in the training set includes a pair of sample images and label data. The sample images are TDI-CCD blurred images, and the label data are TDI-CCD clear images. Specifically, the pre-trained deep deblurring network is jointly trained using a training set, and the trained deep deblurring network is used as the artifact removal model. This includes: For each sample image, a two-dimensional fast Fourier transform is performed on the sample image to convert the spatial domain sample image into a frequency domain image; the sample image is a TDI-CCD sample image with line trailing. Input the frequency domain image into the Wiener filtering module to obtain the filtered frequency domain image; The filtered frequency domain image is subjected to a two-dimensional inverse fast Fourier transform to obtain a filtered spatial domain sample image. The filtered spatial domain sample image is input into the generative adversarial network. The generator outputs a motion-blurred sample image, and the discriminator judges the generator's output as real or fake. Specifically, the frequency domain image is input into the Wiener filtering module to obtain the filtered frequency domain image, which is represented as follows: ； in, This is the filtered frequency domain image. For frequency domain images, For Wiener filtering modules, The regularization coefficient is . This is the spectrum of the point spread function of the Wiener filter module. This represents the complex conjugate operation. Represents the energy spectrum , The spatial frequency domain component in the horizontal direction. The spatial frequency domain component is perpendicular to the horizontal direction; The point spread function spectrum of the Wiener filter module is expressed as follows: ； ； in, Let be the point spread function. For the horizontal coordinates of the spatial domain, For the spatial domain vertical coordinates, For two-dimensional fast Fourier transform, L The number of lines in the trailing image.

2. The TDI-CCD image artifact removal method according to claim 1, characterized in that, The generative adversarial network is a DeblurGAN-v2 network, and the generator in the DeblurGAN-v2 network is a dual-scale relative discriminator, which includes a global discriminator and a local discriminator. When the pre-trained deep deblurring network is jointly trained using the training set, both the global discriminator and the local discriminator use the RaLSGAN loss function, and the total loss value is the sum of the RaLSGAN loss values ​​of the global discriminator and the local discriminator.

3. A TDI-CCD image artifact removal device, characterized in that, The TDI-CCD image artifact removal device applies the TDI-CCD image artifact removal method according to any one of claims 1-2, and the TDI-CCD image artifact removal device comprises: A deep deblurring network construction module is used to construct a deep deblurring network; the deep deblurring network includes a Wiener filtering module and a generative adversarial network, the Wiener filtering module is set at the input end of the generator of the generative adversarial network; the Wiener filtering module is used to filter the frequency domain image; A deep deblurring network training module is used to train the deep deblurring network and use the trained deep deblurring network as an artifact removal model. During the training of the deep deblurring network, the point spread function and regularization parameters of the Wiener filter module are used as learnable network parameters. The artifact removal model application module is used to process images with line trailing shadows acquired from a TDI-CCD camera using the artifact removal model to obtain an artifact-removed image.

4. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the TDI-CCD image artifact removal method according to any one of claims 1-2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the TDI-CCD image artifact removal method according to any one of claims 1-2.

6. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the TDI-CCD image artifact removal method according to any one of claims 1-2.

Images