A pre-processing method for image thick cloud removal in high-performance video coding application

Abstract

The present application relates to a kind of image thick cloud removal pre-processing method for high-performance video coding application, it is related to image processing technical field, solve the technical problem that it is difficult to fully capture cross-scale complementary features, there is deficiency in the balance between global structure and local detail.The pre-processing method includes: based on the guided image estimation of non-subsampled shearlet transform and the cloud removal model of full connection tensor decomposition guided by non-subsampled shearlet transform to carry out multi-time remote sensing image thick cloud removal.The present application is an image thick cloud removal pre-processing method for high-performance video coding application, realizes the generation of high-quality reference image: to obtain the reference image containing more details, first, the multi-time remote sensing image is carried out gamma correction, to normalize radiation intensity;Subsequently, improved adaptive PCNN fusion algorithm is applied in NSST domain, thereby enhancing detail representation while maintaining image energy distribution, finally, high-quality reference image is obtained.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a preprocessing method for removing thick clouds from images in high-performance video coding applications. Background Technology

[0002] Optical remote sensing imagery plays a crucial role in numerous fields due to its rich spatial and spectral information. For example, high-resolution satellite imagery provides reliable data support for applications such as refined urban mapping, land cover classification, and change detection, while multispectral imagery supports quantitative analysis in precision agriculture and ecological monitoring. However, in practical applications, thick cloud cover often leads to significant data loss, thereby reducing the reliability of related applications. This problem is particularly prominent in tasks requiring high-precision surface reflectance data, such as vegetation index calculation and target identification, where cloud contamination can cause significant measurement errors. Therefore, developing algorithms that can effectively remove thick cloud interference and accurately recover true surface reflectance is of significant practical importance for enhancing the application value of optical remote sensing data.

[0003] Methods for removing thick clouds from remote sensing images can be broadly categorized into blind and non-blind methods. Blind methods do not require externally provided cloud masks; instead, they directly infer the cloud locations from the observed images and perform the removal. For example, Guo et al. proposed a cloud perception network based on Fast Fourier Convolution [Y. Guo, W. He, Y. Xia, and H. Zhang, “Blindsingle-image-based thin cloud removal using a cloud perception integratedfast fourier convolutional network,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 206, pp. 63–86, 2023.]. This method utilizes a frequency domain attention mechanism to detect clouds globally, thus achieving cloud removal without prior information. Chen et al. decomposed observed images into low-rank components and structurally sparse cloud components, and then proposed a model-driven thick cloud removal method [Y. Chen, M. Chen, W. He, J. Zeng, M. Huang, and Y.-B. Zheng, “Thick cloud removal in multitemporal remote sensing images via low-rank regularized self-supervised network,” IEEE Transactions on Geoscience and Remote Sensing, vol. 62, pp. 1–13, 2024]. Lin et al. introduced sparse components and combined them with a thresholding strategy to detect cloud masks [J. Lin, T. Huang, X. Zhao, M. Ding, Y. Chen, and T. Jiang, “A blind cloud / shadow removal strategy for multi-temporal remote sensing images,” in 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, 2021, pp. 4656–4659]. Although these methods do not rely on external masks in form, most of them explicitly or implicitly integrate cloud detection mechanisms within the algorithm, thus being conceptually compatible with non-blind methods.Non-blinded methods rely on external cloud mask information, thus enabling more reliable guidance of the repair process. Based on the type of information they primarily utilize, non-blinded methods can be further categorized into spatial domain methods, temporal domain methods, spectral domain methods, and hybrid methods.

[0004] Spatial-based methods utilize spatial autocorrelation and local continuity to infer the content of cloud-covered areas, such as spatial interpolation, partial differential equation methods, and total variational methods. These methods perform well in handling localized cloud contamination, but often struggle to recover details when large-scale cloud cover exists in a single image acquisition. Temporal-based methods, on the other hand, utilize multi-temporal cloud-free images to compensate for missing spatial information. For example, Ebel et al. proposed a stitching method based on neighboring temporal phase replacement [P. Ebel, Y. Xu, M. Schmitt, and XX Zhu, “Sen12ms-cr-ts: A remote-sensing data set for multimodal multitemporal cloud removal,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–14, 2022], which is suitable for scenarios with small short-term changes. Gao et al. developed a spatiotemporally adaptive reflectance fusion model [F. Gao, J. Masek, M. Schwaller, and F. Hall, “On the blending of the landsat and modissurface reflectance: predicting daily landsat surface reflectance,” IEEE Transactions on Geoscience and Remote Sensing, vol. 44, no. 8, pp. 2207–2218, 2006], which combines high-frequency temporal information from multi-source satellites with high-resolution spatial data. Chen et al., on the other hand, employed a spatiotemporally weighted regression method [B. Chen, B. Huang, L. Chen, and B. Xu, “Spatially and temporally weighted regression: A novel method to produce continuous cloud-ree landsat imagery,” IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 1, pp. 27–37, 2017], integrating complementary information from invariant similar pixels, thereby achieving lower reconstruction errors and higher stability. Spectral-based methods utilize the strong correlation between spectral bands to reconstruct cloud-covered areas.Meng et al. [M. Xu, X. Jia, M. Pickering, and S. Jia, “Thin cloud removal from optical remotesensing images using the noise-adjusted principal component transform,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 149, pp. 215–225, 2019] pointed out that when the satellite revisit period is long, there may be significant differences between the target image and the reference image, in which case multispectral methods are more effective. They proposed a noise-adjusted principal component transform method that simultaneously introduces spatial and multispectral features into cloud signal-to-noise ratio estimation, which is suitable for thin cloud removal. Shen et al. [H. Shen, X. Li, L. Zhang, D. Tao, and C. Zeng, “Compressed sensing-based inpainting of aqua moderateresolution imaging spectroradiometer band 6 using adaptive spectrum-weighted sparse bayesian dictionary learning,” IEEE Transactions on Geoscience and Remote Sensing, vol. 52, no. 2, pp. 894–906, 2014] proposed an adaptive spectrum-weighted sparse Bayesian dictionary learning method based on compressed sensing theory, which effectively recovered the 6th band of Aqua MODIS. However, this method failed when all bands were covered by clouds in a single acquisition.

[0005] Hybrid methods, by combining spatial, spectral, and temporal features, can better capture the intrinsic correlations in multi-temporal data, thus significantly improving the accuracy of thick cloud removal. With the development of satellite technology, such as the Sentinel-2 remote sensing satellite with a five-day global revisit cycle, rich multidimensional data has been provided for hybrid methods, making their applications more feasible. Many hybrid methods employ matrix or tensor modeling strategies with low-rank and sparsity priors.For example, Deng et al. [X.-X. Deng, H. Sun, G. Xu, Y.-F. Yu, and H. Wang, “RLGC: residual low rank group sparsity constraint for image denoising,” in International Symposium on Artificial Intelligence and Robotics 2021, H. Lu, S. Mu, and S. Nakashima, Eds., vol. 11884, International Society for Optics and Photonics. SPIE, 2021, p. 118841E] enhanced sparsity by imposing a low-rank constraint on the sparse coefficient matrix of similar block groups, thereby achieving denoising; Zhang et al. [Q. Zhang, Q. Yuan, Z. Li, F. Sun, and L. Zhang, “Combined deepprior with low-rank tensor svd for thick cloud removal in multitemporalimages,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 177, pp. 161–173,]

[2021] Combining third-order tensor singular value decomposition with convolutional neural networks to extract deep spatiotemporal features for cloud removal; Liu et al. proposed a locally sensitive low-order tensor learning method based on Tucker decomposition [J. Liu, P. Musialski, P. Wonka, and J. Ye, “Tensor completion for estimating missing values ​​in visual data,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 1, pp. 208–220, 2013] to obtain a more natural approximation effect.

[0006] Furthermore, Zheng Y et al. proposed a fully connected tensor network decomposition method [Y.-B. Zheng, T.-Z. Huang, X.-L. Zhao, Q. Zhao, and T.-X. Jiang, “Fully-connected tensor network decomposition and its application to higher-order tensor completion,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no.12, pp. 11 071–11 078, May 2021], which can capture the correlation between any two tensor modes, providing a new perspective for tensor completion problems such as cloud removal. Zheng W et al. [W.-J. Zheng, X.-L. Zhao, Y.-B. Zheng, J. Lin, L. Zhuang, and T.-Z. Huang, “Spatial-spectral-temporalconnective tensor network decomposition for thick cloud removal,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 199, pp. 182–194, 2023] expanded multi-temporal data along the spectral dimension and applied fully connected tensor network decomposition, effectively improving computational speed and reducing color bias. Li et al. [L.-Y. Li, T.-Z. Huang, Y.-B. Zheng, W.-J. Zheng, J. Lin, G.-C. Wu, and X.-L. Zhao, “Thick cloud removal for multitemporal remote sensing images: When tensor ring decomposition meets gradient domain fidelity,” IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp. 1–14, 2023] introduced gradient domain fidelity-preserving terms into tensor ring decomposition to enhance edge and texture quality.Building on this, Zheng W et al. [W.-J.Zheng, X.-X. Bai, Y.-B. Zheng, Y.-R. Fan, T.-Z. Huang, and X.-L. Zhao, “Feature-domain fidelity and tensor low-rank regularization for cloud removal in remote sensing images,” in IGARSS 2024 – 2024 IEEE International Geoscience and Remote Sensing Symposium, 2024, pp. 8274–8277] further extended gradient domain fidelity to the feature domain, using the results of feature extraction operators to constrain fully connected tensor networks, thereby better capturing spatiotemporal correlations while preserving details and textures.

[0007] In recent years, deep learning methods have been increasingly applied to thick cloud removal tasks. These methods typically require large-scale training datasets, but obtaining high-quality, large-sample data is costly in the remote sensing field. Furthermore, deep learning methods still face challenges in generalization and interpretability; therefore, combining model-driven and data-driven approaches has potential advantages. For example, Mikolaj et al. proposed a method based on Deep Image Prior (DIP) [M. Czerkawski, P. Upadhyay, C. Davison, A. Werkmeister, J. Cardona, R. Atkinson, C. Michie, I. Andonovic, M. Macdonald, and C. Tachtatzis, “Deep internal learning for inpainting of cloud-affected regions in satellite imagery,” Remote Sensing, vol. 14, no. 6, 2022], which can automatically capture image features under unsupervised conditions and achieve cloud removal through internal learning. Zuo et al. [Z. Zou, L. Chen, and X. Jiang, “Spectral–temporal low-rank regularization with deep prior for thick cloud removal,” IEEE Transactions on Geoscience and Remote Sensing, vol. 62, pp. 1–16, 2024] introduced spectral–temporal low-rank regularization into a deep prior framework, effectively alleviating the overfitting problem.

[0008] Although some scholars have introduced gradient domains or feature domains with explicit physical meaning into tensor network models and achieved good results, these methods all rely on guide images. In existing methods, guide images are usually obtained from simple mean or linear regression results of cloudless phases, which are easily affected by local similarity interference.

[0009] Thick cloud contamination poses a significant challenge to the analysis and application of multi-temporal remote sensing images. Although existing thick cloud removal methods attempt to utilize spatiotemporal spectral correlations, they often fail to fully capture complementary features across scales, particularly in terms of balancing global structure and local details. This results in deficiencies in detail fidelity, color consistency, and robustness when reconstructing images from thick cloud-contaminated areas in multi-temporal remote sensing images. Summary of the Invention

[0010] The present invention aims to solve the technical problems in the prior art and provide a preprocessing method for thick cloud removal in images for high-performance video coding applications.

[0011] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0012] A preprocessing method for thick cloud removal in high-performance video coding applications is proposed. This method utilizes guided image estimation based on non-subsampled shear wave transform (NSFT) and a fully connected tensor decomposition (FTTD) cloud removal model guided by NFT. The method is a multi-temporal remote sensing image thick cloud removal approach.

[0013] Guided image estimation based on non-subsampled shear wave transform includes the following steps:

[0014] Step 1: Dataset decomposition;

[0015] Multi-time remote sensing images Divide the data into two mutually exclusive sets based on the time series:

[0016]

[0017] in, M Indicates spatial height, N Indicates the width of the space. B Indicates the number of spectral bands. T, T 1. T 2 both represent the length of the time series and T = T 1+ T 2, The abbreviation for "cloud" indicates the presence of cloud pollution. The abbreviation "free" indicates that it does not contain cloud pollution.

[0018] Step 2: Gamma correction;

[0019] For each cloud pollution image For all cloudless images Perform gamma correction independently. , Represents time series index and , Thus we get: ;

[0020] Step 3: Fusion of non-subsampled shear wave decomposition and parameter-adaptive pulse-coupled neural network;

[0021] For the gamma-corrected image generated in step two Performing non-subsampled shear wave decomposition on it yields:

[0022] ;

[0023] in, Represents the gamma transform coefficients. Indicates the low-frequency sub-band. Indicates the number of decomposition layers With direction The high-frequency subband below; NSST indicates non-subsampled shear decomposition; This represents a gamma-corrected image containing cloud contamination.

[0024] The merged subbands of low-frequency and high-frequency subbands are obtained according to the following rules:

[0025] ;

[0026] ;

[0027] in, Represents the pixels in the input image of a parameter-adaptive pulse-coupled neural network. A fixed grayscale value at a given location. m Indicates spatial height index, n Indicates the space width index. b Indicates the index of the number of spectral bands. , Represents a time series index; This represents the cumulative number of pulse triggers obtained in the current round during the iteration process. This represents the cumulative number of pulse triggers obtained in the previous round during the iteration process. This represents the output of the neuron in the current iteration of the adaptive pulse-coupled neural network. Indicates the iteration count index. express Along the index The image with the highest cumulative pulse trigger count in the direction is denoted by its index. , express Along the index direction and The image with the highest peak signal-to-noise ratio (PSNR) is denoted by the index of the highest SNR. , Indicates peak signal-to-noise ratio, This represents the set of indices for cloudless pixels. The projection operator is used to preserve the set. Set the elements within the specified range to zero, and set the elements at other positions to zero. This represents the merged image. This represents the inverse transform of the non-subsampled shear wave;

[0028] For high-frequency components, they are used as input to a parameter-adaptive pulse-coupled neural network, specifically... The number of pulse triggers accumulated during the iteration process is denoted as . Select for each pixel position The image pixel value with the highest number of pulse triggers is used as The image index with the highest number of pulse triggers is denoted as... For low-frequency components, select the position of each pixel. Zhongyu The image with the highest peak signal-to-noise ratio as The image index with the highest peak signal-to-noise ratio is denoted as Finally, the fused image was reconstructed through inverse NSST transform. ;

[0029] Subsequently, the images will be merged. The results are added to a clean image free of cloud contamination. In this way, an augmented, cloud-contaminated, clean multi-temporal remote sensing dataset is constructed. ;

[0030] Step 4: Least squares fitting;

[0031] Least square regression was used to establish the relationship between cloudless areas in images from different time periods, and the optimal fitting coefficients were determined to obtain the guiding image. ;

[0032] The fully connected tensor decomposition cloud removal model guided by non-subsampled shear wave transform is expressed as follows:

[0033] ;

[0034] in, Indicates the reconstructed cloudless image. This indicates images containing cloud pollution. for The four FCTN factors; For guiding images, FCTN represents Fully Connected Tensor Network Decomposition. Indicates the number of decomposition layers and direction The high-frequency subband coefficients below , These are the regularization parameters. Denotes the Frobenius norm. Indicates constraint terms;

[0035] In the above technical solution, in step two of guided image estimation based on non-subsampled shear wave transform, for each cloud contamination image... For all cloudless images Perform gamma correction independently, the specific process is as follows:

[0036] ;

[0037] in, Indicates a pair of images The specific gamma parameters obtained are calculated as follows:

[0038] ;

[0039] in, This indicates an arbitrary selection operation. This indicates the gamma transform operation. Represents the gamma transform parameters. Indicates peak signal-to-noise ratio. This represents the set of indices for cloudless pixels. The projection operator is used to preserve the set. Set the elements within the specified range to zero, and set the elements at other positions to zero.

[0040] Further, all sub-gamma correction results are then... Merge along the dimensions to obtain the same as Corresponding gamma-corrected image set In the same manner, gamma-corrected images were obtained for all cloud contamination time points. .

[0041] In the above technical solution, the optimization problem of determining the optimal fitting coefficients in step four of guided image estimation based on non-subsampled shear wave transform is expressed as follows:

[0042]

[0043] in, Representing an image and The radiation gain coefficient between For image The bias coefficient, The image in the multi-temporal remote sensing dataset containing cloud pollution corresponds to the index, with index number [index number missing]. , The image corresponding to the index in the augmented, cloud-free, clean multi-temporal remote sensing dataset is indexed as follows: , Denotes the Frobenius norm;

[0044] Based on the estimated parameters, a least-squares estimate of the cloud-polluted area is further generated, in the following form:

[0045]

[0046] in, This is a guide image showing the time points containing cloud pollution. The guide images from other time points without cloud pollution were directly taken. ;

[0047] Through constraints To preserve all cloudless areas in the original image, at the locations where cloudless pixels exist, The estimated region in the middle is replaced with The corresponding real value; the multi-time remote sensing images obtained at this point are the design guidance images.

[0048] In the above technical solution, the fully connected tensor decomposition cloud removal model guided by non-subsampled shear wave transform is solved using a proximal alternating minimization algorithm, including:

[0049] Under the PAM framework and Alternating updates:

[0050] ;

[0051] Among them, During the iteration process, For factor indexes of fully connected tensor networks, , Index for iteration count, Indicates the current round. Indicates the previous round. The factorization factor of the fully connected tensor network to be solved in the current iteration is assigned to the solution result. , This represents the solution result of the fully connected tensor network decomposition factor to be solved in the current iteration during the iteration process. This refers to the results of reconstructing multi-time remote sensing images using a fully connected tensor network from the previous iteration in the iterative process. This refers to the factorization factor of the fully connected tensor network that has been solved in the current iteration during the iteration process. For the factorization factor of the fully connected tensor network that has been solved in the previous iteration, These are proximal parameters used to control the update step size.

[0052] Among them, During the iteration process, Index for iteration count, Indicates the current round. Indicates the previous round. The solution is the reconstruction result of the fully connected tensor network to be solved in the current iteration during the iteration process, and the solution result is assigned to... , This represents the solution result of reconstructing multi-temporal remote sensing images using a fully connected tensor network in the current iteration during the iteration process. This refers to the reconstruction results of the fully connected tensor network solved in the previous iteration during the iterative process. This refers to the factorization factor of the fully connected tensor network that has been solved in the current iteration during the iteration process. To guide the image, These are proximal parameters used to control the update step size.

[0053] In the above technical solution, the specific steps for solving the fully connected tensor decomposition cloud removal model guided by non-subsampled shear wave transform include:

[0054] Initialize FCTN factor Initialized reconstructed cloudless image Number of iterations Maximum number of iterations ;

[0055] Step 1: Obtain the guide image: Obtained from the guided image estimation part based on non-subsampled shear wave transform;

[0056] Step 2: Determine if the current iteration count is the maximum iteration count. If it is, skip to step 9; otherwise, continue to the next step.

[0057] Step 3: Update factors based on fully connected tensor network decomposition theory ;

[0058] Step 4: Perform a non-subsampled shear wave transform on the guiding image obtained from the guiding image estimation part based on the non-subsampled shear wave transform. The number of non-subsampled shear wave decomposition layers is... Number of non-subsampled shear wave decomposition directions ;

[0059] Step 5: Factorize the updated fully connected tensor network Perform non-subsampled shear wave transform, and determine the number of non-subsampled shear wave decomposition layers. Number of non-subsampled shear wave decomposition directions ;

[0060] Step 6: Decomposition factor of the fully connected tensor network after non-subsampled shear wave transform An optimization model based on Frobenius norm constraints is established to update the guided image after non-subsampled shear wave transformation. ;

[0061] Step 7: Check if the convergence condition of the following expression is met. If it is met, end the loop; otherwise, continue to step 8.

[0062] ;

[0063] Step 8: Order Proceed to step two;

[0064] Step 9: End the loop;

[0065] Step 10: Update the current round As a reconstructed cloudless image .

[0066] The present invention has the following beneficial effects:

[0067] This invention presents a preprocessing method for thick cloud removal in high-performance video coding applications, achieving a comprehensive characterization of cross-scale correlations in multi-temporal images: It utilizes the multi-scale decomposition and translation invariance of non-subsampled shear wave transform (NSST) to guide a fully connected tensor network. Regularization constraints are applied at different scales, with high-frequency constraints enhancing texture and detail in the reconstructed region, while low-frequency constraints ensure color consistency. Simultaneously, translation invariance effectively maintains reconstruction accuracy even with slight registration errors in multi-temporal images.

[0068] The preprocessing method for thick cloud removal in high-performance video coding applications of the present invention achieves the generation of high-quality reference images: in order to obtain reference images containing more details, gamma correction is first performed on multi-temporal remote sensing images to normalize the radiance; then an improved adaptive PCNN fusion algorithm is applied in the NSST domain to enhance the detail representation while maintaining the energy distribution of the images, and finally a high-quality reference image is obtained. Attached Figure Description

[0069] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0070] Figure 1 This is a schematic diagram of the overall framework of the preprocessing method for thick cloud removal in high-performance video coding applications according to the present invention; wherein:

[0071] (a) Image containing clouds (b) Grouping of multi-temporal remote sensing datasets;

[0072] (c) Gamma correction; (d) NSST decomposition; (e) PA-PCNN fusion;

[0073] (f) Least squares fitting; (g) Estimation of cloudless remote sensing images of cloud-polluted areas;

[0074] (h) Guide image ;(i) NSST decomposition;

[0075] (j) Regularization constraints based on Frobenius norm; (k) NSST decomposition;

[0076] (l) FCTN decomposition; (m) Cloudless reconstruction results . Detailed Implementation

[0077] Definitions and Prerequisites

[0078] Definition 1 (K-mode expansion) [L.-Y. Li, T.-Z. Huang, Y.-B. Zheng, W.-J. Zheng, J.Lin, G.-C. Wu, and X.-L. Zhao, “Thick cloud removal for multitemporal remotesensing images: When tensor ring decomposition meets gradient domainfidelity,” IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp.1–14, 2023]: For a order tensor Its K-mode expansion result is a matrix: , where matrix elements With tensor elements The correspondence is given by the following mapping: Its inverse operation is called K-mode folding, which can reconstruct the matrix into the original tensor: .

[0079] Definition 2 (K-modal product) [L.-Y. Li, T.-Z. Huang, Y.-B. Zheng, W.-J. Zheng, J.Lin, G.-C. Wu, and X.-L. Zhao, “Thick cloud removal for multitemporal remotesensing images: When tensor ring decomposition meets gradient domainfidelity,” IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp.1–14, 2023]: For a order tensor and a matrix Their K-mode product is: The calculation method for each element is as follows: .

[0080] Definition 3 (FCTN decomposition) [Y.-B. Zheng, T.-Z. Huang, X.-L. Zhao, Q. Zhao, and T.-X. Jiang, “Fully-connected tensor network decomposition and its application to higher-order tensor completion,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 12, pp. 11 071–11 078, May 2021]: For a fourth-order tensor Its fully connected tensor network (FCTN) decomposition consists of four factor tensors: , , , Among them, let This is called the FCTN rank. The original tensor It can be reconstructed using the following formula:

[0081]

[0082] In this invention, FCTN decomposition is concisely represented as: ,in For complete mathematical details, please refer to the literature [Y.-B. Zheng, T.-Z. Huang, X.-L. Zhao, Q. Zhao, and T.-X. Jiang, “Fully-connected tensor network decomposition and its application to higher-order tensor completion,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 12, pp. 11 071–11078, May 2021].

[0083] Definition 4 (Nonsubsampled Shearlet Transform (NSST) [G. Easley, D. Labate, and W.-Q. Lim, “Sparse directional image representations using the discrete shearlet transform,” Applied and Computational Harmonic Analysis, vol. 25, no. 1, pp.25–46, 2008]): The Nonsubsampled Shearlet Transform (NSST) is a multi-scale geometric analysis method that combines strong direction sensitivity and translation invariance, making it highly suitable for remote sensing image analysis. Given an input image matrix... ,That layer, The NSST decomposition of the direction is defined as follows:

[0084]

[0085] in, Indicates passage Low-frequency approximation obtained by layer-by-layer non-subsampled pyramid filtering; Indicated in scale With direction Below, the directional details are obtained through translation-invariant shearlet filtering. For more complete mathematical details, please refer to the literature [G. Easley, D. Labate, and W.-Q. Lim, “Sparsedirectional image representations using the discrete shearlet transform,” Applied and Computational Harmonic Analysis, vol. 25, no. 1, pp. 25–46, 2008].

[0086] Definition 5 (Parameter-Adaptive Pulse-Coupled Neural Network (PA-PCNN) Model) [M. Yin, X. Liu, Y. Liu, and X. Chen, “Medical image fusion with parameter-adaptive pulse coupled neural network in nonsubsampled shearlet transform domain,” IEEE Transactions on Instrumentation and Measurement, vol. 68, no. 1, pp. 49–64, 2019]: The Parameter-Adaptive Pulse-Coupled Neural Network (PA-PCNN) is a model based on iterative computation that requires no training process. Its definition is as follows:

[0087]

[0088]

[0089]

[0090] In this model: For pixels in the input image Fixed grayscale value, Indicates the previous iteration The outputs of eight neurons in the neighborhood are processed by a weight matrix. Connected to the current neuron, This represents the internal activity value of the neuron. This represents the dynamic threshold of the neuron. The model contains four key parameters: (Coupling strength of eight neighborhoods). ( (attenuation rate) ( (attenuation rate) (In update) hour (coupling strength).

[0091] The model's output functions similarly to a pulse generator. Its binary output... Indicates the state of the neuron: 0 for not triggered, 1 for triggered. Exceeding the threshold of the previous iteration When the neuron is triggered, the calculation method and complete mathematical details of the relevant adaptive parameters can be found in the literature [M. Yin, X. Liu, Y. Liu, and X. Chen, “Medical image fusion with parameter-adaptive pulse coupled neural network in nonsubsampled shearlet transform domain,” IEEE Transactions on Instrumentation and Measurement, vol. 68, no. 1, pp. 49–64, 2019].

[0092] The present invention will now be described in detail with reference to the accompanying drawings.

[0093] Multi-temporal remote sensing images Containing highly correlated spatial, spectral, and temporal information, it can be represented as a fourth-order tensor. ,in, Indicates spatial height, Indicates the width of the space. Indicates the number of spectral bands. Indicates the length of the time series.

[0094] This invention proposes a preprocessing method for thick cloud removal in images for high-performance video coding applications (its overall framework is as follows). Figure 1 As shown in the figure, it is a method for removing thick clouds from multi-temporal remote sensing images based on non-subsampled shear wave transform (NSST) guided fully connected tensor network (FCTN) decomposition.

[0095] Specifically:

[0096] This invention proposes an adaptive parametric pulse-coupled neural network feature fusion method for multi-temporal remote sensing images in the non-subsampled shear wave transform domain. Although the adaptive parametric pulse-coupled neural network feature fusion method has been used in medical image fusion, the method of this invention adds gamma correction and extends the application scenario to multi-temporal remote sensing images.

[0097] This invention proposes a non-sampling shear wave transform-guided fully connected tensor network decomposition method. Although the fully connected tensor network decomposition method has been used for thick cloud removal in multi-temporal remote sensing images, the method of this invention adds a step to reconstruct the thick cloud contamination area image by using a guided image constrained fully connected tensor network in the decomposed sub-bands.

[0098] FCTN decomposition utilizes strongly correlated internal similarities in multi-temporal remote sensing images to reconstruct areas contaminated by thick clouds. However, the similarities obtained by such methods are usually limited to spatial correlation at the macroscopic scale and single-scale representation, and they fail to distinguish the importance of low-frequency and high-frequency components during the restoration process. This results in low-frequency regions (containing the main energy of the image) being more easily reconstructed, while high-frequency detail information is inevitably lost. To address this issue, this invention employs NSST to decompose the spatial domain into multi-frequency, multi-directional subbands and applies scale-specific constraints to FCTN at multiple frequency scales, thereby improving reconstruction accuracy at fine scales. The constraints guided by NSST can be expressed as:

[0099] (1)

[0100] in, This indicates cloud contamination images awaiting reconstruction. Indicates a guide image. Indicates the number of decomposition layers With direction The high-frequency subband coefficients are obtained. This model fully explores the intrinsic correlations of various dimensions of multi-temporal data, while effectively preserving texture and structural details by adding a guided image regularization term constraint under multi-scale decomposition.

[0101] The preprocessing method for thick cloud removal in high-performance video coding applications of the present invention can be generally divided into two parts: guided image estimation based on non-subsampled shear wave transform and a fully connected tensor decomposition cloud removal model guided by non-subsampled shear wave transform.

[0102] Specifically:

[0103] 1. Guided image estimation based on non-subsampled shear wave transform;

[0104] This section presents a method for estimating the guide image. (Guide image) It needs to maintain a strong similarity to cloud-polluted images while preserving rich spatial details.

[0105] Step 1: Dataset Decomposition (corresponding to) Figure 1 (b));

[0106] In multi-temporal remote sensing images, even if there is severe thick cloud contamination at certain time points, cloud-free images can usually still be obtained at other time points. Based on this prior knowledge, multi-temporal remote sensing images... Divide the data into two mutually exclusive sets based on the time series:

[0107] (2)

[0108] in, M Indicates spatial height, N Indicates the width of the space. B Indicates the number of spectral bands. T, T 1. T 2 both represent the length of the time series and T = T 1+ T 2, The abbreviation for "cloud" indicates the presence of cloud pollution. The abbreviation "free" indicates that it does not contain cloud pollution.

[0109] After grouping, the following three steps are used to generate the guide image. .

[0110] Step 2: Gamma correction (corresponding to) Figure 1 (c));

[0111] To mitigate the illumination inconsistency issue during the acquisition of multi-time remote sensing images, a dedicated guiding image is designed for each cloud contamination image. Specifically, for each cloud contamination image... All cloudless images need to be processed. Perform gamma correction independently. , Represents time series index and , Thus we get: The process can be represented as follows:

[0112] (3)

[0113] in, Indicates a pair of images The specific gamma parameters obtained are calculated as follows:

[0114] (4)

[0115] in, This indicates an arbitrary selection operation. This indicates the gamma transform operation. Represents the gamma transform parameters. Indicates peak signal-to-noise ratio. This represents the set of indices for cloudless pixels. The projection operator is used to preserve the set. Set the elements within the specified range to zero, and set the elements in other positions to zero.

[0116] Further, all sub-gamma correction results are then... Merging along dimensions yields the same result as... Corresponding gamma-corrected image set In the same way, gamma-corrected images for all cloud contamination time points can be obtained. .

[0117] Step 3: Fusion of non-subsampled shear wave decomposition and parameter-adaptive pulse-coupled neural network (corresponding to...) Figure 1 (d) Figure 1 (e));

[0118] For the gamma-corrected image generated in step two Performing non-subsampled shear wave decomposition on it yields:

[0119] (5)

[0120] in, Represents the gamma transform coefficients. Indicates the low-frequency sub-band. Indicates the number of decomposition layers With direction The high-frequency subband below, NSST represents non-subsampled shear decomposition; This represents a gamma-corrected image containing cloud contamination.

[0121] The merged subbands of low-frequency and high-frequency subbands are obtained according to the following rules:

[0122] (6)

[0123] (7)

[0124] in, Represents the pixels in the input image of a parameter-adaptive pulse-coupled neural network. A fixed grayscale value at a given location. m Indicates spatial height index, n Indicates the space width index. b Indicates the index of the number of spectral bands. , Represents a time series index; This represents the cumulative number of pulse triggers obtained in the current round during the iteration process. This represents the cumulative number of pulse triggers obtained in the previous round during the iteration process. This represents the output of the neuron in the current iteration of the adaptive pulse-coupled neural network. Indicates the iteration count index. express Along the index The image pixel value with the highest cumulative pulse trigger count in the direction is denoted as [index of the highest trigger count]. , express Along the index direction and The image with the highest peak signal-to-noise ratio (PSNR) is denoted by the index of the highest SNR. , Indicates peak signal-to-noise ratio, This represents the set of indices for cloudless pixels. The projection operator is used to preserve the set. Set the elements within the specified range to zero, and set the elements at other positions to zero. This represents the merged image. This represents the inverse transform of the non-subsampled shear wave.

[0125] For high-frequency components, they are used as input to a parameter-adaptive pulse-coupled neural network, specifically... The number of pulse triggers accumulated during the iteration process is denoted as . Select for each pixel position The image pixel value with the highest number of pulse triggers is used as The image index with the highest number of pulse triggers is denoted as... For low-frequency components, select the position of each pixel. Zhongyu The image pixel value with the highest peak signal-to-noise ratio is used as The image index with the highest peak signal-to-noise ratio is denoted as Finally, the fused image was reconstructed through inverse NSST transform. ;

[0126] Subsequently, the images will be merged. The results are added to a clean image free of cloud contamination. In this way, an augmented, cloud-contaminated, clean multi-temporal remote sensing dataset is constructed. .

[0127] Step 4: Least squares fitting (corresponding to) Figure 1 (f));

[0128] Least square regression was used to establish the relationship between cloudless areas in images from different time periods, and the optimal fitting coefficients were determined to obtain the guiding image. Its optimization problem can be expressed as:

[0129] (8)

[0130] in, Representing an image and The radiation gain coefficient between For image The bias coefficient, The image in the multi-temporal remote sensing dataset containing cloud pollution corresponds to the index, with index number [index number missing]. , The image corresponding to the index in the augmented, cloud-free, clean multi-temporal remote sensing dataset is indexed as follows: , This represents the Frobenius norm.

[0131] Based on the estimated parameters, a least squares estimate is further generated for the cloud-polluted area. Figure 1 (g)), which takes the following form:

[0132] (9)

[0133] in, This is a guide image showing the time points containing cloud pollution. The guide images from other time points without cloud pollution were directly taken. To preserve the original radiation information to the greatest extent possible, constraints are used. To preserve all cloudless areas in the original image, this operation will, at the location where cloudless pixels exist, The estimated region in the middle is replaced with The corresponding real values ​​are used to obtain the multi-time remote sensing images, which are then used as the design guidance images. Figure 1 (h)).

[0134] Summary: The guided image estimation process based on non-subsampled shear wave transform is summarized as shown in Algorithm 1 in the table below.

[0135]

[0136] 2. A fully connected tensor decomposition cloud removal model guided by non-subsampled shear wave transform;

[0137] Based on the Non-Subsampled Shear Wave Transform (NSST) guided framework, this invention proposes an improved Fully Connected Tensor Network (FCTN) decomposition model for thick cloud removal in multi-temporal remote sensing images. Its expression is as follows:

[0138] (10)

[0139] in, Indicates the reconstructed cloudless image. This indicates an image containing thick cloud pollution. for The four FCTN factors. For guiding purposes, FCTN stands for Fully Connected Tensor Network Decomposition (see Definition 3 in the Definitions and Preliminary Knowledge section for details). Indicates the number of decomposition layers and direction The high-frequency subband coefficient (see Definition 4 in the Definitions and Preliminary Knowledge section for a detailed definition), and and The definition is shown in equation (4). The present invention uses the Proximal Alternating Minimization (PAM) algorithm to solve the model. , These are the regularization parameters. Denotes the Frobenius norm. Indicates a constraint term.

[0140] Under the PAM framework and Alternating updates:

[0141] (11)

[0142] Among them, During the iteration process, For factor indexes of fully connected tensor networks, , Index for iteration count, Indicates the current round. Indicates the previous round. The factorization factor of the fully connected tensor network to be solved in the current iteration is assigned to the solution result. , This represents the solution result of the fully connected tensor network decomposition factor to be solved in the current iteration during the iteration process. This refers to the results of reconstructing multi-time remote sensing images using a fully connected tensor network from the previous iteration in the iterative process. This refers to the factorization factor of the fully connected tensor network that has been solved in the current iteration during the iteration process. For the factorization factor of the fully connected tensor network that has been solved in the previous iteration, These are proximal parameters used to control the update step size.

[0143] Among them, During the iteration process, Index for iteration count, Indicates the current round. Indicates the previous round. The solution is the reconstruction result of the fully connected tensor network to be solved in the current iteration during the iteration process, and the solution result is assigned to... , This represents the solution result of reconstructing multi-temporal remote sensing images using a fully connected tensor network in the current iteration during the iteration process. This refers to the reconstruction results of the fully connected tensor network solved in the previous iteration during the iterative process. This refers to the factorization factor of the fully connected tensor network that has been solved in the current iteration during the iteration process. To guide the image, The proximal parameters are used to control the update step size; this optimization problem can be transformed into solving the following two subproblems:

[0144] Sub-problem 1: Update :

[0145] According to the literature [Y.-B. Zheng, T.-Z. Huang, X.-L. Zhao, Q. Zhao, and T.-X. Jiang, “Fully-connected tensor network decomposition and its application to higher-order tensor completion,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 12, pp. 11 071–11 078, May 2021], each involving The subproblems can all be transformed into the following matrix form:

[0146] (12)

[0147] in, and This indicates the removal of the first [unit] in the FCTN decomposition. One FCTN factor. Operator Let K represent the K-mode expansion operation (see Definition 1 in the Definitions and Preliminary Knowledge section for details). Therefore, the optimal solution of equation (11) can be expressed in the following closed-form form:

[0148] (13)

[0149] in, This indicates the transpose operation. This represents the matrix inversion operation, further factoring. It can be reconstructed from its K-mode expansion matrix through K-mode folding operations (the definition of which is detailed in the definition and definition 1 in the preparatory knowledge section).

[0150] Sub-problem 2, Update :

[0151] The subproblems integrate spatial domain constraints and and shear wave domain constraints To achieve unified optimization, the entire problem needs to be reformulated in the shear wave domain. According to the literature [K. Guo and D. Labate, “Optimally sparse multidimensional representation using shearlets,” SIAM Journal on Mathematical Analysis, vol. 39, no. 1, pp. 298–318, 2007], the design of the non-subsampled shear wave transform (NSST) operator strictly follows the Parseval frame principle at each scale and in each direction. The Parseval frame has the following fundamental properties:

[0152] (14)

[0153] in, Indicates the target image. Denotes the set of basis functions of the Passevar frame. , and These represent the scale, orientation, and translation parameters, respectively. This property is then applied... The tensor elements in the subproblem can be obtained as follows:

[0154] (15)

[0155] Furthermore, the NSST decomposition is essentially a linear transformation, as it consists only of linear convolutions and additive operations. Therefore, The subproblem can be restated as:

[0156] (16)

[0157] Rephrased The subproblem takes the form of least squares optimization, and each component can be solved in closed form as follows:

[0158] (17)

[0159] in, Indicates the low-frequency sub-band. Indicates the number of decomposition layers With direction The next high-frequency subband. Finally. Through the Reconstruction is performed by applying the inverse NSST operation, and during the iteration process, the following methods are used: Preserve the true values ​​of pixels in cloudless areas, where, and The definition is shown in equation (4).

[0160] The specific steps are summarized as follows:

[0161] Initialize FCTN factor ,(correspond Figure 1 (l) 4 factors Initialized reconstructed cloudless imagery (correspond Figure 1 (l) Input via the arrow on the left), number of iterations Maximum number of iterations .

[0162] Step 1: Obtain the guide image: Obtained by Algorithm 1. (Corresponding to...) Figure 1 (h))

[0163] Step 2: Determine if the current iteration count is the maximum iteration count. If it is, skip to step 9; otherwise, continue to the next step.

[0164] Step 3: Update factors based on fully connected tensor network decomposition theory See equation (13) (corresponding to) Figure 1 (l), because of the update Since it does not involve non-subsampled shear wave transform, it will be included in the network structure diagram of the fully connected tensor network decomposition. (Drawn on the left).

[0165] Step 4: Perform a non-subsampled shearing transform on the guiding image obtained from the guiding image estimation part of the preprocessing method for thick cloud removal in high-performance video coding applications of this invention. The number of non-subsampled shearing decomposition layers is... Number of non-subsampled shear wave decomposition directions .

[0166] (correspond Figure 1 (i)).

[0167] Step 5: Factorize the updated fully connected tensor network Perform non-subsampled shear wave transform, and determine the number of non-subsampled shear wave decomposition layers. Number of non-subsampled shear wave decomposition directions .

[0168] (correspond Figure 1 (k)).

[0169] Step 6: Decomposition factor of the fully connected tensor network after non-subsampled shear wave transform (correspond Figure 1 (k) and the guided image after non-subsampled shear wave transformation (corresponding to Figure 1 (i) Establish a system based on F-norm constraints (corresponding to...) Figure 1 (j) Optimization model update (The model is shown in Equation (10), and the solution result, i.e. the specific update calculation formula, is shown in Equation (17)).

[0170] Step 7: Check if the convergence condition of equation (18) is met. If it is met, end the loop. If it is not met, continue to step 8.

[0171] (18)

[0172] Step 8: Order Proceed to step two.

[0173] Step 9: End the loop.

[0174] Step 10: Update the current round As a reconstructed cloudless image .

[0175] (correspond Figure 1 (m)).

[0176] Summary: The complete process of cloud removal based on NSST-guided low-rank tensor decomposition is summarized as shown in Algorithm 2 in the table below.

[0177]

[0178] This invention presents a preprocessing method for thick cloud removal in high-performance video coding applications, achieving a comprehensive characterization of cross-scale correlations in multi-temporal images: It utilizes the multi-scale decomposition and translation invariance of non-subsampled shear wave transform (NSST) to guide a fully connected tensor network. Regularization constraints are applied at different scales, with high-frequency constraints enhancing texture and detail in the reconstructed region, while low-frequency constraints ensure color consistency. Simultaneously, translation invariance effectively maintains reconstruction accuracy even with slight registration errors in multi-temporal images.

[0179] The preprocessing method for thick cloud removal in high-performance video coding applications of the present invention achieves the generation of high-quality reference images: in order to obtain reference images containing more details, gamma correction is first performed on multi-temporal remote sensing images to normalize the radiance; then an improved adaptive PCNN fusion algorithm is applied in the NSST domain to enhance the detail representation while maintaining the energy distribution of the images, and finally a high-quality reference image is obtained.

[0180] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A pre-processing method for image thick cloud removal in high performance video coding application, characterized in that, It is a multi-temporal remote sensing image thick cloud removal method based on non-subsampled shearlet transform guided image estimation and non-subsampled shearlet transform guided full connected tensor decomposition cloud removal model, wherein: The non-subsampled shearlet transform guided image estimation comprises the following steps: Step one: data set decomposition; multiple temporal remote sensing images are divided into two mutually exclusive sets in chronological order: ； wherein M represents the spatial height, N represents the spatial width, B represents the number of spectral bands, T, T 1, T 2 both represent the length of the time series and T T 1 + T 2, cloud is an abbreviation for containing cloud contamination, free is an abbreviation for not containing cloud contamination;​​ Step two: gamma correction; For each cloud-contaminated image For all cloud-free images Independently gamma-correcting, denotes the time series index and resulting in ;​​ Step three: non-subsampled shearlet decomposition and parameter adaptive impulse coupled neural network fusion; For the gamma corrected image generated in step two to which a non-subsampled shearlet decomposition is applied, resulting in: ； wherein, denotes a gamma transform coefficient, denotes a low frequency subband, denotes a high frequency subband at decomposition level with direction NSST denotes a non-subsampled shearlet decomposition; denotes a gamma corrected image containing cloud pollution; For the low-frequency subband and the high-frequency subband, the fused subband is obtained according to the following rules: ； ； wherein, represents the parameter adaptive pulse coupled neural network input image pixel at position, m represents the spatial height index, n represents the spatial width index, b represents the spectral band number index, , represents the time series index; represents the pulse trigger number accumulated in the current round of iteration, represents the pulse trigger number accumulated in the last round of iteration, represents the parameter adaptive pulse coupled neural network neuron output in the current round of iteration, represents the iteration number index, represents the image with the highest pulse trigger number accumulated along the index direction, the index of the highest number is denoted as , represents the image with the highest peak signal-to-noise ratio along the index direction with , the index of the highest signal-to-noise ratio is denoted as , represents the peak signal-to-noise ratio, represents the index set of cloud-free pixels, is a projection operator, which acts to retain elements within the set and set other elements to zero, represents the fused image, represents the non-subsampled shearlet inverse transform; For high-frequency components, they are used as input to a parameter-adaptive pulse-coupled neural network, specifically as follows: The number of pulse triggers accumulated during the iteration process is denoted as . Select for each pixel position The image pixel value with the highest number of pulse triggers is used as The image index with the highest number of pulse triggers is denoted as... For low-frequency components, select the position of each pixel. Zhongyu The image with the highest peak signal-to-noise ratio as The image index with the highest peak signal-to-noise ratio is denoted as Finally, the fused image was reconstructed through inverse NSST transform. ; Subsequently, the results of the fusion image are added to the cloud-pollution-free clean images , thereby constructing an augmented cloud-pollution-free clean multi-temporal remote sensing dataset ; Step four: least square fitting; Least squares regression is used to establish a relationship between cloud-free regions in different time images and to determine the best fit coefficients, resulting in a guide image: ; The non-subsampled shearlet transform guided full connected tensor decomposition cloud removal model is expressed as: ； wherein, denotes a reconstructed cloud-free image, denotes a thick cloud-polluted image, is four FCTN factors; is a guided image, FCTN denotes a fully connected tensor network decomposition, denotes high-frequency sub-band coefficients at decomposition levels and directions , , are regularization parameters, respectively, denotes a Frobenius norm, denotes a constraint term.

2. The pre-processing method for image thick cloud removal for high performance video coding application according to claim 1, wherein, In step two of the guided image estimation based on the non-subsampled shearlet transform, for each cloud-polluted image For all cloud-free images Gamma correction is performed independently, and the specific process is as follows: ； wherein, denotes the image pair the specific gamma parameters obtained by calculating ； wherein, represents a take operation, represents a gamma transform operation, represents a gamma transform parameter, represents a peak signal-to-noise ratio, represents a set of indices of cloud-free pixels, is a projection operator that acts to retain elements within the set and zero elements at other positions; Further, all the sub-gamma correction results are merged in the dimension of time, obtaining a set of gamma correction images corresponding to the time nodes ; in the same way, the gamma correction images for all the cloud pollution time nodes are obtained .

3. The pre-processing method for image thick cloud removal for high performance video coding application according to claim 1, wherein, In step four of the non-subsampled shearlet transform guided image estimation, the optimization problem of determining the optimal fitting coefficient is expressed as: ； wherein, denotes the radiation gain coefficient between the images and , is the bias coefficient of the image , is the image with cloud pollution in the multi-temporal remote sensing data set corresponding to the index, the index number is , is the image corresponding to the index in the augmented multi-temporal remote sensing data set without cloud pollution, the index number is , denotes the Frobenius norm; Based on the estimated parameters, the least square estimation of the cloud contaminated area is further generated, and the form is as follows: ； wherein represents a guided image containing a cloud pollution time point, The guided images in the remaining cloud pollution time points are directly taken from the guided image By constraining to preserve all cloud-free areas in the original image, at the locations where cloud-free pixels exist, replace the estimated areas in with the corresponding true values in ; the multi-temporal remote sensing image thus obtained is the designed guidance image.

4. The pre-processing method for image thick cloud removal for high performance video coding application according to claim 1, wherein, The non-subsampled shearlet transform guided full connected tensor decomposition cloud removal model is solved by using a proximal alternating minimization algorithm, comprising: Under the PAM framework, With Alternating updates: ； wherein, in in the iteration process, is the factor index of the full connection tensor network decomposition, , is the iteration number index, represents the current round, represents the last round, is the full connection tensor network decomposition factor to be solved in the current round of the iteration process, and the solution result is assigned to , is the solution result of the full connection tensor network decomposition factor to be solved in the current round of the iteration process, is the full connection tensor network reconstruction multi-temporal remote sensing image result solved in the last round of the iteration process, is the full connection tensor network decomposition factor solved in the current round of the iteration process, is the full connection tensor network decomposition factor solved in the last round of the iteration process, is a proximal parameter used to control the update step size; wherein, in in the iteration process, is an index of the iteration number, denotes the current round, denotes the last round, is the reconstruction result of the full connection tensor network to be solved in the current round of the iteration process, and the solution result is assigned to , is the solution result of the full connection tensor network reconstruction of the multi-temporal remote sensing image to be solved in the current round of the iteration process, is the reconstruction result of the full connection tensor network that has been solved in the last round of the iteration process, is the decomposition factor of the full connection tensor network that has been solved in the current round of the iteration process; is a proximal parameter used to control the update step.

5. The pre-processing method for image thick cloud removal for high performance video coding application according to claim 4, wherein, The solving steps of the non-subsampled shearlet transform guided full connected tensor decomposition cloud removal model comprise: initializing the fctn factor , initializing the reconstructed cloud-free image , number of iterations , maximum number of iterations ; Step one: obtaining a guidance image: obtained from a guidance image estimation section based on a non-subsampled shearlet transform; Step two: determine whether the current iteration number is the maximum iteration number, if yes, jump to step nine, if not, continue to execute the next step; Step three: update factors based on full connection tensor network decomposition theory ; Step four: performing non-subsampled shearlet transform on the guided image obtained by the guided image estimation part based on non-subsampled shearlet transform, number of non-subsampled shearlet decomposition layers , number of non-subsampled shearlet decomposition directions ​ Step five: update the fully connected tensor network decomposition factors Performing non-subsampled shearlet transform, number of non-subsampled shearlet decomposition layers , number of non-subsampled shearlet decomposition directions ; Step six: decompose the factor using the non-subsampled shearlet transform post full connected tensor network Update the optimization model based on the Frobenius norm constraint with the non-subsampled shearlet transform post guided image ; Step seven: check whether the following formula convergence condition is met, if yes, end the loop, if not, continue to execute step eight; ； Step eight: Let perform step two; Step nine: end the loop; Step ten: the current round of updates as reconstructed cloud-free imagery .

Images