A hyperspectral image restoration method and system based on unsupervised guided network

Abstract

The present application belongs to the field of image restoration technology, and discloses a hyperspectral image restoration method and system based on an unsupervised guided network, the method comprising: forming an unsupervised guided network by constructing a deep feature extraction network and an attention-based feature reconstruction network; setting a first optimization loss function and a second optimization loss function, respectively optimizing the super-resolution task and the denoising task; when processing the target hyperspectral initial image, selecting the corresponding preprocessing method according to the task type, and inputting the unsupervised guided network for processing, and finally obtaining a hyperspectral restored image. The method can solve the problem that the existing hyperspectral image restoration methods are overly dependent on large-scale training data sets and lack versatility, resulting in poor quality of hyperspectral image restoration when performing the two representative tasks of hyperspectral image super-resolution and hyperspectral image denoising.

Description

Technical Field

[0001] The present invention belongs to the technical field of image restoration, and in particular relates to a hyperspectral image restoration method and system based on an unsupervised guided network. Background Art

[0002] Hyperspectral images (HSI) provide rich spectral and spatial details about the target area by collecting data in a continuous wavelength range. This rich information makes HSI widely used in many fields such as object classification, unmixing and anomaly detection. However, limited by the physical characteristics of the sensor or complex lighting conditions, HSI often suffers from various forms of degradation, including low spatial / spectral resolution and mixed noise. This degradation not only affects the visual quality of the image, but also causes problems such as noise, unclearness, occlusion and content loss in subsequent image analysis and processing, such as image segmentation and face recognition. Therefore, restoring the degraded HSI is a key preprocessing step that helps to recover high-quality images from low-quality images.

[0003] At present, hyperspectral image (HSI) restoration methods are roughly divided into traditional methods and deep learning methods. Traditional methods directly build optimization models by introducing appropriate prior regularization terms and fidelity terms to restore the desired HSI. In order to capture the intrinsic characteristics of HSI, researchers have developed a variety of hand-designed priors to constrain the restoration process. Deep learning methods, on the other hand, rely on neural networks to learn the end-to-end mapping function between input and output images. With the development of deep learning, many deep learning-based methods have been proposed. Depending on whether a training dataset is required, deep learning-based methods are mainly divided into supervised learning methods and unsupervised learning methods. However, the competitive performance of the above methods depends heavily on sufficient training groups, which makes them impractical in practical scenarios. On the one hand, since the deep image prior (DIP) method does not use any training data, it is particularly suitable for data-scarce applications such as hyperspectral imaging. On the other hand, due to the complex distribution of noise in actual situations, supervised learning methods may not be applicable to different scenarios. Unsupervised learning methods, which do not rely on training datasets, are gradually being widely used in HSI denoising.

[0004] Meanwhile, there are two representative tasks in hyperspectral image restoration: hyperspectral image super-resolution and hyperspectral image denoising. Due to the significant trade-off between spatial and spectral resolution of traditional hyperspectral image (HSI) equipment, hyperspectral images (HSI) usually have high spectral resolution but low spatial resolution. In contrast, RGB or multispectral sensors can easily collect images with higher spatial resolution, but at the expense of lower spectral resolution. Hyperspectral image (HSI) super-resolution aims to obtain a high spatial resolution hyperspectral image (HR-HSI) by fusing a low spatial resolution hyperspectral image (LR-HSI) with a high spatial resolution high-resolution multispectral image (HR-MSI). Due to physical mechanisms, sensor sensitivity, and imaging conditions, HSI is inevitably contaminated by various noises, such as Gaussian noise and sparse noise (fringe noise, dead line noise, and impulse noise). Hyperspectral image (HSI) denoising focuses on recovering clean HSI from the observed noisy HSI.

[0005] Therefore, existing hyperspectral image (HSI) restoration methods are overly dependent on large-scale training datasets and lack generality, resulting in poor hyperspectral image restoration quality when performing two representative tasks: hyperspectral image super-resolution and hyperspectral image denoising. Summary of the invention

[0006] Based on this, this application proposes a hyperspectral image restoration method and system based on an unsupervised guided network, aiming to solve the problem that existing hyperspectral image (HSI) restoration methods are overly dependent on large-scale training datasets and lack universality, resulting in poor hyperspectral image restoration quality when performing the two representative tasks of hyperspectral image super-resolution and hyperspectral image denoising.

[0007] The first aspect of the present application provides a hyperspectral image restoration method based on an unsupervised guided network, the image restoration method is divided into performing a super-resolution task and performing a denoising task, the method comprising:

[0008] Constructing an unsupervised guided network, the unsupervised guided network comprising a deep feature extraction network and an attention-based feature reconstruction network, the deep feature extraction network being used to extract multi-scale encoding features and decoding features, the attention-based feature reconstruction network being used to adaptively extract key features based on the encoding features and decoding features and reconstruct them into an output image;

[0009] Constructing a first optimization loss function and a second optimization loss function, wherein the first optimization loss function is used to perform a super-resolution task, and the second optimization loss function is used to perform a denoising task;

[0010] Acquire a target hyperspectral initial image, and determine whether the target hyperspectral initial image is to perform a super-resolution task or a denoising task;

[0011] If a super-resolution task is performed, the target hyperspectral initial image is subjected to a first preprocessing to obtain a high-resolution multispectral image as a first guide image, and based on the first optimization loss function, the first guide image is input into the unsupervised guide network to obtain a first hyperspectral restoration image;

[0012] If a denoising task is performed, the target hyperspectral initial image is subjected to a second preprocessing to obtain a clean band as a second guide image, and based on the second optimization loss function, the second guide image is input into the unsupervised guide network to obtain a second hyperspectral restoration image.

[0013] As an optional implementation manner of the first aspect, the deep feature extraction network includes an encoder and a decoder, wherein:

[0014] The encoding process of the encoder is expressed as:

[0015] ,

[0016] in, and Respectively represent and Encoded block features, represents the kth encoding block operation;

[0017] The decoding process of the decoder is expressed as:

[0018] ,

[0019] in, and Respectively represent and Aggregate features, q represents the weight, Indicates Aggregate feature operations, represents the skip connection operation, Indicates encoding features, and K represents the number of encoding blocks.

[0020] As an optional implementation of the first aspect, the attention-based feature reconstruction network includes a feature refinement module and a compressed feature excitation module, wherein:

[0021] The operation process of the feature refinement module is expressed as:

[0022] ,

[0023] in, represents the kth feature refinement feature, Indicates compression features, represents the kth feature refinement operation;

[0024] The operation process of the compressed feature excitation module is expressed as:

[0025] ,

[0026] in, Represents a compression feature operation.

[0027] As an optional implementation manner of the first aspect, the first optimization loss function is:

[0028] ,

[0029] Where Y represents the low-resolution hyperspectral image, represents high-resolution multispectral imagery, stands for high-resolution hyperspectral image, , denote the blur and downsampling degradation matrices in spatial height and width modes, respectively, represents the spectral response matrix, , and denote the tensor-matrix products of the 1st, 2nd, and 3rd modes, respectively, Representation paradigm, represents the equilibrium parameter, represents the regularization term that describes the prior information of high-resolution hyperspectral images.

[0030] As an optional implementation manner of the first aspect, the iteration of the first optimization loss function is defined as a first optimization strategy, and the first optimization strategy is expressed as:

[0031] ,

[0032] in, represent high-resolution multispectral images and random noise respectively, represents the unsupervised guided network, represents the parameters of the unsupervised guided network.

[0033] As an optional implementation manner of the first aspect, the second optimization loss function is:

[0034] ,

[0035] in, represents the minimization fidelity term, represents the sparse regularization term for sparse noise, represents the equilibrium parameter, represents the optimization parameters, and Represent the clean bands and random noise in the noisy image respectively.

[0036] As an optional implementation manner of the first aspect, the iteration of the second optimization loss function is defined as a second optimization strategy, and the second optimization strategy includes Alternating optimization of subproblems and S subproblems,

[0037] The sub-problem is expressed as:

[0038] ;

[0039] The S subproblem is expressed as:

[0040] .

[0041] A second aspect of an embodiment of the present application provides a hyperspectral image restoration system based on an unsupervised guided network, the system comprising:

[0042] Constructing a network module, for constructing an unsupervised guided network, the unsupervised guided network comprising a deep feature extraction network and an attention-based feature reconstruction network, the deep feature extraction network being used to extract multi-scale encoding features and decoding features, the attention-based feature reconstruction network being used to adaptively extract key features according to the encoding features and decoding features and reconstruct them into an output image;

[0043] A construction function module is used to construct a first optimization loss function and a second optimization loss function, wherein the first optimization loss function is used to perform a super-resolution task, and the second optimization loss function is used to perform a denoising task;

[0044] A task determination module is used to obtain a target hyperspectral initial image and determine whether the target hyperspectral initial image is to perform a super-resolution task or a denoising task;

[0045] A first task execution module is used to execute a super-resolution task, and performs a first preprocessing on the target hyperspectral initial image to obtain a high-resolution multispectral image as a first guide image, and based on the first optimization loss function, inputs the first guide image into the unsupervised guide network to obtain a first hyperspectral restoration image;

[0046] The second task execution module is used to perform a denoising task, and performs a second preprocessing on the target hyperspectral initial image to obtain a clean band as a second guide image. Based on the second optimization loss function, the second guide image is input into the unsupervised guide network to obtain a second hyperspectral restoration image.

[0047] A third aspect of an embodiment of the present application provides a computer device, which includes a memory, a processor, and a processing program stored in the memory and executable on the processor, wherein the processing program implements the above-mentioned image restoration method when executed by the processor.

[0048] A fourth aspect of an embodiment of the present application provides a storage medium, on which a processing program is stored. When the processing program is executed by a processor, the above-mentioned image restoration method is executed.

[0049] Compared with the prior art, the hyperspectral image restoration method based on an unsupervised guided network provided by the present application introduces a new network structure, uses a high-quality guided image as input, and uses random noise to generate a hyperspectral image, thereby improving the quality of the hyperspectral image (the super-resolution task uses a multispectral image as a guided image, and the denoising task uses a clean band as a guided image). Specifically, the new network structure (unsupervised guided network) consists of two main sub-networks and a connection module:

[0050] Deep Feature Extraction Network (DFE): This subnetwork is responsible for extracting multi-scale encoding and decoding features from the guidance image. Specifically, it consists of multiple convolutional layers, pooling layers, and upsampling layers, which are cascaded together to capture image features at different scales. The encoding-decoding structure effectively aggregates multi-level information to provide rich features for subsequent recovery;

[0051] Attention-based Feature Reconstruction Network (AFR): This subnetwork generates an output image starting from random noise. Its internal structure uses an attention mechanism module. Through the self-attention method, it selectively focuses on the parts of the feature map with important recovery information, thereby improving the accuracy of image restoration. The input of this subnetwork is the features extracted by the deep feature extraction network and the randomly initialized noise image;

[0052] Feature Refinement Module (FRM): As a bridge connecting the above two sub-networks, the feature refinement module embeds the multi-scale features output by the deep feature extraction network into the feature recovery network. The specific implementation method is: through layer-by-layer feature fusion, combined with feature selection strategies (such as feature weighting, feature selection), to maintain effective feature flow and transmission in the network.

[0053] Additional aspects and advantages of the present application will be given in part in the following description, and in part will become apparent from the following description, or will be understood through the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A flowchart of a hyperspectral image restoration method based on an unsupervised guided network proposed in the first embodiment of the present application;

[0055] Figure 2 This is a general architecture diagram of the unsupervised guidance network (UFFGNet) in the second embodiment of the present application;

[0056] Figure 3 This is a diagram of the architecture of a feature aggregation module (FAM) in the second embodiment of the present application;

[0057] Figure 4 This is a diagram of the architecture of a feature refinement module (FRM) in the second embodiment of the present application;

[0058] Figure 5 This is a diagram of the architecture of a compressed feature excitation module (SFEM) in the second embodiment of the present application;

[0059] Figure 6 A schematic diagram of the structure of a hyperspectral image restoration system based on an unsupervised guided network proposed in the third embodiment of the present application.

[0060] The following specific implementation methods will further illustrate the present application in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0062] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here. In addition, the "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated with each other are in an "or" relationship.

[0063] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.

[0064] See also Figure 1 , is a flow chart of a hyperspectral image restoration method based on an unsupervised guided network proposed in the first embodiment of the present application. The image restoration method is divided into performing a super-resolution task and performing a denoising task. The method includes:

[0065] S01: Constructing an unsupervised guided network, wherein the unsupervised guided network includes a deep feature extraction network and an attention-based feature reconstruction network, wherein the deep feature extraction network is used to extract multi-scale encoding features and decoding features, and the attention-based feature reconstruction network is used to adaptively extract key features according to the encoding features and decoding features and reconstruct them into an output image.

[0066] It should be noted that the proposed unsupervised guidance network (UFFGNet) consists of two main parts: a deep feature extraction network (DFE) and an attention-based feature reconstruction network (AFR). Figure 2 The figure shows the overall architecture of the unsupervised guidance network (UFFGNet). These networks are connected by a feature refinement module (FRM), which is specially designed to effectively utilize and integrate the multi-scale spatial-spectral information (i.e., multi-scale encoding and decoding features) extracted from the guidance image into the attention-based feature reconstruction network (AFR).

[0067] Deep Feature Extraction Network (DFE): Similar to U-net, it uses an encoder-decoder network to extract multi-scale hierarchical features of the guidance image, from low-level to high-level.

[0068] First, the guidance image is converted into a feature map through a convolution operation to generate features Then, multiple encoding blocks are used to obtain multi-scale encoding features. . represents the kth encoding block operation, represents the kth encoding block feature. The entire encoding process can be expressed as follows:

[0069] ,

[0070] Decoded features can be generated by using decoding blocks paired with encoded features in the spatial-channel dimension. It is worth noting that most commonly used U-Net encoding and decoding feature fusion techniques involve simple concatenation or addition operations combined with skip connections. However, these operations make it difficult to depict the interaction of encoding and decoding features, resulting in insufficient feature fusion. In addition, the convolution operations in the encoding and decoding blocks have local receptive fields and cannot utilize contextual information outside the receptive field. In order to address the limitations of the existing U-Net structure in the fusion of encoding and decoding features, this application proposes a feature aggregation module (FAM), such as Figure 3 Shown is the architecture diagram of the feature aggregation module (FAM). To reduce the loss of information in the encoding-decoding process and retain their complementary semantic features. Specifically, the feature aggregation module (FAM) can dynamically aggregate encoding and decoding features through spatial and channel attention mechanisms. The feature aggregation module (FAM) first fuses the encoding and decoding features through element-wise summation convolution operations, and then performs global average pooling (GAP) in the spatial and channel dimensions to generate statistical information. Spatial statistics generate spatial weights through a sigmoid activation function, aiming to fully capture spatial correlations. In contrast, channel statistics generate a weight vector through a multi-layer perceptron (MLP) and a sigmoid activation function , and the other channel weight is calculated by 1-q. The decoding process is defined as follows:

[0071] ,

[0072] in, represents the kth decoding block operation, represents the features extracted from the kth encoded feature, and represents the features generated by the Feature Aggregation Module (FAM), and Respectively represent and Aggregate features, q represents the weight, Indicates Aggregate feature operations, represents the skip connection operation, Indicates encoding features, and K represents the number of encoding blocks.

[0073] Attention-based feature reconstruction network (AFR): including feature refinement module (FRM) and compressed feature excitation module (SFEM). Figure 4 As shown in Figure 1, the architecture diagram of the Feature Refinement Module (FRM) is designed to utilize the rich multi-scale spatial-spectral information in the guidance image; Figure 5 As shown in the figure, it is the architecture diagram of the compressed feature excitation module (SFEM), which is used to efficiently utilize the dependencies between channels. Since this application uses the feature aggregation module (FAM) to aggregate the encoding and decoding features, the feature refinement module (FRM) can process more fine-grained features and rich semantic information. In the Feature Refinement Module (FRM), these fine-grained features are first used to generate attention weights, and then these weights are used to guide semantic recovery. In summary, the operation process of the Feature Refinement Module (FRM) module is as follows:

[0074] ,

[0075] in, represents the kth feature refinement feature, Indicates compression features, Represents the k-th feature refinement operation.

[0076] In order to enhance channel dependency, this application further designs a compressed feature excitation module (SFEM) to improve feature representation capabilities through dynamic channel feature recalibration. The compressed feature excitation module (SFEM) can compress global spatial information into channel descriptors, effectively capturing key details while filtering out irrelevant information. The operation process of the SFEM module is as follows:

[0077] ,

[0078] in, Represents a compression feature operation.

[0079] Finally, a simple convolution operation is used to transform the previously computed rich features into an output image.

[0080] S02: constructing a first optimization loss function and a second optimization loss function, wherein the first optimization loss function is used to perform a super-resolution task, and the second optimization loss function is used to perform a denoising task;

[0081] The goal of super-resolution task is to obtain high-quality low-resolution hyperspectral images (LR-HSI). and high-quality high-resolution multispectral images (HR-MSI) Recovering high-quality high-resolution hyperspectral images (HR-HSI) ,in , ,and The degradation process between HR-HSI and LR-HSI / HR-MSI can be expressed as:

[0082] ,

[0083] in, represents a low-resolution hyperspectral image, represents high-resolution multispectral imagery, stands for high-resolution hyperspectral image, and are the blur and downsampling degradation matrices in spatial height and width modes, respectively, is the spectral response matrix, symbol represents the tensor-matrix product of the kth module, , and denote the tensor-matrix products of the 1st, 2nd, and 3rd modes, respectively, Representation paradigm. In general, the high-resolution hyperspectral image (HR-HSI) restoration problem based on unsupervised methods can be solved by constructing appropriate loss functions and optimization strategies to maintain spectral consistency and spatial detail accuracy without pre-labeled data, thereby reducing dependence on training datasets and providing flexible and robust image restoration solutions. Unsupervised restoration process: In practical applications, the unsupervised guidance network (UFFGNet) does not require any labeled datasets for training. Instead, the network gradually learns deep prior knowledge of the image by continuously optimizing the loss function, relying on the input multi-spectral guidance image and the randomly initialized noise image.

[0084] The problem of performing the super-resolution task is summarized as the first optimization loss function:

[0085] ,

[0086] Among them, the first two items ( and ) is the model fidelity term, R(X) is the regularization term describing the prior information of high-resolution hyperspectral image (HR-HSI), and λ is the parameter for the balancing term. Traditional methods exploit handcrafted prior information by designing corresponding regularization terms, i.e. However, task-specific priors are often difficult to extend to other restoration tasks. Instead of using hand-crafted priors, our method uses the proposed unsupervised guidance network (UFFGNet) to estimate HR-HSI by adaptively learning deep priors .

[0087] ,

[0088] in, , represent high-resolution multispectral images and random noise respectively, represents the unsupervised guided network, represents the parameters of the unsupervised guided network.

[0089] It is worth noting that the parameter The optimization of is a regression problem. The proposed unsupervised guided network (UFFGNet) uses the Adam algorithm as the optimizer. The iteration of the loss function of the unsupervised guided network (UFFGNet) is defined as the first optimization strategy (7). The first term ( ) constrains the spatial degradation of the image and Y. Relatively speaking, the second term ( ) constrains the spectrally degraded image and Z. In the process of optimizing the network parameters, only random noise and known guidance images are required as input, and the loss function does not involve external unknown data. Therefore, the proposed method does not require any training dataset, thus establishing an unsupervised deep learning framework.

[0090] In the denoising task, hyperspectral images (HSI) are inevitably contaminated by various noises, such as Gaussian noise, stripe noise, cutoff noise, and impulse noise. In order to enhance the adaptability of the unsupervised guided network (UFFGNet) in dealing with these complex restoration challenges, this method further extends the unsupervised guided network (UFFGNet) for robust unsupervised restoration. The goal of HSI denoising is to obtain a noisy HSI ( ) to restore the required HSI ( ). The degradation model of noisy HSI can be expressed as follows.

[0091] ,

[0092] in, represents sparse noise (including fringe noise, cutoff noise, and impulse noise), and represents Gaussian noise.

[0093] Similarly, this paper proposes an unsupervised guided network (UFFGNet) to learn deep priors and estimate the target hyperspectral image (HSI). In order to eliminate the negative impact of sparse noise (such as impulse noise, stripe noise, cutoff line noise, etc.) on the quality of hyperspectral image restoration, this paper introduces a robust restoration model, which is specifically implemented as follows:

[0094] Model framework: Sparse noise is regarded as an independent error term, and a sparse regularization method is introduced to model it. The model assumes that the restored image is formed by the superposition of real image data, Gaussian noise, and sparse noise. By minimizing the weighted combination of the fidelity term and the sparse regularization term, the sparse noise is suppressed and the image is accurately restored.

[0095] Loss function design: The designed loss function contains two main parts: one is the fidelity term, which is used to measure the difference between the restored image and the original image; the other is the sparse regularization term, which is used to limit the distribution of sparse noise. The loss function is optimized by the gradient descent method, continuously updating the network parameters, and gradually improving the accuracy and robustness of image restoration.

[0096] In order to eliminate the influence of sparse noise, a robust recovery model is proposed and summarized as the second optimization loss function:

[0097]

[0098] in, represents the minimization fidelity term, represents the sparse regularization term for sparse noise, which is defined as the sum of the absolute values ​​of all elements, represents the equilibrium parameter, represents the optimization parameters, Represent the clean bands and random noise in the noisy image, respectively. Since the noisy hyperspectral image (HSI) contains multiple bands with high signal-to-noise ratio, this method uses these high-quality bands as the guidance image G of the unsupervised guidance network (UFFGNet) to make full use of the information in the noisy data.

[0099] There are two variables in formula (9) that need to be optimized. This method uses an effective alternating minimization method for optimization. When the variable S is fixed, The sub-problems are of the following form:

[0100] ,

[0101] In the unsupervised guided network (UFFGNet) of the present invention, the Adam algorithm is used as the optimizer, and the loss function is set as formula (10). The specific optimization process is as follows: Optimizer selection: The Adam (Adaptive Moment Estimation) algorithm is selected as the optimizer because it can adaptively adjust the learning rate of each parameter and accelerate the convergence process by the momentum weighted average of the first-order and second-order moments. The Adam algorithm combines the advantages of the momentum method and the RMSProp method, performs well in processing sparse gradients and non-stationary objective functions, and is very suitable for unsupervised deep learning tasks. Fixed variables After that, the S subproblem is expressed as:

[0102]

[0103] The subproblem of S is a sparse optimization problem, and its update process can be achieved by the soft-threshold shrinkage method. The soft-threshold shrinkage method is a commonly used sparsification technique used to optimize variables in sparse problems so that it can effectively remove sparse noise.

[0104] ,

[0105] It is worth noting that the robust unsupervised learning strategy proposed in the present invention is as follows: after every 20 iterations based on the loss function (10), the network will update the S subproblem (12). This alternating optimization process will continue until convergence is reached.

[0106] Furthermore, the description of the above denoising task is as follows:

[0107] 1. Strategy Overview: The learning strategy of this invention aims to achieve efficient restoration of hyperspectral images by alternately optimizing network parameters and sparse variables. The strategy periodically updates sparse variables to eliminate noise and continuously optimizes network parameters to improve restoration accuracy.

[0108] 2. Alternating optimization process:

[0109] Network parameter update: In each round of optimization, the network first updates its parameters based on the loss function (10). The optimization process of these parameters involves image fidelity and regularization processing to ensure that the network can effectively extract and learn useful features from the input data.

[0110] Sparse variable update: After every 20 iterations, the update of network parameters is stopped and the sparse variable S subproblem (12) is updated instead. Sparse variables are updated using sparse techniques such as soft threshold shrinkage to remove noise and maintain the sparse characteristics of the image, thereby improving the image restoration effect.

[0111] 3. Iterate until convergence: This alternating optimization process continues, and the network continuously optimizes network parameters and updates sparse variables until the value of the loss function stabilizes below the set threshold or reaches the predetermined maximum number of iterations. This process ensures the stability and efficiency of the recovery results.

[0112] 4. Advantages and applications: Compared with the traditional method of optimizing only network parameters or only sparse variables, this strategy can more comprehensively consider the impact of sparse noise and improve the overall effect of image restoration. It is particularly suitable for noise processing in hyperspectral image restoration and can significantly improve the restoration quality and robustness.

[0113] Through this robust unsupervised learning strategy, this method can effectively process sparse noise in hyperspectral images under unsupervised conditions and achieve high-quality image restoration.

[0114] S03: Acquire a target hyperspectral initial image, and determine whether the target hyperspectral initial image is to perform a super-resolution task or a denoising task.

[0115] S04: If a super-resolution task is to be performed, a first preprocessing is performed on the target hyperspectral initial image to obtain a high-resolution multispectral image as a first guide image, and based on the first optimization loss function, the first guide image is input into the unsupervised guide network to obtain a first hyperspectral restoration image.

[0116] It should be noted that the first guide image obtained after the first preprocessing is a high-resolution multispectral image, and the first optimization loss function is shown in formula (6), which can obtain a first hyperspectral restoration image, that is, a high-resolution hyperspectral image.

[0117] S05: If a denoising task is to be performed, a second preprocessing is performed on the target hyperspectral initial image to obtain a clean band as a second guide image, and based on the second optimization loss function, the second guide image is input into the unsupervised guide network to obtain a second hyperspectral restoration image.

[0118] It should be noted that the second guide image obtained after the second preprocessing is a clean band in the noisy image. The clean band can be judged according to the specific situation and empirical data, and is mainly manifested as a stable and non-contaminated noise band. The second optimization loss function is shown in formula (9), and the second hyperspectral restoration image, that is, the denoised hyperspectral image, can be obtained.

[0119] See also Figure 6 , which is a schematic diagram of the structure of a hyperspectral image restoration system based on an unsupervised guided network proposed in the second embodiment of the present application, and the system includes:

[0120] Constructing a network module 100, for constructing an unsupervised guided network, the unsupervised guided network comprising a deep feature extraction network and an attention-based feature reconstruction network, the deep feature extraction network being used to extract multi-scale encoding features and decoding features, the attention-based feature reconstruction network being used to adaptively extract key features according to the encoding features and decoding features and reconstruct them into an output image;

[0121] A function construction module 200 is used to construct a first optimization loss function and a second optimization loss function, wherein the first optimization loss function is used to perform a super-resolution task, and the second optimization loss function is used to perform a denoising task;

[0122] A task determination module 300 is used to obtain a target hyperspectral initial image and determine whether the target hyperspectral initial image is to perform a super-resolution task or a denoising task;

[0123] The first task execution module 400 is used to execute the super-resolution task, and performs a first preprocessing on the target hyperspectral initial image to obtain a high-resolution multispectral image as a first guide image, and based on the first optimization loss function, inputs the first guide image into the unsupervised guide network to obtain a first hyperspectral restoration image;

[0124] The second task execution module 500 is used to perform a denoising task, and performs a second preprocessing on the target hyperspectral initial image to obtain a clean band as a second guide image. Based on the second optimization loss function, the second guide image is input into the unsupervised guide network to obtain a second hyperspectral restoration image.

[0125] On the other hand, the present application also proposes a computer device, which includes a memory, a processor, and a processing program stored in the memory and executable on the processor, and the processing program implements the above-mentioned image restoration method when executed by the processor.

[0126] On the other hand, the present application further proposes a storage medium, on which a processing program is stored, and when the processing program is executed by a processor, the above-mentioned image restoration method is executed.

[0127] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0128] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0129] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A hyperspectral image restoration method based on an unsupervised guided network, characterized in that: The method comprises: Constructing an unsupervised guided network, the unsupervised guided network comprising a deep feature extraction network and an attention-based feature reconstruction network, the deep feature extraction network being used to extract multi-scale encoding features and decoding features, the attention-based feature reconstruction network being used to adaptively extract key features based on the encoding features and decoding features and reconstruct them into an output image; Constructing a first optimization loss function and a second optimization loss function, wherein the first optimization loss function is used to perform a super-resolution task, and the second optimization loss function is used to perform a denoising task; Acquire a target hyperspectral initial image, and determine whether the target hyperspectral initial image is to perform a super-resolution task or a denoising task; If a super-resolution task is performed, the target hyperspectral initial image is subjected to a first preprocessing to obtain a high-resolution multispectral image as a first guide image, and based on the first optimization loss function, the first guide image is input into the unsupervised guide network to obtain a first hyperspectral restoration image; The first optimization loss function is: ， Where Y represents the low-resolution hyperspectral image, represents high-resolution multispectral imagery, stands for high-resolution hyperspectral image, , denote the blur and downsampling degradation matrices in spatial height and width modes, respectively, represents the spectral response matrix, , and denote the tensor-matrix products of the 1st, 2nd, and 3rd modes, respectively, Representation paradigm, represents the equilibrium parameter, represents the regularization term that describes the prior information of high-resolution hyperspectral images; If a denoising task is performed, a second preprocessing is performed on the target hyperspectral initial image to obtain a clean band as a second guide image, and based on the second optimization loss function, the second guide image is input into the unsupervised guide network to obtain a second hyperspectral restoration image; The second optimization loss function is: ， in, represents the minimization fidelity term, represents the sparse regularization term of sparse noise, represents the equilibrium parameter, represents the optimization parameters, and Represent the clean bands and random noise in the noisy image respectively.

2. The image restoration method according to claim 1, characterized in that: The deep feature extraction network includes an encoder and a decoder, wherein: The encoding process of the encoder is expressed as: ， in, and Respectively represent and Encoded block features, represents the kth encoding block operation; The decoding process of the decoder is expressed as: ， in, and Respectively represent and Aggregate features, q represents the weight, Indicates Aggregate feature operations, represents the skip connection operation, Indicates encoding features, and K represents the number of encoding blocks.

3. The image restoration method according to claim 2, characterized in that: The attention-based feature reconstruction network includes a feature refinement module and a compressed feature excitation module, wherein: The operation process of the feature refinement module is expressed as: ， in, represents the kth feature refinement feature, Indicates compression features, represents the kth feature refinement operation; The operation process of the compressed feature excitation module is expressed as: ， in, Represents a compression feature operation.

4. The image restoration method according to claim 1, characterized in that: The iteration of the first optimization loss function is defined as a first optimization strategy, and the first optimization strategy is expressed as: ， in, represent high-resolution multispectral images and random noise respectively, represents the unsupervised guided network, represents the parameters of the unsupervised guided network.

5. The image restoration method according to claim 1, characterized in that: The iteration of the second optimization loss function is defined as a second optimization strategy, and the second optimization strategy includes Alternating optimization of subproblems and S subproblems, The sub-problem is expressed as: ； The S subproblem is expressed as: 。 6. A hyperspectral image restoration system based on an unsupervised guided network, characterized in that: The system comprises: Constructing a network module, for constructing an unsupervised guided network, the unsupervised guided network comprising a deep feature extraction network and an attention-based feature reconstruction network, the deep feature extraction network being used to extract multi-scale encoding features and decoding features, the attention-based feature reconstruction network being used to adaptively extract key features according to the encoding features and decoding features and reconstruct them into an output image; A construction function module is used to construct a first optimization loss function and a second optimization loss function, wherein the first optimization loss function is used to perform a super-resolution task, and the second optimization loss function is used to perform a denoising task; A task determination module is used to obtain a target hyperspectral initial image and determine whether the target hyperspectral initial image is to perform a super-resolution task or a denoising task; The first task execution module is used to perform the super-resolution task, and the target hyperspectral initial image is subjected to a first preprocessing to obtain a high-resolution multispectral image as a first guide image. Based on the first optimization loss function, the first guide image is input into the unsupervised guidance network to obtain a first hyperspectral restoration image; the first optimization loss function is: , where Y represents the low-resolution hyperspectral image, represents high-resolution multispectral imagery, stands for high-resolution hyperspectral image, , denote the blur and downsampling degradation matrices in spatial height and width modes, respectively, represents the spectral response matrix, , and denote the tensor-matrix products of the 1st, 2nd, and 3rd modes, respectively, Representation paradigm, represents the equilibrium parameter, represents the regularization term that describes the prior information of high-resolution hyperspectral images; The second task execution module is used to perform the denoising task. The target hyperspectral initial image is subjected to a second preprocessing to obtain a clean band as a second guide image. Based on the second optimization loss function, the second guide image is input into the unsupervised guidance network to obtain a second hyperspectral restoration image. The second optimization loss function is: ,in, represents the minimization fidelity term, represents the sparse regularization term of sparse noise, represents the equilibrium parameter, represents the optimization parameters, and Represent the clean bands and random noise in the noisy image respectively.

7. A computer device, characterized in that: The computer device includes a memory, a processor, and a processing program stored in the memory and executable on the processor. When the processing program is executed by the processor, the image restoration method according to any one of claims 1 to 5 is implemented.

8. A storage medium, characterized in that: The storage medium stores a processing program, and when the processing program is executed by a processor, the image restoration method according to any one of claims 1 to 5 is implemented.

Images