A deep robust fusion method of hyperspectral and multispectral images for watershed detection

Abstract

The present invention relates to the technical field of hyperspectral remote sensing image quality enhancement, and provides a high-multispectral image deep robust fusion method for watershed detection: performing amplitude normalization processing on observed hyperspectral and multispectral data; stripping; extracting fine spatial features of multispectral data to supplement spatial information of hyperspectral data; extracting identification spectral features of hyperspectral data to supplement spectral information of multispectral data; performing selective spatial-spectral fusion on the spatial enhancement results of hyperspectral data and the spectral dimension enhancement results of multispectral data; performing spatial and spectral degradation processing on fused data to reconstruct hyperspectral and multispectral inputs; achieving parameter optimization of stripping, spatial-spectral enhancement and spatial-spectral fusion modules by minimizing loss functions, and outputting the results of spatial-spectral fusion modules. The present invention effectively eliminates the strip noise commonly present in hyperspectral data and improves the robustness of image fusion to strip noise.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral remote sensing image quality enhancement, which is a high-multispectral image deep robust fusion method for watershed detection, used to generate clean high-resolution hyperspectral images and provide effective data for watershed observation tasks. Background Art

[0002] The river basin occupies more than 48% of the country's land area and is rich in resources. It is planned to be listed as a key construction and development object. Carrying out monitoring of soil properties, geological structures and vegetation types in the river basin is the key to developing, managing and protecting the river basin. The river basin has complex and similar landforms, and the species types are diverse and similar. Fortunately, hyperspectral remote sensing can effectively identify soil composition, geological types and vegetation attributes by providing diagnostic spectra of the target. However, narrow-band hyperspectral remote sensing under a reasonable signal-to-noise ratio inevitably requires sacrificing spatial resolution. In practical applications, in addition to diagnostic spectral features, extracting fine spatial information is very important for improving the accuracy of hyperspectral remote sensing data interpretation and analysis. Therefore, it is urgent to introduce wide-band optical remote sensing data to improve the spatial resolution of hyperspectral data. In high-resolution wide-band optical remote sensing, multispectral data can provide more spectral information than panchromatic data. The super-resolution reconstruction of hyperspectral data can be achieved with the help of multispectral data, which can better preserve the diagnostic spectral features. Therefore, the fusion of hyperspectral and multispectral images has become a major means of super-resolution of hyperspectral images. Currently, the existing hyperspectral and multispectral image fusion methods can be roughly divided into two categories: regularized modeling and deep learning.

[0003] Regularized modeling fusion methods usually combine one or more prior assumptions to establish the fusion of hyperspectral and multispectral images as a prior regularized convex problem, and then use a suitable iterative optimization method to obtain the optimal solution of the convex problem. However, artificial prior assumptions are difficult to characterize the characteristics of complex watershed environments, resulting in unstable fusion results. In addition, the complexity of iterative optimization algorithms is so high that it is difficult to achieve real-time applications. Fortunately, the multi-layer perception model of deep learning has shown outstanding advantages in extracting the intrinsic spatial-spectral characteristics of multi-channel data, which helps to accurately characterize complex watershed environments. In addition, deep learning methods driven by big data can transfer the computational burden to the training stage, which can effectively solve the problem of long optimization time consumption in regularized modeling methods.

[0004] So far, the hyperspectral and multispectral image fusion methods based on deep learning have achieved remarkable results, mainly including supervised and self-supervised modes. The fusion in the supervised mode directly learns the nonlinear reconstruction mapping of hyperspectral and multispectral images to the fusion target, and needs to use the high-resolution and multispectral sensor functions to construct sufficient high-resolution hyperspectral / low-resolution hyperspectral and high-resolution multispectral image group training networks. Existing supervised fusion methods are mostly based on public data sets and sensor functions to construct simulated sample sets. The scale of public data is small and the imaging conditions are ideal. However, the actual watershed has many objects and complex environment, so this type of fusion method has poor generalization in watershed scenes. The fusion in the self-supervised mode generates high-resolution hyperspectral data by fitting the reconstruction process of high-resolution and multispectral data-fusion target-data to be fused, which can avoid the construction of simulated sample sets. The representative self-supervised fusion method is to extract the fine spatial features of multispectral data and the diagnostic spectral features of hyperspectral data respectively under the support of linear spectral unmixing theory, and the high-resolution hyperspectral image can be obtained by fusing the spatial and spectral features, and then the hyperspectral or multispectral input can be reconstructed after the degradation of the sensor function. However, on the one hand, the distortion-free extraction of the intrinsic spectral and spatial features of hyperspectral and multispectral data cannot be guaranteed; on the other hand, the full preservation and utilization of the inherent spatial and spectral features of hyperspectral and multispectral data are ignored. This is likely to lead to the problem of spatial spectrum loss in the fusion result, and it is difficult to obtain high-resolution hyperspectral data that fully characterizes the diversity of the two source data and the comprehensiveness of the target features. In addition, the poor uniformity of the internal photosensitive devices of the hyperspectral sensor and the incomplete correction of the response function usually lead to obvious striping artifacts in the captured hyperspectral data. Most of the existing hyperspectral and multispectral image fusion methods are based on clean data and neglect to deal with the common striping noise in hyperspectral data. Summary of the invention

[0005] In order to achieve the fusion of hyperspectral and multispectral data under the premise of ensuring the robustness of stripes and maintaining the spectral and spatial information of hyperspectral and multispectral data without loss, the present invention proposes a deep robust fusion method of hyper-multispectral images for watershed detection. This method can effectively remove the stripe noise in hyperspectral data, improve its spatial resolution while maintaining the observed spectrum of hyperspectral data without loss, and enhance its spectral dimension characteristics while maintaining the observed space of multispectral data without loss, thereby realizing the fusion of the spatial enhancement results of hyperspectral data and the spectral enhancement results of multispectral data in a self-supervised mode, and finally generating high-resolution hyperspectral data.

[0006] The specific technical solution is a deep robust fusion method of high-multispectral images for watershed detection, comprising the following steps:

[0007] S1, amplitude normalization processing is performed on the observed hyperspectral and multispectral data respectively.

[0008] S2, based on the three-dimensional convolution and residual structure, the stripping module is designed to effectively estimate the three-dimensional strip components in the hyperspectral data, and to achieve stripping under the premise of ensuring the spatial-spectral structure of the hyperspectral data. In order to accurately represent the spatial-spectral structure of the strip components and thus ensure the intrinsic spatial-spectral consistency of the hyperspectral data, the stripping module uses a three-dimensional convolution layer for feature extraction and representation. Given that the mainstream de-stripping method assumes that the stripping noise is additive noise, the stripping module uses a residual structure that can characterize additive operations to connect the three-dimensional convolution layer. At the same time, in order to avoid the risk of information overload caused by the traditional residual structure directly transmitting shallow features, the self-attention mechanism is used to guide the transmission of important information in the shallow features.

[0009] S3, based on two-dimensional convolution and dense connection, the spatial enhancement module is designed to extract the fine spatial features of multispectral data to supplement the spatial information of hyperspectral data; at the same time, based on the fully connected network and dense connection, the spectral enhancement module is designed to extract the identification spectral features of hyperspectral data to supplement the spectral information of multispectral data. The spatial enhancement module is composed of two networks with the same structure connected in parallel. One uses two-dimensional convolution to extract multi-level fine spatial features in multispectral data, and uses dense connection to realize feature connection between multi-layer networks to prevent feature omission during feature deepening. The other uses the same structure to characterize the hyperspectral data to ensure that the two source data are injected and fused at a consistent representation angle for spatial features. The spectral enhancement module is also composed of two networks with the same structure connected in parallel. One uses a fully connected network to extract identification spectral features in the hyperspectral data matrix, and uses dense connection to realize multi-level feature reuse and prevent feature omission. The other performs characterization on the multispectral data to ensure that the two source data are injected and fused at a consistent representation angle for spectral features.

[0010] S4, based on the attention mechanism, designs a spatial-spectral fusion module to selectively fuse the spatial enhancement results of hyperspectral data and the spectral enhancement results of multispectral data. Under the guidance of the cross-attention mechanism, the spatial-spectral fusion module calculates a two-dimensional attention map based on the hyperspectral and multispectral data enhancement results, and uses the obtained attention map to guide the selective fusion of multispectral and hyperspectral data enhancement results. Finally, a multi-layer three-dimensional convolutional layer is used to further promote the fusion to generate high-resolution hyperspectral data.

[0011] S5, based on the linear degradation model, performs spatial and spectral degradation processing on the fused data to reconstruct the hyperspectral and multispectral inputs.

[0012] S6, by minimizing the loss function, realizes the parameter optimization of the stripping, spatial spectrum enhancement and spatial spectrum fusion modules, and outputs the result of the spatial spectrum fusion module.

[0013] Furthermore, step S1 specifically includes the following sub-steps:

[0014] S101: Observing Hyperspectral Data Divide each pixel in the data by the maximum value of all pixels in the data, and then assign all pixel values ​​of the hyperspectral data to the interval .in, , , Represents the number of rows, columns, and channels of hyperspectral data respectively.

[0015] S102: Observe multispectral data Divide each pixel in the data by the maximum value of all pixels in the data, and then put all pixel values ​​of the multispectral data into the interval .in, , , represents the number of rows, columns, and channels of multispectral data, respectively, and , , .

[0016] Furthermore, step S2 specifically includes the following sub-steps:

[0017] S201: Normalizing hyperspectral data using a 3D convolutional layer and a ReLu function cascaded input layer Perform primary characterization to obtain the primary feature tensor , where the three-dimensional convolutional layer contains The size is 3D convolution kernel.

[0018] S202: Using the self-attention mechanism and the self-attention residual block constructed by the residual structure, the primary features are deepened and selectively transferred to obtain the advanced feature tensor ,in, , indicating the A self-attention residual block.

[0019] Specifically, the feature output of the first self-attention residual block For example: ,in, and Describe the self-attention mechanism and the dense blocks of the cascade of three-dimensional convolutional layer, BN layer and ReLu function, namely The output of the self-attention residual block From the previous level feature tensor Self-attention selection features Its deepening characteristics of superposition.

[0020] S203: Further integration using short connections between 3D convolutional layers and BN layers The deep and shallow features contained in it are The cascade of the three-dimensional convolutional layer and the tanh activation function reduces the dimension of the fusion feature and estimates the normalized hyperspectral data Medium stripe weight .

[0021] Furthermore, step S3 specifically includes the following sub-steps:

[0022] S301: Observing the spatial dimensions of multispectral data As the target, bicubic interpolation is used to normalize the hyperspectral data. Perform super-resolution initialization processing to obtain .

[0023] S302: Super-resolution initialization results of hyperspectral data using two-dimensional convolutional layer and short connection of ReLu function and normalized multispectral data Perform feature processing to obtain the primary feature tensor and , where the two-dimensional convolutional layer contains The size is 3D convolution kernel.

[0024] S303: The primary feature tensor is respectively processed by densely connecting dense blocks of two-dimensional convolutional layers, BN layers and ReLu functions. and To deepen, add units to deepen the features Injection Features and enter the branch where the hyperspectral data is located. dense blocks, where and Respectively represent the first The output of a dense block.

[0025] Specifically, the feature output of the first dense block in the two-way dense connection and And the feature input of the second dense block of the branch where the hyperspectral and multispectral data are located and For example: , , , .in, , , represents a dense block of cascaded 2D convolutional layers, BN layers, and ReLu functions, Represents dense connection, that is, the feature tensors are concatenated along the channel dimension.

[0026] S304: Use the short connection of the two-dimensional convolution layer and the tanh activation function cascade to output the features of the last dense block of the branch where the hyperspectral data is located Mapped to the image domain and injected into it through the addition unit , and obtain the spatial enhancement result of hyperspectral data .

[0027] S305: Expand the normalized hyperspectral and multispectral data into their respective corresponding Casorati matrices and , and Transpose to get .

[0028] S306: Densely connected pairs of dense blocks using two fully connected layers and ReLu function cascade and Perform feature extraction and Features extracted from dense blocks and Fusion , and enter the branch where the multispectral data is located. A dense block.

[0029] Specifically, the feature output of the first dense block in the two-way dense connection and And the feature input of the third dense block of the branch where the hyperspectral and multispectral data are located and For example: , , , .in, , , , , , , , yes The transpose of represents a dense block of fully connected layers and ReLu functions cascaded, Indicates dense connection, that is, the feature matrix is ​​concatenated along the column dimension.

[0030] S307: Output the last dense block of the two branches and The fusion result Reconstruction into three-dimensional data , which is the spectral enhancement result of multispectral data.

[0031] Furthermore, step S4 specifically includes the following sub-steps:

[0032] S401: respectively and Input two two-dimensional convolutional layers and the Softmax activation function to get the corresponding attention map and .

[0033] S402: Under the cross guidance of the above attention map, the multiplication and addition units are used to achieve and The spatial spectrum fusion is performed to obtain the preliminary fusion result. : .in, Represents an element-wise multiplication operation.

[0034] S403: Use multiple layers of 3D convolutional layers to promote spatial-spectral fusion and obtain the final fusion result : .in, Represents multiple layers of 3D convolutional layers.

[0035] Furthermore, step S5 specifically includes the following sub-steps:

[0036] S501: Using the point spread function to characterize hyperspectral sensors Perform spatial degradation processing on the fusion result to obtain reconstructed hyperspectral data : .in, Represents a downsampling operation.

[0037] S502: Using the spectral response function that describes the characteristics of the multi-spectral sensor The fusion results Perform spectral degradation processing to obtain reconstructed multispectral data : Among them, the subscript Represents modulo 3 multiplication.

[0038] Furthermore, step S6 specifically includes the following sub-steps:

[0039] S601: Based on normalized hyperspectral data , Normalized multispectral data , estimated strip component , Hyperspectral data enhancement results , Reconstructing hyperspectral data , Multispectral data enhancement results , and reconstruction of multispectral data Calculating the loss function : .in, is the regularization parameter, is the longitudinal gradient operator.

[0040] S602: By minimizing the loss function , use Adma optimizer to optimize the network parameters involved in S2-S4 until The value is smaller than the preset value.

[0041] S603: Output the result of the spatial-spectral fusion module , and then restore its amplitude to obtain high-resolution hyperspectral data fused with high-resolution and multi-spectral data. : . Beneficial Effects

[0042] 1. The hyperspectral image super-resolution method provided by the present invention designs a stripping network using the self-attention mechanism and residual structure, which effectively eliminates the stripe noise commonly present in hyperspectral data and improves the robustness of image fusion to stripe noise.

[0043] 2. The hyperspectral image super-resolution method provided by the present invention extracts the fine spatial features of multispectral data and injects them into the hyperspectral data to improve its spatial resolution, extracts the identification spectral features of hyperspectral data and injects them into the multispectral data to improve its spectral resolution, and realizes data resolution enhancement under the premise of maintaining the original spatial spectrum information of hyperspectral and multispectral data.

[0044] 3. The hyperspectral image super-resolution method provided by the present invention utilizes the cross-attention mechanism to realize the selective spatial-spectral fusion of hyperspectral and multispectral data enhancement results, fully exploits and integrates the recognition spectrum and fine spatial detection advantages of the two source data, and finally obtains high spatial and high spectral resolution data. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0046] Figure 1 A flow chart is provided for the present invention. DETAILED DESCRIPTION

[0047] The present invention provides a method for deep robust fusion of high-multispectral images for watershed detection, which can achieve the fusion of fine spatial features and identification spectral features of two source data while maintaining the spatial spectrum information of hyperspectral and multispectral data, thereby obtaining high spatial and high spectral resolution images, providing a data basis for fine watershed detection. Figure 1 shown.

[0048] The process of the present invention is described below with specific embodiments:

[0049] In this embodiment, the hyperspectral and multispectral remote sensing images are taken from the same area of ​​the Yellow River Estuary. The hyperspectral image contains stripe noise with a size of , the multispectral image size is , taking two-source remote sensing data with a spatial resolution difference of three times as an example, the super-resolution method provided by the present invention is specifically described. The present invention provides a high-multispectral image deep robust fusion method for watershed detection as follows:

[0050] Step S1, respectively normalize the observed hyperspectral and multispectral data. Specifically, it includes the following sub-steps:

[0051] S101: Observing Hyperspectral Data Divide each pixel in the data by the maximum value of all pixels in the data, and then assign all pixel values ​​of the hyperspectral data to the interval .in, , , Represents the number of rows, columns, and channels of hyperspectral data respectively.

[0052] S102: Observe multispectral data Divide each pixel in the data by the maximum value of all pixels in the data, and then put all pixel values ​​of the multispectral data into the interval .in, , , represents the number of rows, columns, and channels of multispectral data, respectively, and , , .

[0053] Step S2, based on the three-dimensional convolution and residual structure, the strip stripping module is designed to effectively estimate the three-dimensional strip components in the hyperspectral data and realize strip stripping under the premise of ensuring the spatial spectrum structure of the hyperspectral data. Specifically, it includes the following sub-steps:

[0054] S201: Normalizing hyperspectral data using a 3D convolutional layer and a ReLu function cascaded input layer Perform primary characterization to obtain the primary feature tensor , where the three-dimensional convolutional layer contains The size is 3D convolution kernel.

[0055] S202: Using the self-attention mechanism and the self-attention residual block constructed by the residual structure, the primary features are deepened and selectively transferred to obtain the advanced feature tensor ,in, , indicating the A self-attention residual block.

[0056] Specifically, the feature output of the first self-attention residual block For example: ,in, and Describe the self-attention mechanism and the dense blocks of the cascade of three-dimensional convolutional layer, BN layer and ReLu function, namely The output of the self-attention residual block From the previous level feature tensor Self-attention selection features Its deepening characteristics of superposition.

[0057] S203: Further integration using short connections between 3D convolutional layers and BN layers The deep and shallow features contained in it are The cascade of the three-dimensional convolutional layer and the tanh activation function reduces the dimension of the fusion feature and estimates the normalized hyperspectral data Medium stripe weight .

[0058] Step S3, based on two-dimensional convolution and dense connection, the spatial enhancement module is designed to extract the fine spatial features of multispectral data to supplement the spatial information of hyperspectral data; at the same time, based on the fully connected network and dense connection, the spectral enhancement module is designed to extract the identification spectral features of hyperspectral data to supplement the spectral information of multispectral data. Specifically, it includes the following sub-steps:

[0059] S301: Observing the spatial dimensions of multispectral data As the target, bicubic interpolation is used to normalize the hyperspectral data. Perform super-resolution initialization processing to obtain .

[0060] S302: Super-resolution initialization results of hyperspectral data using two-dimensional convolutional layer and short connection of ReLu function and normalized multispectral data Perform feature processing to obtain the primary feature tensor and , where the two-dimensional convolutional layer contains The size is 3D convolution kernel.

[0061] S303: The primary feature tensor is respectively processed by densely connecting dense blocks of two-dimensional convolutional layers, BN layers and ReLu functions. and To deepen, add units to deepen the features Injection Features and enter the branch where the hyperspectral data is located. dense blocks, where and Respectively represent the first The output of a dense block.

[0062] Specifically, the feature output of the first dense block in the two-way dense connection and And the feature input of the second dense block of the branch where the hyperspectral and multispectral data are located and For example: , , , .in, , , represents a dense block of cascaded 2D convolutional layers, BN layers, and ReLu functions, Represents dense connection, that is, the feature tensors are concatenated along the channel dimension.

[0063] S304: Use the short connection of the two-dimensional convolution layer and the tanh activation function cascade to output the features of the last dense block of the branch where the hyperspectral data is located Mapped to the image domain and injected into it through the addition unit , and obtain the spatial enhancement result of hyperspectral data .

[0064] S305: Expand the normalized hyperspectral and multispectral data into their respective corresponding Casorati matrices and , and Transpose to get .

[0065] S306: Densely connected pairs of dense blocks using two fully connected layers and ReLu function cascade and Perform feature extraction and Features extracted from dense blocks and Fusion , and enter the branch where the multispectral data is located. A dense block.

[0066] Specifically, the feature output of the first dense block in the two-way dense connection and And the feature input of the third dense block of the branch where the hyperspectral and multispectral data are located and For example: , , , .in, , , , , , , , yes The transpose of represents a dense block of fully connected layers and ReLu functions cascaded, Indicates dense connection, that is, the feature matrix is ​​concatenated along the column dimension.

[0067] S307: Output the last dense block of the two branches and The fusion result Reconstruction into three-dimensional data , which is the spectral enhancement result of multispectral data.

[0068] Step S4, designing a spatial-spectral fusion module based on the attention mechanism, and selectively fusing the spatial enhancement results of the hyperspectral data and the spectral enhancement results of the multispectral data. Specifically, it includes the following sub-steps:

[0069] S401: respectively and Input two two-dimensional convolutional layers and the Softmax activation function to get the corresponding attention map and .

[0070] S402: Under the cross guidance of the above attention map, the multiplication and addition units are used to achieve and The spatial spectrum fusion of , and the preliminary fusion result is obtained: .in, Represents an element-wise multiplication operation.

[0071] S403: Use multiple layers of 3D convolutional layers to promote spatial-spectral fusion and obtain the final fusion result .in, Represents multiple layers of 3D convolutional layers.

[0072] Step S5, based on the linear degradation model, performs spatial and spectral degradation processing on the fused data to reconstruct the hyperspectral and multispectral inputs. Specifically, it includes the following sub-steps:

[0073] S501: Using the point spread function to characterize hyperspectral sensors Perform spatial degradation processing on the fusion result to obtain reconstructed hyperspectral data : .in, Represents a downsampling operation.

[0074] S502: Using the spectral response function that describes the characteristics of the multi-spectral sensor The fusion results Perform spectral degradation processing to obtain reconstructed multispectral data : .

[0075] Step S6, by minimizing the loss function, optimize the parameters of the stripping, spatial spectrum enhancement and spatial spectrum fusion modules, and output the result of the spatial spectrum fusion module. Specifically, it includes the following sub-steps:

[0076] S601: Based on normalized hyperspectral data , Normalized multispectral data , estimated strip component , Hyperspectral data enhancement results , Reconstructing hyperspectral data , Multispectral data enhancement results , and reconstruction of multispectral data Calculating the loss function : .in, is the regularization parameter, is the longitudinal gradient operator.

[0077] S602: By minimizing the loss function , use Adma optimizer to optimize the network parameters involved in S2-S4 until The value is smaller than the preset value.

[0078] S603: Output the result of the spatial-spectral fusion module , and then restore its amplitude to obtain high-resolution hyperspectral data fused with high-resolution and multi-spectral data. : .

Claims

1. A deep robust fusion method of hyperspectral images for watershed detection, characterized in that: The following steps are involved: S1, respectively performing amplitude normalization processing on the observed hyperspectral data and multispectral data to obtain normalized hyperspectral data and normalized multispectral data; S2, based on three-dimensional convolution and residual structure, the stripping module is designed to estimate the three-dimensional strip components in the hyperspectral data, and realize stripping under the premise of ensuring the spatial spectrum structure of the hyperspectral data; the stripping module uses a three-dimensional convolution layer for feature extraction and representation; the stripping module uses a residual structure that can represent additive operations to connect the three-dimensional convolution layer, and uses the self-attention mechanism to guide the transmission of important information in the shallow features; S3, based on two-dimensional convolution and dense connection, the spatial enhancement module is designed to extract the fine spatial features of multispectral data to supplement the spatial information of hyperspectral data, and obtain the hyperspectral data enhancement result; at the same time, based on the fully connected network and dense connection, the spectral enhancement module is designed to extract the identification spectral features of hyperspectral data to supplement the spectral information of multispectral data, and obtain the multispectral data enhancement result; The spatial enhancement module is composed of two networks with the same structure connected in parallel. One uses two-dimensional convolution to extract multi-level fine spatial features from multispectral data, and uses dense connection to realize feature connection between multi-layer networks to prevent feature omission during feature deepening. The other uses the same structure to perform feature processing on hyperspectral data to ensure that the spatial features of the two source data are injected and fused at a consistent representation angle. The spectral enhancement module is also composed of two networks with the same structure connected in parallel. One uses a fully connected network to extract spectral features from the hyperspectral data matrix, and uses dense connections to achieve multi-level feature reuse and prevent feature omissions. The other performs feature processing on multispectral data to ensure that the spectral features of the two source data are injected and fused at a consistent representation angle. S4, a spatial-spectral fusion module is designed based on the attention mechanism to selectively fuse the spatial enhancement results of hyperspectral data and the spectral enhancement results of multispectral data; Under the guidance of the cross-attention mechanism, the spatial-spectral fusion module calculates two-dimensional attention maps based on the hyperspectral and multispectral data enhancement results, and uses the obtained attention maps to guide the selective fusion of multispectral and hyperspectral data enhancement results. Finally, multiple three-dimensional convolutional layers are used to further promote the fusion to generate high-resolution hyperspectral data. S5, performs spatial and spectral degradation processing on the fused data based on the linear degradation model, reconstructs the hyperspectral data and reconstructs the multispectral data input; S6, based on the normalized hyperspectral data, the normalized multispectral data, the estimated strip components, the hyperspectral data enhancement results, the multispectral data enhancement results, the reconstructed hyperspectral data and the reconstructed multispectral data, calculates the loss function, realizes parameter optimization by minimizing the loss function, and outputs the result of the spatial-spectral fusion module.

2. The method for deep robust fusion of hyperspectral images for watershed detection according to claim 1, characterized in that: Step S1 specifically includes the following sub-steps: S101: Observing Hyperspectral Data Each pixel in the data is divided by the maximum value of all pixels in the data, and then all pixel values ​​of the hyperspectral data are placed in the interval [0,1]; where h, w, and C represent the number of rows, columns, and channels of the hyperspectral data respectively; S102: Observe multispectral data Each pixel in the multispectral data is divided by the maximum value of all pixels in the data, and then all pixel values ​​of the multispectral data are placed in the interval [0,1]; where H, W, and c represent the number of rows, columns, and channels of the multispectral data respectively, and h<<H, w<<W, and c<<C.

3. The method for deep robust fusion of hyperspectral images for watershed detection according to claim 2, characterized in that: Step S2 specifically includes the following sub-steps: S201: Normalizing hyperspectral data using a 3D convolutional layer and a ReLu function cascaded input layer Perform primary characterization to obtain the primary feature tensor Among them, the three-dimensional convolution layer contains n three-dimensional convolution kernels of size p×p×p; S202: Using the self-attention mechanism and the self-attention residual block constructed by the residual structure, the primary features are deepened and selectively transferred to obtain the advanced feature tensor Among them, k = 1, 2, 3, ..., represents the kth self-attention residual block; S203: Further integration using short connections between 3D convolutional layers and BN layers The deep and shallow features contained in it are then reduced in dimension using a cascade of a 3D convolutional layer with a kernel size of 1×1×1 and a tanh activation function to estimate the normalized hyperspectral data. Medium strip weight 4. The method for deep robust fusion of hyperspectral images for watershed detection according to claim 3, characterized in that: Step S3 specifically includes the following sub-steps: S301: Taking the spatial size H×W of observed multispectral data as the target, bicubic interpolation is used to normalize the hyperspectral data Perform super-resolution initialization processing to obtain S302: Super-resolution initialization results of hyperspectral data using two-dimensional convolutional layer and short connection of ReLu function and normalized multispectral data Perform feature processing to obtain the primary feature tensor and Among them, the two-dimensional convolution layer contains N1 three-dimensional convolution kernels of size p×p; S303: The primary feature tensor is respectively processed by densely connecting dense blocks of two-dimensional convolutional layers, BN layers and ReLu functions. and To deepen, add units to deepen the features Injection Features And input the k+1th dense block of the branch where the hyperspectral data is located, where, and Respectively represent the output of the kth dense block in the two dense connections; S304: Use the short connection of the two-dimensional convolution layer and the tanh activation function cascade to output the features of the last dense block of the branch where the hyperspectral data is located Mapped to the image domain and injected into it through the addition unit Get the spatial enhancement result of hyperspectral data S305: Expand the normalized hyperspectral and multispectral data into their respective corresponding Casorati matrices and And transpose X to get S306: Use the dense connection of the dense blocks of the two fully connected layers and the ReLu function cascade to extract features from X′ and Y, and extract the features from the kth dense block. and Fusion And input the k+1th dense block in the branch where the multispectral data is located; S307: Output the last dense block of the two branches and The fusion result Reconstruction into three-dimensional data This is the spectral enhancement result of multispectral data.

5. The method for deep robust fusion of hyperspectral images for watershed detection according to claim 4, characterized in that: Step S4 specifically includes the following sub-steps: S401: respectively and Input two two-dimensional convolutional layers and the Softmax activation function to get the corresponding attention map and S402: Under the cross guidance of the above attention map, the multiplication and addition units are used to achieve and The spatial spectrum fusion is performed to obtain the preliminary fusion result. Among them, ⊙ represents the element-level multiplication operation; S403: Use multiple layers of 3D convolutional layers to promote spatial-spectral fusion and obtain the final fusion result Among them, f m (·) indicates multiple layers of 3D convolutional layers.

6. The method for deep robust fusion of hyperspectral images for watershed detection according to claim 5, characterized in that: Step S5 specifically includes the following sub-steps: S501: Using the point spread function to characterize hyperspectral sensors Perform spatial degradation processing on the fusion result to obtain reconstructed hyperspectral data Where ds(·) represents the downsampling operation; S502: Using the spectral response function that describes the characteristics of the multi-spectral sensor The fusion results Perform spectral degradation processing to obtain reconstructed multispectral data Here, the subscript ×3 indicates modulo 3 multiplication.

7. The method for deep robust fusion of hyper-multispectral images for watershed detection according to claim 6, characterized in that: Step S6 specifically includes the following sub-steps: S601: Based on normalized hyperspectral data Normalizing multispectral data Estimated strip component Hyperspectral data enhancement results Reconstructing hyperspectral data Multispectral data enhancement results and reconstruction of multispectral data Calculate the loss function L: Among them, λ1,λ2>0 are regularization parameters, is the longitudinal gradient operator; S602: By minimizing the loss function L, the network parameters involved in S2-S4 are optimized using the Adma optimizer until the value of L is less than a preset value; S603: Output the result of the spatial-spectral fusion module Then restore the amplitude to obtain high-resolution hyperspectral data fused with high-resolution and multi-spectral data.

Images