Deep cycle hyperspectral image processing method for ocean sub-pixel target identification

Abstract

The invention discloses a deep circulation hyperspectral image processing method for ocean sub-pixel target identification. According to the hyperspectral image processing method based on simultaneous feature extraction of the high-resolution feature space and the low-resolution feature space, the spectral information processing technology of multi-channel segmentation adaptive weighting and channel modulation is combined, parallel unmixing is carried out on the two types of information through a space spectrum combined network, and the unmixing precision in a complex scene is remarkably improved. The method can perform efficient unmixing and accurate classification on complex spectrum and spatial information in the hyperspectral image, has relatively high robustness and adaptability, and is particularly suitable for the fields of marine environment monitoring, resource exploration, environmental protection and the like; the method can overcome the defects of a traditional method in high-dimensional features and generality in the aspect of spectrum separation and classification of complex landforms, and has important application value.

Description

Technical Field

[0001] The present invention belongs to the technical field of hyperspectral image processing, and in particular relates to a deep cycle hyperspectral image processing method for ocean sub-pixel target recognition. Background Art

[0002] With the rapid development of satellite remote sensing technology, people generally realize that hyperspectral images not only record spatial features on a two-dimensional plane, but also contain spectral information with hundreds of bands. These spectral information can reflect the different material components of the surface and water bodies. Therefore, hyperspectral remote sensing is widely used in environmental monitoring, agriculture, forestry, urban planning and other fields, especially in marine remote sensing. In the marine field, multiple components such as seawater, phytoplankton, suspended matter, benthic organisms and pollutants often exist at the same time, and their spectral features will be mixed, resulting in the mixed pixel problem of hyperspectral images. The unmixing of hyperspectral images involves solving two main challenges: (1) determining the component endmembers, that is, the spectral characteristics of these objects, and (2) estimating the proportion of these endmembers in each hyperspectral pixel, also known as abundance estimation.

[0003] The linear unmixing model considers the spectrum of each mixed pixel as a linear combination of known endmember spectra. In recent years, unmixing methods for hyperspectral images can be mainly divided into traditional spectral-based methods and modern methods that combine spatial information. (1) Traditional unmixing methods are mainly divided into geometric, statistical and sparse regression, which usually rely only on spectral information. These methods either analyze the geometric relationship between pixels to extract endmembers for unmixing, or incorporate prior knowledge into unmixing by constructing a probabilistic relationship between observed data and model parameters. It is difficult to effectively utilize spatial information, resulting in limitations in the analysis of mixed pixels. (2) Deep learning-based methods can be divided into two categories: methods based on the deep convolutional neural network (CNN) framework and methods based on the autoencoder (AE) framework. CNN-based methods extract features by stacking convolutional layers and activation functions, which usually leads to large parameters and overfitting. Autoencoder-based methods can effectively capture important information in hyperspectral data, reduce redundancy, and improve the accuracy of unmixing and robustness to noise. However, these methods also face challenges, such as high data dimensionality. Deep learning models need to effectively process these high-dimensional data to avoid excessive computational overhead or the "dimensionality disaster". At the same time, too many bands may contain redundant information, and how to effectively extract useful features and reduce the computational burden is an important challenge. Summary of the invention

[0004] The purpose of the present invention is to provide a deep cycle hyperspectral image processing method for ocean sub-pixel target recognition to make up for the shortcomings of the prior art.

[0005] The present invention effectively applies multi-feature fusion and spatial information in the unmixing model based on the autoencoder (AE). The network can make full use of high-resolution features and explore more representative features to promote better recovery and solve the technical problems faced by existing hyperspectral image processing methods, such as complex spectral mixing, insufficient unmixing accuracy, and noise interference. The present invention takes into account factors such as spatial receptive field and multi-scale feature fusion, and shows competitive performance in endmember extraction and abundance estimation, providing an effective solution for hyperspectral unmixing tasks.

[0006] In order to achieve the above object, the specific technical scheme adopted by the present invention is as follows:

[0007] A deep cycle hyperspectral image processing method for ocean sub-pixel target recognition comprises the following steps:

[0008] S1: Obtain a hyperspectral image (HSI) matrix containing L spectral segments and n mixed pixels, denoted as HSI(L,n);

[0009] S2: Set the window size to r*r, split HSI(L,n), and build a window centered on each pixel;

[0010] S3: Construct a deep recurrent hyperspectral image processing model, which includes a high-resolution spatial feature module and a low-resolution spatial feature module; the high-resolution spatial feature module includes three BiRNN modules and an adaptive channel modulator; the low-resolution spatial feature module includes three high-low feature fusion modules, a convolution module and a gating mechanism;

[0011] S4: High-resolution and low-resolution spatial features are jointly learned. The former learns multi-level high-resolution features, fuses low-resolution features and reconstructs the residual image, while the latter explores more representative features from high-resolution features, and finally extracts features through the gating mechanism to obtain the abundance result; In the high-resolution spatial feature module, the input image I∈R r×r×L First, after three BiRNN modules, the intermediate features obtained by each BiRNN module are recorded as Where j∈{1,2,3}; the three intermediate features are input into the adaptive channel modulator and fused to obtain the feature In the low-resolution spatial feature module, there are three high- and low-feature fusion modules. First, I and Input the first high and low feature fusion module for modulation to obtain I new , I new The feature fusion after participating in the new I; the feature obtained after low-resolution space learning and After splicing, a 1×1 convolution and gating mechanism are used to obtain the predicted abundance;

[0012] S5: inputting the abundance result into a decoder to restore it into a vector with dimension L to obtain a predicted end member spectrum;

[0013] S6: The deep recurrent hyperspectral image processing model is trained and optimized through a loss function to obtain the final end members and abundances.

[0014] Furthermore, in S1: the hyperspectral data includes but is not limited to NASA's ASTER: providing hyperspectral images of the earth's surface, including 14 spectral bands, covering the visible light to thermal infrared region. NASA's Hyperion: The Hyperion satellite provides high-resolution hyperspectral data with 220 bands, covering the visible light to short-wave infrared. These satellite data can be downloaded or purchased for free through public remote sensing data portals (such as NASA Earthdata, ESA Copernicus, USGS EarthExplorer, etc.). The obtained data is used to determine the number of spectral segments and mixed pixels according to different application fields or the contribution to the unmixing task, recorded as HSI (L, n).

[0015] Furthermore, S2 specifically includes: setting the window size to r, for any pixel point x, starting to divide the window with the pixel point as the center point, and because the edge pixels of the hyperspectral image cannot be used to construct a window with the pixel point as the center point, they are discarded; that is, setting the length and width of HSI (L, n) to c and k, the entire hyperspectral image is divided into a total of (c-1) × (k-1) windows.

[0016] Further, the S4 includes:

[0017] (1) Learning in high-resolution space:

[0018] Given an input window I∈R r×r×L , first through three BiRNN modules, the jth BiRNN module has two submodules divided into vertical and horizontal modules where j∈{1,2,3}, the vertical module divides a window into r two-dimensional vectors along the vertical direction These r vectors are passed through a bidirectional recurrent neural network to generate r Then splice horizontally into Then, the multi-channel segmentation of the BiRNN module and the adaptive weighting module highlight the effective features to generate the final The horizontal module divides a window into r two-dimensional vectors along the horizontal direction These r vectors are passed through a bidirectional recurrent neural network to generate r Then vertically spliced ​​into Then, the multi-channel segmentation and adaptive weighting module highlight the effective features to generate the final Then and Concatenate along the channel dimension to get the output of the j-th BiRNN module where j∈{1,2,3}.

[0019] Will The multi-level features are modulated by the adaptive channel modulator, and the modulated features I ACM ∈R r×r×3L is sent to the low-resolution space and participates in the high-low feature fusion module operation in the low-resolution space; finally, a 1×1 convolution and gating mechanism receives and the shuffled features learned from the low-resolution space to obtain the predicted abundance;

[0020] (2) Learning in low-resolution space

[0021] First, receive the input window I∈R r×r×L Then, I is sent to several high-low feature fusion modules for learning low-resolution features; at the same time, each high-low feature fusion module will also match the modulation feature I in the adaptive channel modulator ACM ∈R r×r×3L In order to generate more representative features, the output of the high and low feature fusion module is used as the new I∈R r×r×L , thereby promoting better recovery.

[0022] Furthermore, in the multi-channel segmentation and adaptive weighting module:

[0023] The size of each input is Where r is the window size, L is the number of original spectral segments; for each input, the weight coefficient of each channel is first calculated through multi-channel segmentation and adaptive weighting mechanism. This weight coefficient helps to adjust the features according to the mutual relationship between channels. This weighting mechanism helps to highlight important features while suppressing irrelevant features. Finally, all convolution outputs will be recombined in the channel dimension. In this way, the advantages of segmentation and reorganization can be combined to highlight the effective channel features.

[0024] First, divide x into S parts along the channel, and for each part x f , except x f The channel features other than f The complement of Next, and It is passed to the adaptive weighted network, through which the curve parameters are estimated to adjust the pixel value range of the feature. Finally, it is spliced ​​together in the channel dimension to obtain

[0025] Furthermore, in the adaptive channel modulator:

[0026] Given First, concatenate them along the channel dimension into Then use 1×1 convolution to expand to X∈R r×r×6L , and split the features into two tensors: X 1 ∈R r×r×3L and X 2 ∈R r×r×3L Next, in X 1 A softmax operation is performed along the channel dimension to obtain the channel weight coefficient. The specific formula is as follows:

[0027]

[0028] Where SoftMax is a SoftMax function and ⊙ is element-wise multiplication.

[0029] Furthermore, in the high and low feature fusion module:

[0030] Will I ACM ∈R r×r×3L With I∈R r×r×L First, concatenate them together in the channel dimension to form x∈R r×r×4L , through a bidirectional recurrent neural network, and then weighted by a channel attention, and finally through a 1×1 convolution to get the new I∈R r×r×L , the formula is as follows:

[0031] I new =Conv(CA(BiRNN(Concat(I ACM ,I))))∈R r×r×L

[0032] Among them, Concat is channel concatenation, BiRNN is a bidirectional recurrent neural network, CA is channel attention, and Conv is a 1×1 convolution.

[0033] Furthermore, in S5, the fused features are decoded by the decoder. Reconstructed into the original spectral dimension LL, the reconstruction process is expressed as:

[0034]

[0035] Among them, W decode ∈R L×2c is the weight matrix of the decoder, bdecode ∈R L is the bias vector. The final output is is the predicted endmember spectrum after reconstruction.

[0036] Furthermore, in S6, in order to reduce the difference between the network prediction result and the true value, the spectral angular distance (SAD) is used as the basic loss function. Considering the sparsity of the abundance vector, the abundance vector is selected to be Regularization is used to achieve better reconstruction. This regularization method makes most elements of the abundance vector zero, thereby enhancing sparsity. The loss function is expressed as:

[0037]

[0038] The root mean square error (RMSE) and spectral angular distance (SAD) are used to evaluate the performance of the model on the dataset. The root mean square error is usually used to compare the abundance estimates of each pixel. The formula is as follows:

[0039]

[0040] where x c is the actual abundance of the pixel, is the predicted abundance of the pixel, and n is the number of pixels.

[0041] Spectral Angle Distance (SAD) is a metric used to measure the spectral similarity in hyperspectral images. It evaluates the similarity of two spectral vectors by calculating the angle between them, rather than directly calculating the numerical difference. Therefore, it is insensitive to changes in spectral features (such as changes in spectral brightness), but focuses more on the similarity of spectral shapes. The formula is as follows:

[0042]

[0043] where a c is the true spectral characteristic of the ground object. It is the predicted spectral feature of the ground object.

[0044] The Adam optimization algorithm is used to optimize the network parameters. The update formula is: Among them, θ t represents the network parameters, and α is the learning rate.

[0045] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows:

[0046] The hyperspectral image processing method based on simultaneous feature extraction of high- and low-resolution feature spaces provided by the present invention combines the spectral information processing technology of multi-channel segmentation adaptive weighting and channel modulation, and uses a spatial-spectral combined network to perform parallel unmixing of two types of information, significantly improving the unmixing accuracy in complex scenes. Specifically, it includes the following:

[0047] 1. Use the BiRNN module to extract spatial and spectral information in high-resolution space to improve unmixing accuracy.

[0048] The present invention divides the input window along the vertical and horizontal directions respectively, performs bidirectional recurrent neural network processing in each direction, and then uses multi-channel segmentation and adaptive weighting modules to highlight effective features, thereby avoiding the interference of useless information. The model can capture the multi-level features of the image more comprehensively. This method enhances the extraction of features in different directions and improves the expression ability of the model.

[0049] 2. Use high-low feature fusion modules to preserve detail information in low-resolution space and promote high-quality extraction effects.

[0050] In each high- and low-resolution feature fusion module, the input low-resolution features are combined with the modulated features from the adaptive channel modulator to further enhance the multi-dimensionality of the feature representation. Through this multi-dimensional feature enhancement, the model is able to fine-tune the different levels and details of the image to ensure that no important information is lost.

[0051] 3. End-to-end deep learning framework to improve demixing efficiency and robustness

[0052] The present invention adopts an end-to-end deep learning framework, and integrates spectral feature extraction, spatial feature fusion, unmixing and data reconstruction in the same network by constructing feature conversion from high-resolution space to low-resolution space. This architecture realizes the simultaneous processing of spectral and spatial information through recurrent neural networks and convolution operations, and automatically learns the optimal feature representation. By using the Adam optimization algorithm, the model can converge quickly and maintain a high unmixing accuracy in complex scenarios. This end-to-end approach not only improves processing efficiency and reduces the complexity of manual parameter adjustment, but also enhances the robustness of the model in different data sets and application scenarios.

[0053] The present invention can efficiently unmix and accurately classify the complex spectral and spatial information in hyperspectral images, has strong robustness and adaptability, and is particularly suitable for the fields of marine environmental monitoring, resource exploration, and environmental protection. In the separation and classification of complex landform spectra, it can overcome the shortcomings of traditional methods in high-dimensional features and versatility, and has important application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a flow chart of the present invention.

[0055] Figure 2 It is a schematic diagram of the process of the hyperspectral processing method of the present invention.

[0056] Figure 3 Schematic diagram of input processing of the network model of the present invention.

[0057] Figure 4 It is a schematic diagram of the hyperspectral input data of the present invention.

[0058] Figure 5 This is the end member effect diagram of the method in Example 2.

[0059] Figure 6 This is the abundance effect diagram of the method in Example 2. DETAILED DESCRIPTION

[0060] In the following description, various aspects of the present invention will be described in detail. However, for those of ordinary skill in the art, the present invention may be implemented using only some or all of the structures and processes of the present invention. For ease of illustration, specific quantities, configurations, and sequences are mentioned in the description, but it is clear that the present invention can be implemented without these specific details. In other cases, some well-known features of the present invention will not be described in detail to avoid confusion.

[0061] Embodiment 1:

[0062] A deep cycle hyperspectral image processing method for ocean sub-pixel target recognition, such as Figure 1 , 2 As shown, the following steps are included:

[0063] Step 1: Obtain hyperspectral data and determine the number of spectral segments and hybrid pixels.

[0064] Currently, there are many remote sensing satellites and spacecraft that provide hyperspectral data, including but not limited to NASA's ASTER: providing hyperspectral images of the surface, including 14 spectral bands, covering from visible light to thermal infrared. NASA's Hyperion: The Hyperion satellite provides high-resolution hyperspectral data with 220 bands, covering from visible light to short-wave infrared. These satellite data can be downloaded or purchased for free through public remote sensing data portals (such as NASA Earthdata, ESACopernicus, USGSEarthExplorer, etc.). The obtained data is used to determine the number of spectral segments and mixed pixels according to different application fields or the contribution to the unmixing task, recorded as HSI (L, n).

[0065] Step 2: Set the window size to r*r, split HSI(L,n), and build a window centered on each pixel.

[0066] A hyperspectral image has tens of thousands of pixels and hundreds or even hundreds of bands. The amount of data is too large to be directly input into the model, so it is very necessary to preprocess the image data in advance. Set the window size to r, and for any pixel x, start dividing the window with the pixel as the center point. Because the edge pixels of the hyperspectral image cannot be used to construct a window with it as the center point, they are discarded. That is, let the length and width of HSI (L, n) be c and k, and the entire hyperspectral image can be divided into a total of (c-1) × (k-1) windows.

[0067] Step 3: High-resolution and low-resolution spaces jointly learn features. The former learns multi-level high-resolution features, fuses low-resolution features and reconstructs the residual image, while the latter explores more representative features from high-resolution features to promote better recovery. Finally, feature extraction is performed through a gating mechanism to obtain the abundance result.

[0068] Step 3.1: Learning in high-resolution space

[0069] like Figure 3 , 4 As shown, given an input window I∈R r×r×L , first through three BiRNN modules, the jth BiRNN module has two submodules divided into vertical and horizontal modules where j∈{1,2,3}, the vertical module divides a window into r two-dimensional vectors along the vertical direction These r vectors are passed through a bidirectional recurrent neural network to generate r Then splice horizontally into Then, the multi-channel segmentation and adaptive weighting module proposed in (3.3) highlight the effective features to generate the final The horizontal module divides a window into r two-dimensional vectors along the horizontal direction These r vectors are passed through a bidirectional recurrent neural network to generate r Then vertically spliced ​​into Then, the multi-channel segmentation and adaptive weighting module proposed in (3.3) highlight the effective features to generate the final Then and Concatenate along the channel dimension to get the output of the j-th BiRNN module where j∈{1,2,3}.

[0070] Will The multi-level features are modulated by the adaptive channel modulator, and the modulated features IACM ∈R r×r×3L It is sent to the low-resolution space and participates in the high-low feature fusion module operation proposed in the low-resolution space (3.5). Finally, a 1×1 convolution and gating mechanism receives and the shuffled features learned from the low-resolution space to obtain the predicted abundance.

[0071] Step 3.2: Learning in low-resolution space

[0072] First, receive the input window I∈R r×r×L Then, I is sent to several high-low feature fusion modules proposed in (3.5) to learn low-resolution features. At the same time, each high-low feature fusion module will also match the modulation feature I in the adaptive channel modulator ACM ∈R r×r×3L In order to generate more representative features, the output of the high and low feature fusion module is used as the new I∈R r×r×L , thereby promoting better recovery.

[0073] Step 3.3: Multi-channel segmentation and adaptive weighting module

[0074] The size of each input is Where r is the window size and L is the number of original spectral segments. For each input, the weight coefficient of each channel is first calculated through multi-channel segmentation and adaptive weighting mechanism. This weight coefficient helps to adjust the features according to the mutual relationship between channels. This weighting mechanism helps to highlight important features while suppressing irrelevant features. Finally, all convolution outputs are recombined in the channel dimension. In this way, the advantages of segmentation and reorganization can be combined to highlight the effective channel features.

[0075] First, divide x into S parts along the channel, and for each part x f , except x f The channel features other than f The complement of Next, and It is passed to the adaptive weighted network, through which the curve parameters are estimated to adjust the pixel value range of the feature. Finally, it is spliced ​​together in the channel dimension to obtain

[0076] Step 3.4: Adaptive Channel Modulator

[0077] Given First, stitch them together Then use 1×1 convolution to expand to X∈R r×r×6L , and split the features into two tensors: X 1 ∈Rr×r×3L and X 2 ∈R r×r×3L Next, in X 1 A softmax operation is performed along the channel dimension to obtain the channel weight coefficient. The specific formula is as follows:

[0078]

[0079] Step 3.5: High and low feature fusion module

[0080] Will I ACM ∈R r×r×3L With I∈R r×r×L First, concatenate them together in the channel dimension to form x∈R r×r×4L , through a bidirectional recurrent neural network, and then weighted by a channel attention, and finally through a 1×1 convolution to get the new I∈R r×r×L , the formula is as follows:

[0081]

[0082] Step 4: Input the abundance result into the decoder and restore it into a vector of dimension L to obtain the predicted end member spectrum.

[0083] The fused features are decoded Reconstructed into the original spectral dimension LL, the reconstruction process can be expressed as:

[0084] Among them, W decode ∈R L×2c is the weight matrix of the decoder, b decode ∈R L is the bias vector. The final output is is the predicted endmember spectrum after reconstruction.

[0085] Step 5: Train and optimize the model through the loss function to obtain the end member and abundance with the minimum error.

[0086] In order to reduce the difference between the network prediction result and the true value, the present invention uses the spectral angular distance (SAD) as the basic loss function. Considering the sparsity of the abundance vector, the present invention chooses to perform Regularization is used to achieve better reconstruction. This regularization method makes most elements of the abundance vector zero, thereby enhancing sparsity. The loss function can be expressed as:

[0087]

[0088] The root mean square error (RMSE) and spectral angular distance (SAD) are used to evaluate the performance of the model on the data set. The root mean square error is usually used to compare the difference between the abundance estimate of each pixel or the spectral reconstruction value of each pixel and the true value. The formula is as follows:

[0089]

[0090] Spectral Angle Distance (SAD) is a metric used to measure the spectral similarity in hyperspectral images. It evaluates the similarity of two spectral vectors by calculating the angle between them, rather than directly calculating the numerical difference. Therefore, it is insensitive to changes in spectral features (such as changes in spectral brightness), but focuses more on the similarity of spectral shapes. The formula is as follows:

[0091]

[0092] The Adam optimization algorithm is used to optimize the network parameters. The update formula is: Among them, θ t represents the network parameters, and α is the learning rate.

[0093] Embodiment 2:

[0094] The simulation experiment of the present invention was carried out in the hardware environment of Intel i7-13700, NVIDIA GTX 4060, 16GB memory and the software environment of Python3.8 and PyTorch 1.0. The simulation experiment data of the present invention is the Jasper Ridge dataset. The dataset is collected by the AVIRIS sensor and contains 100×100 pixels. After removing the low-quality bands, the remaining 198 channels are used for the experiment. Figure 5 Shows a cube image of the Jasper Ridge dataset, which contains four main materials: trees, water, soil, and roads.

[0095] Figure 5 and Figure 6 The end member features and corresponding abundance results extracted by the present invention under Urban data are shown respectively, which represent trees, water bodies, soil and roads from left to right and from top to bottom. Figure 5 The red line in the middle represents the true value, while the blue line represents the end member spectral characteristic curve obtained by the present invention. It can be seen that the spectral characteristic curve obtained by the present invention is very close to the true value. Figure 6 In the figure, the comparison between the abundance results of the present invention and other methods is shown respectively.

[0096] The comparison results of the method of the present invention and the existing advanced unmixing methods are shown in Table 1 and Table 2. The convolutional neural network autoencoder unmixing (hereinafter referred to as CNNAEU) method in the comparative test is proposed in the article "Convolutional Autoencoder for Spectral-Spatial Hyperspectral Unmixing"; the EndNet method is proposed in the article "EndNet: Sparse Autoencoder Network for Endmember Extraction and Hyperspectral Unmixing"; the TANet method is proposed in the article "TANet: An Unsupervised Two-Stream Autoencoder Network for Hyperspectral Unmixing"; and the SSAE method is proposed in the article "Spatial-Spectral Autoencoder Networks for Hyperspectral Unmixing".

[0097] Table 1 SAD of Jasper dataset (×10 -2 ) Experimental results of the method

[0098]

[0099]

[0100] Table 2 RMSE of Jasper dataset (×10 -2 ) Experimental results of the method

[0101] method Trees Water soil the way average CNNAEU 13.56 9.66 10.61 8.64 10.62 EndNet 8.85 6.87 10.55 11.20 9.37 TANet 8.05 8.13 8.59 5.45 7.56 SSAE 5.35 5.42 6.30 7.12 6.05 Method of the present invention 6.45 4.06 6.65 5.72 5.81

[0102] The present invention proposes a method for fusing high- and low-resolution features, aiming to improve the restoration performance of hyperspectral images, which has important applications in the field of marine remote sensing. The method effectively processes and optimizes images by introducing multiple modules, enhances the expression ability of low-resolution features, and thus achieves a better unmixing effect. In marine remote sensing, the complexity and diversity of the marine environment, such as the spectral information of components such as seawater, phytoplankton, suspended matter and benthic organisms, is usually difficult to distinguish, so how to improve the unmixing accuracy and accurately extract and distinguish these components has become a key issue. Specifically, the present invention combines BiRNN modules, vertical and horizontal modules, and adaptive weighting and high- and low-resolution feature fusion strategies, which can effectively extract image features from multiple dimensions and fully capture the spatial information of marine targets through multi-directional feature extraction. Through the adaptive weighting module, the weights are dynamically adjusted according to the importance of the features, the effective features are highlighted, the interference of invalid information is reduced, and the unmixing accuracy of the image is further improved. In marine remote sensing images, many pixels contain multiple material components, and traditional unmixing methods often find it difficult to effectively separate these components. The present invention successfully overcomes this problem by optimizing the details in low-resolution images. In terms of ecosystem monitoring applications, this technology can provide support for ecosystem monitoring and monitor marine biodiversity and ecological health by deeply analyzing various components in the marine environment. In marine remote sensing applications, this method can not only accurately restore and unmix the spectral information of multiple components, but also deeply analyze various components in the marine environment to provide accurate monitoring and analysis results. In terms of resource management, this technology can assist in the exploration and assessment of marine resources and provide a scientific basis for the rational development and protection of marine resources. In summary, the application of this technology has significant performance advantages in marine pollution assessment, ecosystem monitoring, and resource management, and can provide strong technical support for the protection and sustainable development of the marine environment.

[0103] The above is a detailed description of the remote sensing image change detection method based on the twin network provided by the present invention, but it is obvious that the specific implementation form of the present invention is not limited thereto. For those skilled in the art, various obvious changes made to it without departing from the scope of the claims of the present invention are within the scope of protection of the present invention.

Claims

1. A deep cycle hyperspectral image processing method for marine sub-pixel target recognition, characterized in that: The steps include: S1: Get the HSI matrix of the hyperspectral image containing L spectral segments and n mixed pixels, expressed as HSI(L,n); S2: Set the window size to r*r, split HSI(L,n), and build a window centered on each pixel; S3: Construct a deep recurrent hyperspectral image processing model, which includes a high-resolution spatial feature module and a low-resolution spatial feature module; the high-resolution spatial feature module includes three BiRNN modules and an adaptive channel modulator; the low-resolution spatial feature module includes three high-low feature fusion modules, a convolution module and a gating mechanism; S4: High-resolution and low-resolution space jointly learn features, learn multi-level high-resolution features, fuse low-resolution features and reconstruct residual images, and finally extract features through the gating mechanism to obtain the abundance result; in the high-resolution spatial feature module, the input image I∈R r×r×L After the BiRNN module, the intermediate features obtained by each BiRNN module are recorded as Where j∈{1,2,3}; the three intermediate features are input into the adaptive channel modulator and fused to obtain the feature In the low-resolution spatial feature module, I and Input the first high and low feature fusion module for modulation to obtain I new , I new The feature fusion after participating in the new I; the feature obtained after low-resolution space learning and After splicing, the predicted abundance is obtained through convolution and gating mechanisms; S5: inputting the abundance result into a decoder to restore it into a vector with dimension L to obtain a predicted end member spectrum; S6: The deep recurrent hyperspectral image processing model is trained and optimized through a loss function to obtain the final end members and abundances.

2. The deep cycle hyperspectral image processing method according to claim 1, characterized in that: The S2 specifically includes: setting the window size to r, for any pixel point x, dividing the window with the pixel point as the center point, that is, setting the length and width of HSI (L, n) to c and k, the entire hyperspectral image is divided into a total of (c-1)×(k-1) windows.

3. The deep cycle hyperspectral image processing method according to claim 1, characterized in that: The S4 includes: (1) Learning in high-resolution space: Given an input window I∈R r×r×L , first through three BiRNN modules, the jth BiRNN module has two submodules divided into vertical and horizontal modules where j∈{1,2,3}, the vertical module divides a window into r two-dimensional vectors along the vertical direction These r vectors are passed through a bidirectional recurrent neural network to generate r Then splice horizontally into Then, the multi-channel segmentation of the BiRNN module and the adaptive weighting module highlight the effective features to generate the final The horizontal module divides a window into r two-dimensional vectors along the horizontal direction These r vectors are passed through a bidirectional recurrent neural network to generate r Then vertically spliced ​​into Then, the multi-channel segmentation and adaptive weighting module highlight the effective features to generate the final Then and Concatenate along the channel dimension to get the output of the j-th BiRNN module where j∈{1,2,3}; Will The multi-level features are modulated by the adaptive channel modulator, and the modulated features I ACM ∈R r×r×3L is sent to the low-resolution space and participates in the high-low feature fusion module operation in the low-resolution space; finally, a 1×1 convolution and gating mechanism receives and the shuffled features learned from the low-resolution space to obtain the predicted abundance; (2) Learning in low-resolution space First, receive the input window I∈R r×r×L Then, I is sent to several high-low feature fusion modules for learning low-resolution features; at the same time, each high-low feature fusion module will also match the modulation feature I in the adaptive channel modulator ACM ∈R r ×r×3L In order to generate more representative features, the output of the high and low feature fusion module is used as the new I∈R r×r×L .

4. The deep cycle hyperspectral image processing method according to claim 3, characterized in that: In the multi-channel segmentation and adaptive weighting module: the size of each input is Where r is the window size, L is the number of original spectral segments; for each input, firstly, the weight coefficient of each channel is calculated by multi-channel segmentation and adaptive weighting mechanism; x is divided into S parts along the channel, and for each part x f , except x f The channel features other than f The complement of and It is passed into the adaptive weighted network, through which the curve parameters are estimated to adjust the pixel value range of the feature; finally, it is spliced ​​together in the channel dimension to obtain 5. The deep cycle hyperspectral image processing method according to claim 3, characterized in that: In adaptive channel modulator: given First, concatenate them along the channel dimension into Then use 1×1 convolution to expand to X∈R r×r×6L , and split the features into two tensors: X1∈R r×r×3L and x2∈R r×r×3L , perform a softmax operation along the channel dimension on X1 to obtain the channel weight coefficient. The specific formula is as follows: Where SoftMax is a SoftMax function and ⊙ is element-wise multiplication.

6. The deep cycle hyperspectral image processing method according to claim 3, characterized in that: In the high and low feature fusion module: Will I ACM ∈R r×r×3L With I∈R r×r×L First, concatenate them together in the channel dimension to form x∈R r×r×4L , through a bidirectional recurrent neural network, and then weighted by a channel attention, and finally through a 1×1 convolution to get the new I∈R r×r×L , the formula is as follows: I new =Conv(CA(BiRNN(Concat(I ACM ,I))))∈R r×r×L Among them, Concat is channel concatenation, BiRNN is a bidirectional recurrent neural network, CA is channel attention, and Conv is a 1×1 convolution.

7. The deep cycle hyperspectral image processing method according to claim 1, characterized in that: In S5, the fused features are decoded Reconstructed into the original spectral dimension LL, the reconstruction process is expressed as: Among them, W decode ∈R L×2c is the weight matrix of the decoder, b decode ∈R L is the bias vector. The final output is is the predicted endmember spectrum after reconstruction.

8. The deep cycle hyperspectral image processing method according to claim 1, characterized in that: In S6, the spectral angle distance SAD is used as the basic loss function, and the abundance vector is selected Regularization, the loss function is expressed as: The Adam optimization algorithm is used to optimize the network parameters, and the update formula is: Among them, θ t represents the network parameters, and α is the learning rate.

Images