Deep recurrent hyperspectral image processing method for marine sub-pixel target recognition

Abstract

The application discloses a deep cycle hyperspectral image processing method for marine subpixel target recognition. The hyperspectral image processing method provided by the application is based on simultaneous feature extraction of high and low resolution feature spaces, combines multi-channel segmentation adaptive weighting and channel modulation spectral information processing technology, utilizes a network combining spatial spectrum to perform parallel unmixing on two types of information, and significantly improves unmixing precision in a complex scene. The application can efficiently unmix and accurately classify complex spectral and spatial information in a hyperspectral image, has strong robustness and adaptability, is particularly suitable for marine environment monitoring, resource exploration and environmental protection and the like, can overcome the deficiency of traditional methods in high-dimensional features and generality in complex landform spectral separation and classification, and has important application value.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of hyperspectral image processing, and particularly relates to a deep cycle hyperspectral image processing method for marine sub-pixel target identification. BACKGROUND

[0002] With the rapid development of satellite remote sensing technology, it is generally recognized that hyperspectral images not only record spatial characteristics in two-dimensional planes, but also contain spectral information with hundreds of bands. These spectral information can reflect different material compositions of the earth's surface and water bodies. Therefore, hyperspectral remote sensing is widely used in environmental monitoring, agriculture, forestry, urban planning and other fields, and has important application value in marine remote sensing. In the field of ocean, seawater, phytoplankton, suspended matter, benthic organisms and pollutants often exist at the same time, and their spectral characteristics will be mixed, resulting in the mixed pixel problem of hyperspectral images. The unmixing of hyperspectral images involves solving two main challenges: (1) determining the component endmember, i.e. the spectral characteristics of these objects, (2) estimating the proportion of these endmembers in each hyperspectral pixel, also known as abundance estimation.

[0003] Linear unmixing model regards the spectrum of each mixed pixel as a linear combination of known endmember spectra. In recent years, the unmixing methods for hyperspectral images can be mainly divided into traditional methods based on spectrum and modern methods combining spatial information. (1) The traditional unmixing methods mainly include geometric, statistical and sparse regression, which usually only rely on spectral information. These methods either analyze the geometric relationship between pixels to extract endmembers for unmixing, or incorporate prior knowledge into unmixing by constructing the probability relationship between observation data and model parameters, which is difficult to effectively use spatial information, resulting in limitations in analyzing mixed pixels. (2) The methods based on deep learning can be divided into two categories: methods based on deep convolutional neural network (CNN) framework and methods based on autoencoder (AE) framework. The method based on CNN extracts features by stacking convolutional layers and activation functions, which usually leads to large number of parameters and overfitting. The method based on autoencoder can effectively capture important information in hyperspectral data, reduce redundancy and improve the accuracy and robustness of unmixing to noise. However, these methods also face challenges, such as high dimensionality of data, deep learning models need to effectively process these high-dimensional data to avoid excessive computational overhead or "dimensional disaster". At the same time, there may be redundant information in too many bands, how to effectively extract useful features and reduce computational burden is an important challenge. SUMMARY

[0004] The purpose of the present application is to provide a deep cycle hyperspectral image processing method for marine sub-pixel target identification to make up for the shortcomings of the prior art.

[0005] This invention effectively applies multi-feature fusion and spatial information in an autoencoder (AE)-based unmixing model. This network can fully utilize high-resolution features to explore more representative features, thereby promoting better image recovery and addressing technical challenges in existing hyperspectral image processing methods, such as complex spectral mixing, insufficient unmixing accuracy, and noise interference. This invention considers factors such as spatial receptive field and multi-scale feature fusion, demonstrating competitive performance in endmember extraction and abundance estimation, and providing an effective solution for hyperspectral unmixing tasks.

[0006] To achieve the above objectives, the specific technical solution adopted by the present invention is as follows:

[0007] A depth-cycle hyperspectral image processing method for sub-pixel target recognition in the ocean includes the following steps:

[0008] S1: Obtain the hyperspectral image (HSI) matrix containing L spectral bands and n mixed pixels, denoted as HSI(L,n);

[0009] S2: Set the window size to r*r, divide HSI(L,n), and construct a window centered on each pixel;

[0010] S3: Construct a deep recurrent hyperspectral image processing model, which includes a high-resolution spatial feature module and a low-resolution spatial feature module; the high-resolution spatial feature module includes three BiRNN modules and an adaptive channel modulator; the low-resolution spatial feature module includes three high-low feature fusion modules, a convolution module, and a gating mechanism;

[0011] S4: High-resolution and low-resolution spatial features are jointly learned. The former learns multi-level high-resolution features, fuses low-resolution features, and reconstructs the residual image, while the latter explores more representative features from the high-resolution features. Finally, a gating mechanism is used to extract features and obtain the abundance result. In the high-resolution spatial feature module, the input image I∈R r×r×L First, the data is processed through three BiRNN modules. The intermediate features obtained from each BiRNN module are denoted as follows: Where j∈{1,2,3}; the three intermediate features are input into an adaptive channel modulator and fused to obtain the features. It is fed into the low-resolution space; in the low-resolution space feature module, there are three high-low feature fusion modules. First, I and The input is modulated in the first high-low feature fusion module to obtain I. new I new The features obtained after low-resolution spatial learning will be used as a new I in the feature fusion process. After the splicing, a 1x1 convolution and a gating mechanism are performed to obtain the predicted abundance;

[0012] S5: The abundance result is input into the decoder to restore the predicted endmember spectrum with a dimension of L;

[0013] S6: The deep recurrent hyperspectral image processing model is trained and optimized by a loss function, thereby obtaining the final endmember and abundance.

[0014] Further, in S1, the hyperspectral data includes but is not limited to NASA ASTER: providing hyperspectral images of the earth's surface, containing 14 spectral bands, covering from visible light to thermal infrared region. NASA Hyperion: Hyperion satellite provides high-resolution hyperspectral data with 220 bands, covering from visible light to short-wave infrared. These satellite data can be downloaded or purchased for free through public remote sensing data portals (such as NASA Earthdata, ESA Copernicus, USGS EarthExplorer, etc.). The obtained data are determined according to different application fields or the contribution size to the unmixing task to determine the spectral band and the number of mixed pixels, denoted as HSI(L, n).

[0015] Further, S2 specifically includes: setting the window size as r, for any pixel point x, dividing the window with the pixel point as the center point, and because the edge pixel points of the hyperspectral image cannot construct a window with them as the center point, they are discarded; that is, assuming that the length and width of HSI(L, n) are c and k, the whole hyperspectral image is divided into (c-1)×(k-1) windows.

[0016] Further, S4 includes:

[0017] (1) Learning in high-resolution space:

[0018] Given an input window I∈R r×r×L , first pass through three BiRNN modules, for the jth BiRNN module, there are two sub-modules divided into vertical and horizontal modules, where j∈{1,2,3}, the vertical module divides a window into r two-dimensional vectors along the vertical direction The r vectors are respectively passed through a bidirectional recurrent neural network to generate r Then spliced into Then the multi-channel segmentation and adaptive weighting module of the BiRNN module highlights the effective features to generate the final The horizontal module divides a window into r two-dimensional vectors along the horizontal direction The r vectors are respectively passed through a bidirectional recurrent neural network to generate r Then spliced into Subsequently, the multi-channel segmentation and adaptive weighting module highlights the effective features to generate the final Then, the and are spliced together along the channel dimension to obtain the output of the jth BiRNN module where j∈{1,2,3}.

[0019] The is modulated by the adaptive channel modulator, and the modulated feature I ACM ∈R r×r×3L is sent to the low-resolution space to participate in the high-low feature fusion module operation in the low-resolution space; finally, a 1×1 convolution and a gating mechanism receive and the disordered features learned from the low-resolution space to obtain the predicted abundance.

[0020] (2) Learning in the low-resolution space

[0021] First, the input window I∈R r×r×L is received, then I is sent to several high-low feature fusion modules for learning low-resolution features; at the same time, each high-low feature fusion module also matches the modulated feature I ACM ∈R r×r×3L in the adaptive channel modulator to generate more representative features, and the output of the high-low feature fusion module is taken as the new I∈R r×r×L , so as to promote better recovery.

[0022] Further, in the multi-channel segmentation and adaptive weighting module:

[0023] The size of each input is where r is the window size, and L is the original spectral band number; for each input, first, the multi-channel segmentation and adaptive weighting mechanism is used to calculate the weighting coefficient of each channel, which helps to adjust the features according to the cross-correlation between channels; this weighting mechanism helps to highlight important features while suppressing irrelevant features; finally, all convolution outputs are recombined in the channel dimension. In this way, the advantages of segmentation and recombination can be combined to highlight effective channel features.

[0024] First, x is divided into S parts along the channel, and for each part x f , the channel features other than x f are connected together as a supplement to x f , denoted as Next, and are passed into an adaptive channel weighting network, through which the curve parameters are estimated for adjusting the pixel value range of the features. Finally, they are concatenated together in the channel dimension to get

[0025] Further, in the adaptive channel modulator, the following operations are performed:

[0026] Given First, they are concatenated along the channel dimension to get Then, a 1x1 convolution is used to expand X ∈ R r×r×6L , and the features are divided into two tensors: X1 ∈ R r×r×3L and X2 ∈ R r×r×3L Next, a softmax operation is performed on X1 along the channel dimension to obtain the channel weighting coefficients, and the specific formula is as follows:

[0027]

[0028] where SoftMax is a SoftMax function, and is element multiplication.

[0029] Further, in the high-low feature fusion module, the following operations are performed:

[0030] I ACM ∈ R r×r×3L is concatenated with I ∈ R r×r×L along the channel dimension to form x ∈ R r×r×4L , which is passed through a bidirectional recurrent neural network, then weighted through a channel attention, and finally passed through a 1x1 convolution to obtain a new I ∈ R r×r×L , and the formula is as follows:

[0031] I new = Conv(CA(BiRNN(Concat(I ACM , I)))) ∈ R r×r×L

[0032] where Concat is channel concatenation, BiRNN is a bidirectional recurrent neural network, CA is channel attention, and Conv is a 1x1 convolution.

[0033] Further, in the S5, the fused features are reconstructed into the original spectral dimension LL through a decoder, and the reconstruction process is represented as:

[0034]

[0035] where W decode ∈ R L×2c is a weight matrix for the decoder, and b decode ∈ R L is a bias vector. The final output is​ For the prediction end-member spectrum after reconstruction.

[0036] Further, in the S6, in order to reduce the difference between the network prediction result and the true value, the spectral angle distance (SAD) is used as a basic loss function, and the sparsity of the abundance vector is considered, and the abundance vector is selected Regularized to achieve better reconstruction effect; this regularization method makes most elements of the abundance vector zero, thereby enhancing sparsity; the loss function is expressed as:

[0037]

[0038] Root mean square error (RMSE) and spectral angle distance (SAD) are used to evaluate the performance of the model on the dataset. Among them, the root mean square error is usually used to compare the abundance estimation of each pixel. The formula is as follows:

[0039]

[0040] Where x c is the actual abundance of the pixel point, is the predicted abundance of the pixel point, and n is the number of pixel points.

[0041] Spectral Angle Distance (Spectral Angle Distance, SAD) is a measurement method for measuring the spectral similarity in hyperspectral images. It evaluates the similarity between two spectral vectors by calculating the angle between them, rather than directly calculating the numerical difference. Therefore, it is not sensitive to changes in spectral features (such as changes in spectral brightness), but pays more attention to the similarity of spectral shape. The formula is as follows:

[0042]

[0043] Where a c is the true spectral feature of the ground object, is the predicted spectral feature of the ground object.

[0044] The Adam optimization algorithm is used to optimize the network parameters, and the update formula is as follows: Where, θ t represents the network parameters, and alpha is the learning rate.

[0045] Compared with the prior art, the advantages and beneficial effects of the present application are as follows:

[0046] The hyperspectral image processing method based on high and low resolution feature space simultaneous feature extraction provided by the present application combines multi-channel segmentation adaptive weighting and channel modulation spectral information processing technology, and uses a spatial-spectral combined network to perform parallel unmixing on the two types of information, thereby significantly improving the unmixing accuracy in complex scenes. Specifically, it includes the following:

[0047] 1. Extract spatial and spectral information in high-resolution space using BiRNN module to improve unmixing accuracy.

[0048] The present application avoids the interference of useless information by splitting the input window in vertical and horizontal directions respectively and performing bidirectional recurrent neural network processing in each direction, and then using a multi-channel segmentation and adaptive weighting module to highlight effective features, so that the model can more comprehensively capture the multi-level features of the image. This way enhances the extraction of features in different directions and improves the expression ability of the model.

[0049] 2. Use high-low feature fusion module to retain detail information in low-resolution space and promote high-quality extraction effect.

[0050] In each high-low resolution feature fusion module, the input low-resolution feature is combined with the modulated feature from the adaptive channel modulator, further improving the multidimensionality of feature representation. Through this multi-dimensional feature enhancement, the model can finely adjust the different levels and detail information of the image, ensuring that important information is not lost.

[0051] 3. End-to-end deep learning framework to improve unmixing efficiency and robustness

[0052] The present application adopts an end-to-end deep learning framework, which integrates spectral feature extraction, spatial feature fusion, unmixing and data reconstruction in the same network by constructing feature conversion from high-resolution space to low-resolution space. This architecture realizes the simultaneous processing of spectral and spatial information through recurrent neural network and convolution operation, and automatically learns the optimal feature representation. By using the Adam optimization algorithm, the model can quickly converge and maintain high unmixing accuracy in complex scenarios. This end-to-end approach not only improves processing efficiency and reduces the complexity of manual parameter tuning, but also enhances the robustness of the model in different data sets and application scenarios.

[0053] The present application can efficiently unmix and accurately classify complex spectral and spatial information in hyperspectral images, with strong robustness and adaptability, especially suitable for marine environment monitoring, resource exploration and environmental protection, etc. In the field of complex topography spectral separation and classification, it can overcome the shortcomings of traditional methods in high-dimensional features and generality, and has important application value. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 is the flowchart of the present application.

[0055] Figure 2 is the flowchart of the present application.

[0056] Figure 3 Input processing schematic diagram for the network model of the present application.

[0057] Figure 4 Schematic diagram of hyperspectral input data for the present application.

[0058] Figure 5 Endmember effect map for the method of Example 2.

[0059] Figure 6 Abundance effect map for the method of Example 2. DETAILED DESCRIPTION

[0060] In the following description, numerous different aspects of the application will be set forth in detail. However, it is clear that not all of these aspects of the application can be implemented only using some or all of the structures and processes of the application. For the sake of illustration, specific numbers, configurations and sequences are mentioned in the description, but it is clear that the application can be implemented without these specific details. In other cases, some well-known features of the application will not be described in detail in order not to obscure the application.

[0061] Example 1:

[0062] A deep recurrent hyperspectral image processing method for marine subpixel target recognition, as shown in Figure 1 , 2 , comprises the following steps:

[0063] Step 1: Obtain hyperspectral data and determine the number of spectral bands and mixed pixels.

[0064] Currently, there are many remote sensing satellites and spacecraft providing hyperspectral data, including but not limited to ASTER of NASA: providing hyperspectral images of the earth's surface, containing 14 spectral bands, covering from visible light to thermal infrared region. Hyperion of NASA: Hyperion satellite provides high-resolution hyperspectral data, with 220 bands, covering from visible light to short-wave infrared. These satellite data can be downloaded or purchased for free through public remote sensing data portals (such as NASA Earthdata, ESA Copernicus, USGS Earth Explorer, etc.). The obtained data are determined according to different application fields or the contribution to the unmixing task to determine the number of spectral bands and mixed pixels, denoted as HSI(L, n).

[0065] Step 2: Set the window size to r*r, segment HSI(L, n), and construct a window centered on each pixel for each pixel.

[0066] A hyperspectral image has tens of thousands of pixels and hundreds or even thousands of bands, and the data volume is too large to be directly input into the model, so it is necessary to preprocess the image data in advance. Set the window size to r, for any pixel point x, divide the window with the pixel point as the center point, and because the edge pixel points of the hyperspectral image cannot construct a window with them as the center point, they are discarded. That is, the length and width of HSI(L, n) are c and k, and the entire hyperspectral image can be divided into (c-1)×(k-1) windows.

[0067] Step 3: High-resolution and low-resolution spaces jointly learn features, the former learns multi-level high-resolution features, fuses low and high-resolution features and reconstructs residual images, and the latter explores more representative features from high-resolution features to promote better recovery, and finally extracts features through a gating mechanism to obtain the abundance result.

[0068] Step 3.1: Learning in high-resolution space

[0069] As shown in Figure 3 , 4 , given an input window I∈R r×r×L , first pass through three BiRNN modules, for the jth BiRNN module, there are two sub-modules divided into vertical and horizontal modules, where j∈{1,2,3}, the vertical module divides a window into r two-dimensional vectors along the vertical direction, and the r vectors are respectively passed through a bidirectional recurrent neural network to generate r , which are spliced into along the horizontal direction, and then the multi-channel segmentation and adaptive weighting module proposed in (3.3) is used to highlight effective features to generate the final The horizontal module divides a window into r two-dimensional vectors along the horizontal direction, and the r vectors are respectively passed through a bidirectional recurrent neural network to generate r , which are spliced into along the vertical direction, and then the multi-channel segmentation and adaptive weighting module proposed in (3.3) is used to highlight effective features to generate the final , where j∈{1,2,3}.

[0070] , modulate the multi-level features through an adaptive channel modulator, and the modulated features I ACM ∈R r×r×3Lis sent into the low resolution space to participate in the operation of the high-low feature fusion module proposed in the low resolution space (3.5). Finally, a 1x1 convolution and gating mechanism receives and the scrambled features learned from the low resolution space to get the predicted abundance.

[0071] Step 3.2: Learning in the low resolution space

[0072] Firstly, the input window I ∈ R r×r×L is received. ACM Then, I is sent into several high-low feature fusion modules proposed in (3.5) to learn low resolution features. Meanwhile, each high-low feature fusion module also matches the modulation features I r×r×3L ∈ R r×r×L to generate more representative features, and the output of the high-low feature fusion module is taken as the new I ∈ R f to facilitate better recovery.

[0073] Step 3.3: Multi-channel segmentation and adaptive weighting module

[0074] The size of each input is where r is the window size and L is the original number of spectral bands. For each input, firstly, the multi-channel segmentation and adaptive weighting mechanism is used to calculate the weighting coefficient of each channel, which helps to adjust the features according to the cross-correlation between channels. This weighting mechanism helps to highlight important features while suppressing irrelevant features. Finally, all the convolution outputs are recombined in the channel dimension. In this way, the advantages of segmentation and recombination can be combined to highlight effective channel features.

[0075] Firstly, x is divided into S parts along the channel, and for each part x f , the channel features except x f are concatenated together as a supplement to x f , denoted as Next, and are passed into the adaptive weighting network to estimate the curve parameters for adjusting the pixel value range of the features. Finally, they are concatenated together in the channel dimension to get

[0076] Step 3.4: Adaptive channel modulator

[0077] Given , they are first concatenated into Then, a 1x1 convolution is used to expand it to X ∈ R r×r×6L , and the features are divided into two tensors: X1 ∈ R r×r×3L and X2 ∈ R r×r×3L, then a softmax operation is performed on X1 along the channel dimension to obtain the channel weighting coefficients, and the specific formula is as follows:

[0078]

[0079] Step 3.5: High-low feature fusion module

[0080] I ACM ∈R r×r×3L and I∈R r×r×L First, they are spliced together in the channel dimension to form x∈R r×r×4L , and then a bidirectional recurrent neural network is used to weight them through a channel attention, and finally a 1x1 convolution is used to obtain a new I∈R r×r×L , and the formula is as follows:

[0081]

[0082] Step 4: The abundance result is input into the decoder to restore it to a vector with dimension L to obtain the predicted endmember spectrum.

[0083] The fused features are reconstructed into the original spectral dimension LL through the decoder, and the reconstruction process can be represented as:

[0084] where W decode ∈R L×2c is the weight matrix for the decoder, and b decode ∈R L is the bias vector. The final output is the reconstructed predicted endmember spectrum.

[0085] Step 5: Train and optimize the model through the loss function to obtain the endmember with the smallest error and abundance.

[0086] In order to reduce the difference between the network prediction result and the true value, the spectral angle distance (SAD) is used as the basic loss function, and considering the sparsity of the abundance vector, the abundance vector is regularized to achieve better reconstruction effect. This regularization method makes most elements of the abundance vector zero, thereby enhancing the sparsity. The loss function can be expressed as:

[0087]

[0088] The root mean square error (RMSE) and the spectral angle distance (SAD) are used to evaluate the performance of the model on the dataset. The root mean square error is usually used to compare the difference between the abundance estimate of each pixel or the spectral reconstruction value of each pixel and the true value. The formula is as follows: ​

[0089]

[0090] Spectral Angle Distance (SAD) is a measure method for measuring the spectral similarity in hyperspectral images. It evaluates the similarity between two spectral vectors by calculating the angle between them, rather than directly calculating the numerical difference. Therefore, it is not sensitive to the change of spectral features (such as spectral brightness change), but pays more attention to the similarity of spectral shape. The formula is as follows:

[0091]

[0092] The Adam optimization algorithm is used to optimize the network parameters, and the update formula is as follows: where, θ t represents the network parameters, and α is the learning rate.

[0093] Example 2:

[0094] The simulation experiment of the present application is carried out in the hardware environment of Intel i7-13700, NVIDIA GTX 4060 and memory 16GB, and the software environment of Python 3.8 and PyTorch 1.0. The simulation experiment data of the present application is the Jasper Ridge dataset. The dataset is collected by AVIRIS sensor, and contains 100x100 pixels. After removing low-quality bands, the remaining 198 channels are used for experiments. Figure 5 The cube image of the Jasper Ridge dataset is shown, which contains four main materials: trees, water, soil and roads.

[0095] Figure 5 and Figure 6 respectively show the endmember feature and the corresponding abundance result extracted by the present application under the Urban data. From left to right and from top to bottom, they represent trees, water, soil and roads respectively. Figure 5 The middle line represents the true value, and the blue line represents the endmember spectral feature curve obtained by the present application. It can be seen that the spectral feature curve obtained by the present application is very close to the true value. Figure 6 In the above, the abundance results of the present application and other methods are compared.

[0096] The comparison results of the method of the present application and the existing advanced unmixing methods are shown in Tables 1 and 2. The convolutional neural network autoencoder unmixing (hereinafter abbreviated as CNNAEU) method in the comparison test is proposed in the article "Convolutional Autoencoder for Spectral-Spatial Hyperspectral Unmixing"; the EndNet method is proposed in the article "EndNet: Sparse AutoEncoder Network for Endmember Extraction and Hyperspectral Unmixing"; the TANet method is proposed in the article "TANet: An Unsupervised Two-Stream Autoencoder Network for Hyperspectral Unmixing"; and the SSAE method is proposed in the article "Spatial-Spectral Autoencoder Networks for Hyperspectral Unmixing".

[0097] Table 1 SAD (x 10-4) of Jasper dataset -2 Method experimental results

[0098]

[0099]

[0100] Table 2 RMSE (x 10-4) of Jasper dataset -2 Method experimental results

[0101] Method Trees Water body Soil Road Average CNNAEU 13.56 9.66 10.61 8.64 10.62 EndNet 8.85 6.87 10.55 11.20 9.37 TANet 8.05 8.13 8.59 5.45 7.56 SSAE 5.35 5.42 6.30 7.12 6.05 The method of the invention 6.45 4.06 6.65 5.72 5.81

[0102] The present application proposes a method of high and low resolution feature fusion, aiming to improve the restoration performance of hyperspectral images, especially having important applications in the field of marine remote sensing. The method effectively processes and optimizes the image through the introduction of multiple modules, enhances the expression ability of low resolution features, and thus realizes better unmixing effect. In marine remote sensing, the complexity and diversity of marine environment, such as the spectral information of seawater, phytoplankton, suspended matter and benthic organisms, are usually difficult to distinguish, so how to improve the unmixing precision and accurately extract and distinguish these components becomes a key problem. Specifically, the present application combines BiRNN module, vertical and horizontal module, and adaptive weighting and high and low resolution feature fusion strategy, which can effectively extract image features from multiple dimensions and capture spatial information of marine targets through multi-directional feature extraction. Through the adaptive weighting module, the weight is dynamically adjusted according to the importance of the feature, the effective feature is highlighted, and the interference of invalid information is reduced, further improving the unmixing precision of the image. In marine remote sensing images, many pixels contain multiple material components, and traditional unmixing methods often cannot effectively separate these components, while the present application successfully overcomes this problem by optimizing the details in the low resolution image. In the aspect of ecosystem monitoring application, by deeply analyzing various components in the marine environment, the present application can provide support for ecosystem monitoring, monitoring of marine biodiversity and ecological health status. In marine remote sensing application, the method can not only accurately restore and unmix the spectral information of multiple components, but also deeply analyze various components in the marine environment, providing accurate monitoring and analysis results. In the aspect of resource management, the present application can assist in the exploration and evaluation of marine resources, and provide scientific basis for the rational development and protection of marine resources. In summary, the application of the present application has significant performance advantages in marine pollution assessment, ecosystem monitoring and resource management, and can provide strong technical support for the protection and sustainable development of marine environment.

[0103] The above describes in detail the remote sensing image change detection method based on twin network provided by the present application, but obviously the specific implementation form of the present application is not limited to this. For those skilled in the art, various obvious changes made to the present application without departing from the scope of the claims of the present application are within the scope of protection of the present application.

Claims

1. A deep recurrent hyperspectral image processing method for marine sub-pixel target recognition, characterized in that, The method comprises the following steps: S1: obtaining a hyperspectral image HSI matrix containing L spectral bands and n mixed pixels, denoted as HSI(L, n); S2: setting the window size as r*r, segmenting HSI(L, n), and constructing a window centered on each pixel; S3: constructing a deep recurrent hyperspectral image processing model, which comprises a high-resolution spatial feature module and a low-resolution spatial feature module; the high-resolution spatial feature module comprises three BiRNN modules and an adaptive channel modulator; the low-resolution spatial feature module comprises three high-low feature fusion modules, a convolution module and a gating mechanism; S4: High-resolution and low-resolution spaces learn features together, learn multi-level high-resolution features, fuse low and high-resolution features and reconstruct residual images, and finally extract features through a gating mechanism to obtain abundance results; in the high-resolution space feature module, the input image I ∈ R r×r×L After the BiRNN module, the intermediate feature obtained by each BiRNN module is denoted as Where j ∈ {1, 2, 3}; the three intermediate features are input into the adaptive channel modulator to obtain the feature into the low-resolution space; in the low-resolution space feature module, first I and are input into the first high-low feature fusion module to obtain I new , I new serves as a new I to participate in subsequent feature fusion; after learning in the low-resolution space, the obtained feature is concatenated with , and then a convolution and a gating mechanism are performed to obtain the predicted abundance; S5: inputting the abundance result into a decoder to restore a vector with a dimension of L to obtain a predicted endmember spectrum; S6: training and optimizing the deep recurrent hyperspectral image processing model through a loss function, thereby obtaining a final endmember and abundance.

2. The deep recurrent hyperspectral image processing method of claim 1, wherein, S2 specifically comprises: setting the window size as r, and for any pixel point x, starting to divide the window with the pixel point as the center point, that is, setting the length and width of HSI(L, n) as c and k, and dividing the whole hyperspectral image into (c-1)×(k-1) windows.

3. The deep recurrent hyperspectral image processing method of claim 1, wherein, S4 comprises: (1) learning in a high-resolution space Given an input window I∈R r×r×L First, it goes through three BiRNN modules. The j-th BiRNN module has two sub-modules: a vertical module and a horizontal module, where j∈{1,2,3}. The vertical module divides a window into r two-dimensional vectors along the vertical direction. These r vectors are processed by a bidirectional recurrent neural network to generate r vectors. Then splice them together horizontally. Subsequently, the BiRNN module performs multi-channel segmentation, and the adaptive weighting module highlights effective features to generate the final product. The horizontal module divides a window into r two-dimensional vectors along the horizontal direction. These r vectors are processed by a bidirectional recurrent neural network to generate r vectors. Then spliced ​​together vertically Subsequently, after multi-channel segmentation and an adaptive weighting module to highlight effective features, the final product is generated. Then and The output of the j-th BiRNN module is obtained by concatenating the data along the channel dimension. Where j∈{1,2,3}; Will The multi-level features are modulated by an adaptive channel modulator, and the modulated features I ACM ∈R r×r×3L are sent into the low-resolution space to participate in the high-low feature fusion module operation in the low-resolution space; finally, a 1x1 convolution and a gating mechanism receive and the disordered features learned from the low-resolution space to obtain the predicted abundance; (2) learning in a low-resolution space First, the input window I ∈ R is received r×r×L Then, I is sent to several high-low feature fusion modules for learning low-resolution features; at the same time, each high-low feature fusion module also matches the modulation features I in the adaptive channel modulator ACM ∈R r ×r×3L To generate more representative features, the output of the high-low feature fusion module is taken as a new I ∈ R r×r×L .

4. The deep recurrent hyperspectral image processing method of claim 3, wherein, The size of each input in the multi-channel segmentation and adaptive weighting module is where r is the window size, and L is the original spectral band number. For each input, first the weighted coefficients for each channel are computed by a multi-channel segmentation and adaptive weighting mechanism; x is divided into S parts along the channel, for each part x f , the channel features except x f are concatenated together as a complement to x f , denoted as x f and are passed into an adaptive weighting network, through which the curve parameters are estimated for adjusting the pixel value range of the features; finally concatenated together in the channel dimension to get 5. The deep recurrent hyperspectral image processing method of claim 3, wherein, In adaptive channel modulator: given First, they are concatenated along the channel dimension into Then, expand using 1x1 convolution to X e R r×r×6L and split the feature into two tensors: X1 e R r×r×3L and X2 e R r×r×3L On X1, softmax operation is performed along the channel dimension to get channel weighting coefficients, the specific formula is as follows: wherein SoftMax is a SoftMax function, and is element multiplication.

6. The deep recurrent hyperspectral image processing method of claim 3, wherein, In the high-low feature fusion module: Will I ACM ∈R r×r×3L With I∈R r×r×L First, concatenate them along the channel dimension to form x∈R r×r×4L The new I∈R is obtained by passing it through a bidirectional recurrent neural network, then through channel attention for weighting, and finally through a 1×1 convolution. r×r×L The formula is as follows: I new = Conv(CA(BiRNN(Concat(I ACM , I))) e R r×r×L wherein Concat is channel concatenation, BiRNN is a bidirectional recurrent neural network, CA is channel attention, and Conv is a 1×1 convolution.

7. The deep recurrent hyperspectral image processing method of claim 1, wherein, In step S5, the fused features are decoded. The original spectral dimensions are reconstructed, and the reconstruction process is represented as follows: where W decode ∈R L×2c is a weight matrix for the decoder, b decode ∈R L is a bias vector; the final output is the reconstructed endmember spectra.

8. The deep recurrent hyperspectral image processing method of claim 1, wherein, In the S6, the spectral angle distance SAD is used as a basic loss function, and the abundance vector is selected by Regularization, the loss function is expressed as: And use Adam optimization algorithm to optimize network parameters, update formula is: Where, θ t Indicates the network parameters, and α is the learning rate.

Images