Marine oil spill hyperspectral unmixing method and system based on multi-scale residual attention

Abstract

The present application discloses a hyperspectral unmixing method and system based on multi-scale residual attention, which belongs to the field of marine monitoring technology. The present application trains the model by pre-acquiring a hyperspectral unmixing data set for marine oil spills, optimizes the parameters of the hyperspectral image unmixing model for marine oil spills, and improves the dynamic monitoring of marine oil spills. When constructing a hyperspectral image unmixing model for marine oil spills, the present application effectively improves the spatial similarity of the hyperspectral image by introducing a spatial attention module, improves the global information capture capability, and effectively improves the efficiency and accuracy of the hyperspectral unmixing of marine oil spills by introducing a spectral attention module to focus on the similarity between highly correlated spectral bands.

Description

Technical Field

[0001] The present application belongs to the field of ocean monitoring technology, and in particular, relates to a method and system for unmixing ocean oil spill hyperspectral data based on multi-scale residual attention. Background Art

[0002] Marine oil spills can cause damage to the marine environment, and it is of great significance to achieve accurate dynamic monitoring of marine oil spill pollution. In the prior art, hyperspectral image data is often used for image segmentation to dynamically monitor marine oil spills, but it is limited by the ground resolution of satellite remote sensing sensors, and in the early stage of marine oil spills, due to the action of wind and waves and ocean breaking waves, the oil spills often appear as strip-shaped broken stripes, and the acquired hyperspectral data are mostly pure oil pixels, pure water pixels or mixed pixels of oil and water, which leads to large deviations in the traditional image segmentation method to estimate the area of ​​oil spills, and it is impossible to monitor marine oil spill pollution in a timely and accurate manner. Summary of the invention

[0003] In order to solve the above problems existing in the prior art of marine oil spill monitoring, this application proposes a hyperspectral unmixing method and system based on multi-scale residual attention. Specifically, it includes the following contents:

[0004] A method for unmixing marine oil spill hyperspectral data based on multi-scale residual attention, characterized in that the method comprises:

[0005] Step S1: Obtain a marine oil spill hyperspectral image F and marine oil spill endmember information E, and construct a marine oil spill hyperspectral image unmixing dataset based on the marine oil spill hyperspectral image F and the marine oil spill endmember information E, wherein the dataset includes a training dataset and a validation dataset;

[0006] Step S2: constructing a marine oil spill hyperspectral image unmixing model based on a multi-scale residual attention mechanism, wherein the unmixing model includes a multi-scale dilated convolution module, a spatial spectrum residual module, and a reconstruction module;

[0007] Step S3: training the marine oil spill hyperspectral image unmixing model according to the training data set of the marine oil spill hyperspectral image unmixing data set, and optimizing the model parameters using the total loss function L to obtain the trained marine oil spill hyperspectral image unmixing model;

[0008] Step S4: input the hyperspectral image to be unmixed into the trained marine oil spill hyperspectral unmixing model, and output the unmixed endmembers and abundance.

[0009] In the step S1, constructing the unmixed data set of the marine oil spill hyperspectral image based on the marine oil spill hyperspectral image F and the marine oil spill end-member information E comprises:

[0010] Sa1) normalizing the acquired hyperspectral image F, converting the dimension from three-dimensional to two-dimensional, and obtaining the pre-processed data Fh;

[0011] Sa2) Convert the acquired marine oil spill end member information E from ascll format to mat format to obtain the end member information Egt in mat format; calculate the abundance matrix Agt and the abundance vector a according to the end member information using the non-negative least squares algorithm nj The jth endmember of the nth pixel represents the proportion of the crop area on the ground surface, satisfying the following conditions:

[0012]

[0013] Where R represents the total number of end members, and the R abundance vectors represent the abundance matrix Agt.

[0014] Sa3) The preprocessed marine oil spill hyperspectral image Fh, endmember information Egt and abundance matrix Agt are used together to construct a hyperspectral marine oil spill unmixing dataset, in which the marine oil spill hyperspectral image Fh is the training set, and the endmember information Egt and abundance matrix Agt are the validation set.

[0015] In step S3, training the ocean oil spill hyperspectral image unmixing model according to the ocean oil spill hyperspectral image unmixing dataset includes:

[0016] Step S31: input the preprocessed marine oil spill hyperspectral image Fh into the multi-scale void convolution module to obtain the multi-scale feature FA;

[0017] Step S32: input the multi-scale feature FA into the spatial-spectral residual attention module to obtain the intrinsic similarity feature FB of the spatial dimension and the spectral dimension;

[0018] Step S33: input the similarity feature FB into the reconstruction module to obtain the unmixed abundance graph Ae and the unmixed end member Ee.

[0019] Step 31 further includes:

[0020] The preprocessed marine oil spill hyperspectral image Fh is input into four dilated convolutions in parallel for branch learning. The marine oil spill hyperspectral image is sampled with 2D convolution kernels with dilation rates of 1, 2, 3, and 4 respectively, and each convolution step size is the same. Then the four feature blocks are concat-fused to obtain the multi-scale feature FA.

[0021] In step S32, the multi-scale feature FA is input into the spatial-spectral residual attention module to obtain the intrinsic similarity features FB in the spatial dimension and the spectral dimension, including:

[0022] S321) The pre-processed hyperspectral ocean oil spill image Fh is divided into multiple patches, each patch is mapped into three feature maps F1, F2, and F3 through a convolutional network, where the spatial feature F of the i-th pixel of the feature map F1 is 1,i , and the spatial feature F of the j-th pixel of the feature map F2 2,j The position correlation value between is defined as F ji , the calculation formula is as follows:

[0023]

[0024] Where M represents the number of pixels in the patch, F ji Represents the position correlation value between the i-th pixel of feature map F1 and the j-th pixel of feature map F2, which makes the correlation between these similar spatial features stronger;

[0025] S322) By using the position correlation value F between the spatial features ji And the mapped F3 is used to calculate the spatial attention map S j , the formula is as follows:

[0026]

[0027] Where α represents the weight parameter value, F 3,i Represents the value of the i-th pixel in feature map F3.

[0028] S323) will get the spatial attention map S j With F h,j Perform residual operation and finally obtain the attention feature map E corresponding to pixel j j , the formula is as follows:

[0029] E j =S j +F h,j

[0030] Among them, F h,j Represents the jth pixel in the hyperspectral image.

[0031] S324) The attention feature map E after the spatial self-attention structure j The image is divided into multiple patches, each patch is flattened in the channel dimension and mapped into three feature maps A1, A2, and A3. The spectral correlation value between the eigenvalue A1i of the i-th band channel of the A1 matrix and the eigenvalue A2j of the j-th band channel of the A2 matrix is ​​defined as A j,i , the calculation formula is as follows:

[0032]

[0033] Where B represents the number of bands, A j,i Represents the spectral correlation value between the i-th band of feature map A1 and the j-th band of feature map A2. Similarly, this makes the correlation between these similar channels stronger;

[0034] S325) by the correlation value A between the channel characteristics of different bands j,i And the mapped feature map A3 is used to calculate the spectral attention map W j , the formula is as follows:

[0035]

[0036] Where α represents the weight parameter value, A 3,i Represents the value corresponding to the i-th band channel of feature map A3.

[0037] S326) The obtained spectral attention map W j With F h,j Perform residual operation and finally obtain the attention feature M corresponding to channel j j Figure, the formula is as follows:

[0038] F A =W j +F h,j

[0039] where F h,j Represents the jth pixel in the hyperspectral image.

[0040] In step S33, the similarity feature FB is input into the reconstruction module to obtain the unmixed abundance graph Ae and the unmixed end member Ee, including:

[0041] The similarity feature FB is input into the reconstruction module, and the features output by the spatial-spectral residual module are integrated through the multi-channel fully connected operation of the perceptron. Then, the corresponding abundance map Ae and endmember Ee are obtained through the fully connected layer and two convolutional layers. The abundance matrix Agt and endmember information Egt in the hyperspectral marine oil spill unmixing dataset are used to verify the abundance map Ae and endmember Ee output by the model.

[0042] In step S3, optimizing the model parameters using the validation data set specifically includes:

[0043] Establish the total loss function L of the model, optimize the connection weights through the selected optimizer and corresponding parameters, and obtain the optimal parameters of the model when the total loss function is minimized; the total loss function L of the model includes the reconstruction error loss function M of the abundance map calculated by RMSE β And calculate the error loss S of the spectral angle φ , the calculation formula of the total loss function M of the model is: Where σ and γ represent the weight parameter values.

[0044] The present application also protects a marine oil spill hyperspectral unmixing system based on multi-scale residual attention, the system comprising;

[0045] A data set construction module is used to construct a hyperspectral marine oil spill unmixing data set based on the marine oil spill hyperspectral image F and the marine oil spill end-member information E;

[0046] The multi-scale void convolution module is used to process the marine oil spill hyperspectral image Fh through different void ratios to obtain the multi-scale features FA of the marine oil spill hyperspectral image;

[0047] The spatial-spectral residual module includes three spatial-spectral self-attention blocks and a convolutional layer, which are used to set multiple spatial-spectral self-attention blocks based on the residual structure for the input multi-scale feature FA image. The spatial-spectral self-attention block inputs continuous non-overlapping small blocks, and each small block is mapped to a different feature map through the convolutional layer. The spatial self-attention structure is used to calculate the position correlation values ​​between these feature maps to construct a spatial attention map; the channel dimension is flattened through a fully connected layer and mapped to three different matrices through a convolutional network. The spectral correlation values ​​between the matrices are calculated using the spectral self-attention structure to construct a channel attention map to capture the correlation of spectral information.

[0048] The reconstruction module, including an MLP-Mixer layer, a fully connected layer and two convolutional layers, is used to output the abundance map after the first convolutional layer and extract the weight coefficients of the network as end-member information after the second convolutional layer.

[0049] In addition, the present application also protects a computer device, which includes a memory, a processor, and program instructions stored in the memory and executable by the processor, wherein the processor executes the program instructions to perform the steps in the aforementioned method.

[0050] Furthermore, the present application also protects a computer-readable storage medium storing a computer program, characterized in that the computer program implements the aforementioned method when executed by a processor.

[0051] The present application constructs an unmixing model for hyperspectral images of marine oil spills based on a multi-scale residual attention mechanism, and trains the model by pre-acquiring a hyperspectral unmixing dataset for marine oil spills, optimizes the parameters of the hyperspectral unmixing model for marine oil spills, and improves the dynamic monitoring of marine oil spills. When constructing an unmixing model for hyperspectral images of marine oil spills, the present application effectively improves the spatial similarity of hyperspectral images and the ability to capture global information by introducing a spatial attention module, and effectively improves the efficiency and accuracy of hyperspectral unmixing for marine oil spills by introducing a spectral attention module to focus on the similarity between highly correlated spectral bands. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a flow chart of the marine oil spill hyperspectral unmixing method based on the multi-scale residual attention mechanism in this application.

[0053] Figure 2 A flowchart of the steps for constructing the marine oil spill unmixing dataset in this application.

[0054] Figure 3 This is the overall framework diagram of the marine oil spill hyperspectral unmixing method based on the multi-scale residual attention mechanism proposed in this application.

[0055] Figure 4 This is a structural block diagram of the multi-scale void convolution module used in the marine oil spill unmixing model in this application. Specific embodiments

[0056] The following will be combined with the drawings in the embodiments of the present application to further clearly and completely describe the technical solutions in the embodiments of the present application. It should be noted that the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application.

[0057] In order to make the invention purpose, technical scheme and advantages of this application clearer, the technical scheme of the present invention is described clearly and completely in conjunction with the accompanying drawings and specific implementation schemes. It should be pointed out that for ordinary technicians in this field, several changes and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of this application.

[0058] The embodiments of the present application are further described in detail below in conjunction with the accompanying drawings:

[0059] This embodiment provides a method for unmixing marine oil spill hyperspectral based on multi-scale residual attention, including:

[0060] Step S1: Obtain a marine oil spill hyperspectral image F and marine oil spill endmember information E, and construct a marine oil spill hyperspectral image unmixing dataset based on the marine oil spill hyperspectral image F and the marine oil spill endmember information E, wherein the dataset includes a training dataset and a validation dataset;

[0061] Step S2: constructing a marine oil spill hyperspectral image unmixing model based on a multi-scale residual attention mechanism, wherein the unmixing model includes a multi-scale dilated convolution module, a spatial spectrum residual module, and a reconstruction module;

[0062] Step S3: training the marine oil spill hyperspectral image unmixing model according to the training data set of the marine oil spill hyperspectral image unmixing data set, and optimizing the model parameters using the total loss function L to obtain the trained marine oil spill hyperspectral image unmixing model;

[0063] Step S4: input the hyperspectral image to be unmixed into the trained marine oil spill hyperspectral unmixing model, and output the unmixed endmembers and abundance.

[0064] In the process of constructing the unmixed data set of the marine oil spill hyperspectral image, it is necessary to preprocess the obtained unmixed data set of the hyperspectral image to form a suitable training data set and a validation set. In the unmixed data set of the marine oil spill hyperspectral image constructed based on the marine oil spill hyperspectral image F and the marine oil spill endmember information E in step S1, the marine oil spill hyperspectral image is the training data set, and the endmember information and abundance information are the validation data set.

[0065] In this embodiment, specifically, in step S1, constructing a marine oil spill hyperspectral image unmixing dataset based on the marine oil spill hyperspectral image F and the marine oil spill endmember information E includes:

[0066] Sa1) normalizing the acquired hyperspectral image F, converting the dimension from three-dimensional to two-dimensional, and obtaining the pre-processed data Fh;

[0067] Sa2) Convert the acquired marine oil spill end member information E from ascll format to mat format to obtain the end member information Egt in mat format; calculate the abundance matrix Agt and the abundance vector a according to the end member information Egt using the non-negative least squares algorithm nj The jth endmember of the nth pixel represents the proportion of the crop area on the ground surface, satisfying the following conditions:

[0068]

[0069] Where R represents the total number of end members, and the R abundance vectors represent the abundance matrix Agt.

[0070] Sa3) The preprocessed marine oil spill hyperspectral image Fh, endmember information Egt and abundance matrix Agt are used together to construct a hyperspectral marine oil spill unmixing dataset, in which the marine oil spill hyperspectral image Fh is the training set, and the endmember information Egt and abundance matrix Agt are the validation set.

[0071] Furthermore, if Figure 2 As shown, in this embodiment, obtaining end member information and abundance information in step Sa2 further includes:

[0072] Sa31, collects low-altitude ocean oil spill images and sea surface oil spill end-member information;

[0073] Sa32, determines the number of end metadata and categories of oil spills;

[0074] Sa33, to make end-member labels of oil spill clean pixels;

[0075] Sa34, using the end-member labels to calculate abundance labels using non-negative least squares method;

[0076] Sa35, obtain the reference end-metatag and reference abundance tag.

[0077] In the embodiment of the present application, the hyperspectral image F is obtained by a drone hyperspectral imager. Compared with the hyperspectral images obtained by aviation or aerospace means, the hyperspectral images obtained by drones have the characteristics of high resolution and good real-time performance. The marine oil spill end member information E is collected by a portable ground feature spectrometer (Analytical Spectral Devices, ASD). By using the ASD spectrometer to collect end member information, and using the non-negative least squares algorithm to calculate the abundance matrix based on the end member information, the end member information and the abundance matrix can optimize the parameters of the model, which solves the problem that the existing technical center hyperspectral data set lacks end member information and abundance labels for verification, and effectively improves the accuracy of the model.

[0078] Furthermore, in this embodiment, an unmixing model of marine oil spill images is constructed based on a multi-scale residual attention mechanism. The spectral unmixing model using spatial spectral residual attention can make full use of the high spectral resolution characteristics to estimate the true degree of oil-water mixing, thereby accurately calculating the actual oil spill area. The hyperspectral image unmixing model in the embodiment of the present application includes a multi-scale atrous convolution module, a spatial spectral residual module, and a reconstruction module.

[0079] The multi-scale atrous convolution module is used to obtain the multi-scale features FA of the marine oil spill hyperspectral image. The multi-scale atrous convolution module inputs the pre-processed marine oil spill hyperspectral image Fh into multiple atrous convolutions in parallel, and fully integrates the contextual information of different scales according to the different atrous convolutions, thereby improving the sensitivity of the model to the spectral features of the ground objects. In this embodiment, the pre-processed marine oil spill hyperspectral image Fh is input into four atrous convolutions in parallel for branch learning, and the marine oil spill hyperspectral image is sampled with 2D convolution kernels with atrous rates of 1, 2, 3, and 4 in turn, and each convolution step size is the same, and then the four feature blocks obtained are concat spliced ​​and fused to obtain the multi-scale feature FA.

[0080] The spatial-spectral residual module (also called the spatial-spectral residual attention module) can fully exploit the intrinsic similarities of the spatial and spectral dimensions, and obtain the intrinsic similarity features FB of the hyperspectral image in the spatial and spectral dimensions. In this embodiment, the spatial-spectral residual module includes three spatial-spectral self-attention blocks and a convolution layer. Among them, each spatial-spectral self-attention block includes a spatial self-attention structure and a spectral self-attention structure, and the spatial self-attention structure and the spectral self-attention structure are both subjected to convolution operations to achieve image segmentation. In addition, both the spatial-spectral self-attention block and the spatial-spectral residual module include a residual connection. The multi-scale feature FA processed by the multi-scale hole convolution module is input into the spatial-spectral residual module, processed by the three spatial-spectral self-attention blocks in sequence, and then processed by the convolution layer to output the intrinsic similarity features FB of the spatial and spectral dimensions.

[0081] In this embodiment, the process of obtaining the intrinsic similarity feature FB by the empty spectrum residual module includes:

[0082] S321) The pre-processed hyperspectral ocean oil spill image Fh is divided into multiple patches, each patch is mapped into three feature maps F1, F2, and F3 through a convolutional network, where the spatial feature F of the i-th pixel of the feature map F1 is 1,i , and the spatial feature F of the j-th pixel of the feature map F2 2,j The position correlation value between is defined as F ji , the calculation formula is as follows:

[0083]

[0084] Where M represents the number of pixels in the patch, F ji Represents the position correlation value between the i-th pixel of feature map F1 and the j-th pixel of feature map F2, which makes the correlation between these similar spatial features stronger;

[0085] S322) By using the position correlation value F between the spatial features ji And the mapped F3 is used to calculate the spatial attention map S j , the formula is as follows:

[0086]

[0087] Where α represents the weight parameter value, F 3,i Represents the value of the i-th pixel in feature map F3.

[0088] S323) will get the spatial attention map S j With F h,j Perform residual operation and finally obtain the attention feature map E corresponding to pixel j j , the formula is as follows:

[0089] E j =S j +F h,j

[0090] where F h,j Represents the jth pixel in the hyperspectral image.

[0091] S324) The attention feature map E after the spatial self-attention structure j The image is divided into multiple patches, each patch is flattened in the channel dimension and mapped into three feature maps A1, A2, and A3. The spectral correlation value between the eigenvalue A1i of the i-th band channel of the A1 matrix and the eigenvalue A2j of the j-th band channel of the A2 matrix is ​​defined as A j,i , the calculation formula is as follows:

[0092]

[0093] Where B represents the number of bands, A j,i Represents the spectral correlation value between the i-th band of feature map A1 and the j-th band of feature map A2. Similarly, this makes the correlation between these similar channels stronger;

[0094] S325) by the correlation value A between the channel characteristics of different bands j,i And the mapped feature map A3 is used to calculate the spectral attention map W j , the formula is as follows:

[0095]

[0096] Where α represents the weight parameter value, A 3,i Represents the value corresponding to the i-th band channel of feature map A3.

[0097] S326) The obtained spectral attention map W j With F h,j Perform residual operation and finally obtain the attention feature M corresponding to channel j j Figure, the formula is as follows:

[0098] F A =W j +F h,j

[0099] where F h,j Represents the jth pixel in the hyperspectral image.

[0100] The reconstruction module unmixes the hyperspectral image features according to the intrinsic similarity features FB in the spatial and spectral dimensions to obtain the unmixed abundance map Ae and the unmixed end member Ee. The reconstruction module includes an MLP-Mixer layer (multi-perceptron), a fully connected layer and two convolutional layers. After the first convolutional layer, the corresponding abundance map Ae is obtained, which is verified using the labeled abundance map label Agt; after the second convolutional layer, the weight coefficient of the network is extracted to obtain the corresponding end member Ee, and the weight coefficient is verified using the labeled end member label Egt.

[0101] In this embodiment, in step S33, the similarity feature FB is input into the reconstruction module to obtain the unmixed abundance map Ae and the unmixed end member Ee, which include:

[0102] The similarity feature FB is input into the reconstruction module, and the features output by the spatial-spectral residual module are integrated through the multi-channel fully connected operation of the perceptron. Then, the corresponding abundance map Ae and endmember Ee are obtained through the fully connected layer and two convolutional layers. The abundance matrix Agt and endmember information Egt in the hyperspectral marine oil spill unmixing dataset are used to verify the abundance map Ae and endmember Ee output by the model.

[0103] After establishing a marine oil spill hyperspectral image unmixing model based on a multi-scale spatial-spectral residual attention mechanism, the hyperspectral image unmixing model is trained using a pre-acquired hyperspectral image dataset, and a model total loss function L is established. The parameters of the model are optimized based on the minimum model total loss function L. In this embodiment, the optimization of the model parameters using the validation dataset in step S3 specifically includes:

[0104] Establish the total loss function L of the model, optimize the connection weights through the selected optimizer and corresponding parameters, and obtain the optimal parameters of the model when the total loss function is minimized; the total loss function L of the model includes the reconstruction error loss function M of the abundance map calculated by RMSE β And calculate the error loss S of the spectral angle φ , the calculation formula of the total loss function M of the model is: Where σ and γ represent the weight parameter values.

[0105] In another embodiment of the present application, a marine oil spill hyperspectral unmixing system based on multi-scale residual attention is provided, the system comprising:

[0106] A data set construction module is used to construct a hyperspectral marine oil spill unmixing data set based on the marine oil spill hyperspectral image F and the marine oil spill end-member information E;

[0107] The multi-scale void convolution module is used to process the marine oil spill hyperspectral image Fh through different void ratios to obtain the multi-scale features FA of the marine oil spill hyperspectral image;

[0108] The spatial-spectral residual module includes three spatial-spectral self-attention blocks and a convolutional layer, which are used to set multiple spatial-spectral self-attention blocks based on the residual structure for the input multi-scale feature FA image. The spatial-spectral self-attention block inputs continuous non-overlapping small blocks, and each small block is mapped to a different feature map through the convolutional layer. The spatial self-attention structure is used to calculate the position correlation values ​​between these feature maps to construct a spatial attention map; the channel dimension is flattened through a fully connected layer and mapped to three different matrices through a convolutional network. The spectral correlation values ​​between the matrices are calculated using the spectral self-attention structure to construct a channel attention map to capture the correlation of spectral information.

[0109] The reconstruction module, including an MLP-Mixer layer, a fully connected layer and two convolutional layers, is used to output the abundance map after the first convolutional layer and extract the weight coefficients of the network as end-member information after the second convolutional layer.

[0110] The multi-scale atrous convolution module is used to obtain the multi-scale features FA of the marine oil spill hyperspectral image. The multi-scale atrous convolution module inputs the pre-processed marine oil spill hyperspectral image Fh into multiple atrous convolutions in parallel, and fully integrates the contextual information of different scales according to the different atrous convolutions, thereby improving the sensitivity of the model to the spectral features of the ground objects. In this embodiment, the pre-processed marine oil spill hyperspectral image Fh is input into four atrous convolutions in parallel for branch learning, and the marine oil spill hyperspectral image is sampled with 2D convolution kernels with atrous rates of 1, 2, 3, and 4 in turn, and each convolution step size is the same, and then the four feature blocks obtained are concat spliced ​​and fused to obtain the multi-scale feature FA.

[0111] The spatial-spectral residual module (also called the spatial-spectral residual attention module) can fully exploit the intrinsic similarities of the spatial dimension and the spectral dimension, and obtain the intrinsic similarity features FB of the hyperspectral image in the spatial dimension and the spectral dimension. In this embodiment, the spatial-spectral residual module includes three spatial-spectral self-attention blocks and a convolution layer. Among them, each spatial-spectral self-attention block includes a spatial self-attention structure and a spectral self-attention structure, and the spatial self-attention structure and the spectral self-attention structure are both subjected to convolution operations to achieve image segmentation. In addition, the spatial-spectral self-attention block and the spatial-spectral residual module both include a residual connection. The multi-scale feature FA processed by the multi-scale hole convolution module is input into the spatial-spectral residual module, processed by the three spatial-spectral self-attention blocks in sequence, and then processed by the convolution layer to output the intrinsic similarity features FB of the spatial dimension and the spectral dimension.

[0112] In this embodiment, the empty spectrum residual module obtains the intrinsic similarity feature FB including:

[0113] (1) The preprocessed hyperspectral ocean oil spill image Fh is divided into multiple patches. Each patch is mapped into three feature maps F1, F2, and F3 through a convolutional network, where the spatial feature F of the i-th pixel of the feature map F1 is 1,i , and the spatial feature F of the j-th pixel of the feature map F2 2,j The position correlation value between is defined as F ji , the calculation formula is as follows:

[0114]

[0115] Where M represents the number of pixels in the patch, F ji Represents the position correlation value between the i-th pixel of feature map F1 and the j-th pixel of feature map F2, which makes the correlation between these similar spatial features stronger;

[0116] (2) Through the position correlation value F between spatial features ji And the mapped F3 is used to calculate the spatial attention map S j , the formula is as follows:

[0117]

[0118] Where α represents the weight parameter value, F 3,i Represents the value of the i-th pixel in feature map F3.

[0119] (3) The spatial attention map S is obtained j With F h,j Perform residual operation and finally obtain the attention feature map E corresponding to pixel j j , the formula is as follows:

[0120] E j =S j +F h,j

[0121] Among them, F h,j Represents the jth pixel in the hyperspectral image.

[0122] (4) The attention feature map E after the spatial self-attention structure j The image is divided into multiple patches, each patch is flattened in the channel dimension and mapped into three feature maps A1, A2, and A3. The spectral correlation value between the eigenvalue A1i of the i-th band channel of the A1 matrix and the eigenvalue A2j of the j-th band channel of the A2 matrix is ​​defined as A j,i , the calculation formula is as follows:

[0123]

[0124] Where B represents the number of bands, A j,i Represents the spectral correlation value between the i-th band of feature map A1 and the j-th band of feature map A2. Similarly, this makes the correlation between these similar channels stronger;

[0125] (5) Through the correlation value A between the channel characteristics of different bands j,i And the mapped feature map A3 is used to calculate the spectral attention map W j , the formula is as follows:

[0126]

[0127] Where α represents the weight parameter value, A 3,i Represents the value corresponding to the i-th band channel of feature map A3.

[0128] (6) The obtained spectral attention map W j With F h,j Perform residual operation and finally obtain the attention feature M corresponding to channel j j Figure, the formula is as follows:

[0129] F A =W j +F h,j

[0130] where F h,j Represents the jth pixel in the hyperspectral image.

[0131] The reconstruction module unmixes the hyperspectral image features according to the intrinsic similarity features FB in the spatial and spectral dimensions to obtain the unmixed abundance map Ae and the unmixed end member Ee. The reconstruction module includes an MLP-Mixer layer (multi-perceptron), a fully connected layer and two convolutional layers. After the first convolutional layer, the corresponding abundance map Ae is obtained; after the second convolutional layer, the weight coefficient of the network is extracted to obtain the corresponding end member Ee.

[0132] In this embodiment, the process of the reconstruction module obtaining the unmixed abundance map Ae and the unmixed end member Ee according to the similarity feature FB includes:

[0133] The similarity feature FB is input into the reconstruction module, and the features output by the multi-scale spatial-spectral residual attention module are integrated through the multi-channel fully connected operation of the perceptron. Then, the corresponding abundance map Ae and end member Ee are obtained through the fully connected layer and two convolutional layers.

[0134] In addition, another embodiment of the present application protects a computer device, which includes a memory, a processor, and program instructions stored in the memory and executable by the processor, wherein the processor executes the program instructions to perform the steps in the aforementioned method.

[0135] In another embodiment of the present application, a computer-readable storage medium is protected, storing a computer program, wherein the computer program implements the aforementioned method when executed by a processor.

[0136] This application uses hyperspectral images and endmember information collected by a spectrometer to obtain a marine oil spill hyperspectral image unmixing dataset through a non-negative least squares algorithm. When constructing a marine oil spill hyperspectral image unmixing model based on multi-scale spatial-spectral residual attention, a spatial attention module and a spectral attention module are introduced, which effectively improves the spatial similarity of hyperspectral images and the similarity between highly correlated spectral bands, effectively improves the global information capture capability, and thus improves the efficiency and accuracy of marine oil spill hyperspectral unmixing, and improves the refined dynamic monitoring of marine oil spill pollution.

Claims

1. A method for unmixing hyperspectral images of marine oil spills based on multi-scale residual attention, characterized in that: The method comprises: Step S1: Obtain a marine oil spill hyperspectral image F and marine oil spill endmember information E, and construct a marine oil spill hyperspectral image unmixing dataset based on the marine oil spill hyperspectral image F and the marine oil spill endmember information E, wherein the dataset includes a training dataset and a validation dataset; Step S2: constructing a marine oil spill hyperspectral image unmixing model based on a multi-scale residual attention mechanism, wherein the unmixing model includes a multi-scale dilated convolution module, a spatial spectrum residual module, and a reconstruction module; Step S3: training the marine oil spill hyperspectral image unmixing model according to the training data set of the marine oil spill hyperspectral image unmixing data set, and optimizing the model parameters using the total loss function L to obtain the trained marine oil spill hyperspectral image unmixing model; Step S4: input the hyperspectral image to be unmixed into the trained marine oil spill hyperspectral unmixing model, and output the unmixed end members and abundance; The step S3 of training the ocean oil spill hyperspectral image unmixing model according to the ocean oil spill hyperspectral image unmixing dataset includes: Step S31: input the pre-processed marine oil spill hyperspectral image Fh into the multi-scale void convolution module to obtain the multi-scale feature FA; Step S32: inputting the multi-scale feature FA into the space-spectrum residual module to obtain the intrinsic similarity feature FB of the spatial dimension and the spectral dimension; Step S33: inputting the similarity feature FB into a reconstruction module to obtain an unmixed abundance graph Ae and an unmixed end member Ee; In step S32, the multi-scale feature FA is input into the space-spectrum residual module to obtain the intrinsic similarity feature FB of the spatial dimension and the spectral dimension, including: S321) The pre-processed hyperspectral ocean oil spill image Fh is divided into multiple patches, each patch is mapped into three feature maps F1, F2, and F3 through a convolutional network, where the spatial feature F of the i-th pixel of the feature map F1 is 1,i , and the spatial feature F of the j-th pixel of the feature map F2 2,j The position correlation value between is defined as F ji , the calculation formula is as follows: Where M represents the number of pixels in the patch, F ji Represents the position correlation value between the i-th pixel of feature map F1 and the j-th pixel of feature map F2, which makes the correlation between these similar spatial features stronger; S322) By using the position correlation value F between the spatial features ji And the mapped F3 is used to calculate the spatial attention map S j , the formula is as follows: Where α represents the weight parameter value, F 3,i Represents the value of the i-th pixel of feature map F3; S323) will get the spatial attention map S j With F h,j Perform residual operation and finally obtain the attention feature map E corresponding to pixel j j , the formula is as follows: E j =S j +F h,j where F h,j Represents the jth pixel in the hyperspectral image; S324) The attention feature map E after the spatial self-attention structure j The image is divided into multiple patches, each patch is flattened in the channel dimension and mapped into three feature maps A1, A2, and A3. The spectral correlation value between the eigenvalue A1i of the i-th band channel of the A1 matrix and the eigenvalue A2j of the j-th band channel of the A2 matrix is ​​defined as A j,i , the calculation formula is as follows: Where B represents the number of bands, A j,i Represents the spectral correlation value between the i-th band of feature map A1 and the j-th band of feature map A2. Similarly, this makes the correlation between these similar channels stronger; S325) by the correlation value A between the channel characteristics of different bands j,i And the mapped feature map A3 is used to calculate the spectral attention map W j , the formula is as follows: Where α represents the weight parameter value, A 3,i Represents the value corresponding to the i-th band channel of feature map A3; S326) The obtained spectral attention map W j With F h,j Perform residual operation and finally obtain the attention feature M corresponding to channel j j Figure, the formula is as follows: F A =W j +F h,j where F h,j Represents the jth pixel in the hyperspectral image.

2. The method for unmixing marine oil spill hyperspectral data based on multi-scale residual attention according to claim 1, characterized in that: In the step S1, constructing the unmixed data set of the marine oil spill hyperspectral image based on the marine oil spill hyperspectral image F and the marine oil spill end-member information E comprises: Sa1) normalizing the acquired hyperspectral image F, converting the dimension from three-dimensional to two-dimensional, and obtaining the pre-processed data Fh; Sa2) Convert the acquired marine oil spill end member information E from ascll format to mat format to obtain the end member information Egt in mat format; calculate the abundance matrix Agt and the abundance vector a according to the end member information Egt using the non-negative least squares algorithm nj The jth endmember of the nth pixel represents the proportion of the crop area on the ground surface, satisfying the following conditions: Where R represents the total number of end members, and the R abundance vectors represent the abundance matrix Agt; Sa3) The preprocessed marine oil spill hyperspectral image Fh, endmember information Egt and abundance matrix Agt are used together to construct a hyperspectral marine oil spill unmixing dataset, in which the marine oil spill hyperspectral image Fh is the training set, and the endmember information Egt and abundance matrix Agt are the validation set.

3. The method for unmixing marine oil spill hyperspectral data based on multi-scale residual attention according to claim 1, characterized in that: The step S31 further comprises: The preprocessed marine oil spill hyperspectral image Fh is input into four dilated convolutions in parallel for branch learning. The marine oil spill hyperspectral image is sampled with 2D convolution kernels with dilation rates of 1, 2, 3, and 4 respectively, and each convolution has the same step size. Then the four feature blocks are concat-fused to obtain the multi-scale feature FA.

4. The method for unmixing marine oil spill hyperspectral data based on multi-scale residual attention according to claim 1, characterized in that: In step S33, the similarity feature FB is input into the reconstruction module to obtain the unmixed abundance graph Ae and the unmixed end member Ee, including: The similarity feature FB is input into the reconstruction module, and the features output by the spatial-spectral residual module are integrated through the multi-channel fully connected operation of the perceptron. Then, the corresponding abundance map Ae and endmember Ee are obtained through the fully connected layer and two convolutional layers. The abundance matrix Agt and endmember information Egt in the hyperspectral marine oil spill unmixing dataset are used to verify the abundance map Ae and endmember Ee output by the model.

5. The method for unmixing marine oil spill hyperspectral data based on multi-scale residual attention according to claim 1, characterized in that: The optimization of the model parameters using the validation data set in step S3 specifically includes: Establish the total loss function L of the model, optimize the connection weights through the selected optimizer and corresponding parameters, and obtain the optimal parameters of the model when the total loss function is minimized; the total loss function L of the model includes the reconstruction error loss function M of the abundance map calculated by RMSE β And calculate the error loss S of the spectral angle φ , the calculation formula of the total loss function M of the model is: Where σ and γ represent the weight parameter values.

6. A marine oil spill hyperspectral unmixing system based on multi-scale residual attention, characterized in that: include: A data set construction module is used to construct a hyperspectral marine oil spill unmixing data set based on the marine oil spill hyperspectral image F and the marine oil spill end-member information E; The multi-scale void convolution module is used to process the marine oil spill hyperspectral image Fh through different void ratios to obtain the multi-scale features FA of the marine oil spill hyperspectral image; The spatial-spectral residual module includes three spatial-spectral self-attention blocks and a convolution layer, which are used to set multiple spatial-spectral self-attention blocks based on residual structures for the input multi-scale feature FA image, divide the preprocessed hyperspectral marine oil spill image Fh into multiple small blocks, and input continuous non-overlapping small blocks into the spatial-spectral self-attention block. Each small block is mapped to a different feature map through the convolution layer, and the position correlation values ​​between these feature maps are calculated using the spatial self-attention structure to construct a spatial attention map; the constructed spatial attention map is divided into multiple patches, each patch is flattened by a fully connected layer to flatten the channel dimension, and is mapped to three different matrices through a convolution network. The spectral self-attention structure is used to calculate the spectral correlation values ​​between the matrices, and a channel attention map is constructed to capture the correlation of spectral information; The reconstruction module, including an MLP-Mixer layer, a fully connected layer and two convolutional layers, is used to output the abundance map after the first convolutional layer and extract the weight coefficients of the network as end-member information after the second convolutional layer.

7. A computer device comprising a memory, a processor and program instructions stored in the memory and executable by the processor, wherein the processor executes the program instructions to implement the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images