A method for classifying underwater hyperspectral benthic based on improved mamba model

Abstract

The present application relates to the field of image classification, and discloses a method for classifying benthic types of coral reef underwater hyperspectral images based on an improved Mamba model, comprising the following steps: constructing a model for classifying benthic types of coral reef underwater hyperspectral images, CRUHI-Mamba; the model comprises a pixel-based embedding layer, a spatial-spectral Mamba backbone network, and a self-adaptive feature-decision fusion module; obtaining a dataset of coral reef underwater hyperspectral images, training the model; verifying the model after training by using a verification set; model prediction; collecting coral reef underwater hyperspectral images of a region to be classified, dividing the images into blocks, inputting the images into the model, splicing the predicted values output by the model by using a block linear weighted average strategy, and obtaining the final classification results. The method disclosed in the present application can reduce the training data requirement of the model, efficiently extract global spatial features and fine spectral features of benthic types, and provide method support for fine mapping of benthic habitats of coral reefs.

Description

Technical Field

[0001] This invention relates to the field of image classification, and in particular to a method for underwater hyperspectral benthic classification of coral reefs based on an improved Mamba model. Background Technology

[0002] Coral reef ecosystems are among the most productive and biodiverse ecosystems on Earth. Given the global trend of declining coral reef health, the need for rapid, efficient, and accurate acquisition of information on benthic habitats, communities, and habitats is more urgent than ever. Over the years, numerous coral reef monitoring methods have been developed and implemented. However, traditional in-situ underwater surveys by investigators are insufficient for efficiently mapping and monitoring large coral reef areas. Image-based remote sensing technology has revolutionized coral reef monitoring; however, underwater coral reef remote sensing monitoring is affected by weather and water conditions, resulting in image obstruction and distortion. Furthermore, limitations in spatial-spectral resolution restrict the scope to broad-based substrate and geomorphological classification studies. In addition, validating the accuracy of remote sensing-derived maps of underwater coral reefs is also challenging.

[0003] Over the past decade, underwater remote sensing imaging technology has become a mature and reliable technique for coral reef monitoring. An increasing number of researchers are utilizing annotated, high spatial and spectral resolution underwater coral reef images for segmentation and classification, benthic habitat mapping, and other studies. Notably, a key challenge in these studies is accurately and efficiently distinguishing corals from other seabed features in underwater images. To accurately and efficiently segment underwater coral reef images or classify corals or other benthic types, numerous studies on benthic type segmentation and classification have been conducted using traditional machine learning and classic deep learning methods based on underwater RGB images of coral reefs. Traditional machine learning methods such as Support Vector Machines (SVM) and KNN can only achieve good classification results on small spatial images, relying on pre-designed salient features such as texture and color. Deep learning segmentation methods for underwater coral reef RGB images based on models such as CNN and U-Net output coarse-grained segmentation results only for coral types. Pixel-level classification methods based on dense annotations, using models such as DeepLabv3+ and Transformer, can perform dense classification of the entire image. However, due to the limited spectral information in underwater RGB images and the high spatial heterogeneity of benthic types, these methods cannot accurately distinguish all benthic types contained in the images, and the classification accuracy is only between 80-90%. The locality and quadratic computational complexity of CNNs also lead to low classification accuracy and efficiency. In addition, these methods all require a large number of labeled samples for training, and a lot of time and effort is needed to build a classification sample database, which greatly reduces the utilization efficiency of underwater hyperspectral image data.

[0004] It is worth noting that the underwater environment of coral reefs is complex. Compared with terrestrial hyperspectral images, underwater hyperspectral images exhibit greater spatial heterogeneity, thus requiring the design of more robust spatial and spectral feature extraction models. Furthermore, the uneven distribution of information density among benthic coral reef types can easily lead to biases in the extracted feature information. Therefore, effectively utilizing and fusing these features to improve the classification accuracy of benthic types is an urgent problem to be solved. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a hyperspectral benthic classification method for coral reefs based on an improved Mamba model, aiming to efficiently extract global spatial features and fine spectral features of benthic types while reducing the amount of training data required for the model.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] A method for underwater hyperspectral benthic classification of coral reefs based on an improved Mamba model includes the following steps:

[0008] Step 1: Construct the underwater hyperspectral image classification model for coral reefs, CRUHI-Mamba;

[0009] The model includes a pixel-based embedding layer, a spatial-spectral Mamba backbone network, and an adaptive feature-decision fusion module;

[0010] The pixel-based embedding layer is used to extract the embedding of each pixel, capture non-linear features, and retain global detail information;

[0011] The spatial-spectral Mamba backbone network is used to extract discriminative spatial-spectral features, including a dual pooling layer for extracting local salient features while preserving overall information, and a dual-branch spatial feature extraction Mamba block and a spectral feature extraction Mamba block connected thereto.

[0012] The adaptive feature-decision fusion module is used to fully integrate spatial and spectral features, perform adaptive fusion at the feature level and classification result level, and output the fused prediction value.

[0013] Step two: Model training and validation;

[0014] A dataset was constructed by acquiring underwater hyperspectral images of coral reefs and dividing it into training and validation sets. The model was trained using the training set data. The underwater hyperspectral images of coral reefs were divided into blocks and input into the model. The model was trained using the labels corresponding to each image data block, and the loss was calculated. After training, the model was validated using the validation set.

[0015] Step 3, Model Prediction;

[0016] Underwater hyperspectral images of coral reefs in the area to be classified are collected, segmented, and then input into the model. The predicted values ​​output by the model are then stitched together using a segmented linear weighted average strategy to obtain the final classification result.

[0017] In the above scheme, underwater hyperspectral image data of coral reefs is represented as follows: ,in H and W It refers to the height and width of the spatial dimension. C Representing spectral dimensions; principal component analysis was applied to underwater hyperspectral image data of coral reefs to obtain a cube. , V This represents the spectral dimension after dimensionality reduction. H and W The spatial dimensions remain unchanged; the cube X As input data for the model;

[0018] The pixel-based embedding layer is in the form of a cube. As input, the following processing is performed:

[0019] ;

[0020] in, This represents the generated embedding result, where D is the embedding dimension; Conv 2D GN and SiLU represent a 1×1 kernel-sized convolutional layer, a grouped normalized layer, and a SiLU activation function, respectively.

[0021] In the above scheme, the feature extraction process of the spatial-spectral Mamba backbone network is represented as follows:

[0022] ;

[0023] Among them, LSSM backbone This represents the main feature extraction framework. Indicates LSSM backbone The output features, j = Pool, Spa, Spe, represent the dual pooling layer, the spatial feature extraction Mamba block, and the spectral feature extraction Mamba block, respectively.

[0024] In the above scheme, the calculation process of the adaptive feature-decision fusion module is as follows:

[0025] ;

[0026] in, The original output predicted probability values ​​are fused. This represents the probability that a pixel at a corresponding location in the image belongs to the k-th class, where K represents the total number of classes, and AFDFM represents the adaptive feature decision fusion module.

[0027] In the above scheme, the dual pooling layer includes a max pooling layer and an average pooling layer, which embeds the feature F. e The input is a max pooling layer for salient feature extraction, followed by an average pooling layer to integrate global information and suppress noise. Then, the extracted pooling features F are... Pool The spatial feature extraction Mamba block and the spectral feature extraction Mamba block are input separately to capture global spatial features and fine spectral features of coral reef benthic types with linear computational complexity. The forward process is described as follows:

[0028] ;

[0029] ;

[0030] in, This represents the output characteristics of the dual pooling layer. and These represent max pooling and average pooling, respectively; LSSM Spa and LSSM Spe These are the Mamba blocks for spatial feature extraction and spectral feature extraction, respectively. These are the extracted spatial features and spectral features, respectively.

[0031] In the above scheme, the calculation process for extracting the spatial features from the Mamba block is as follows:

[0032] ;

[0033] in, Indicates the flattened output. ; This represents the spatial features extracted by standard Mamba; To indicate flattening, Represents standard Mamba; , , These represent feature size reshaping, grouped normalization layer, and SiLU activation function, respectively; F Spa This represents the spatial characteristics of the output.

[0034] In the above scheme, the calculation process of the spectral feature extraction Mamba block is as follows:

[0035] ;

[0036] in, , These represent the spectral grouping output, the flattened spectral group features, and the spectral feature information mined by Mamba, respectively; BandGroup m,nIt is a spectral grouping layer, where m is the number of groups. Indicates the number of channels in each group; F Spe This indicates the spectral characteristics of the output.

[0037] In the above scheme, the adaptive feature-decision fusion module first adaptively fuses spatial features F at the feature level. Spa and spectral characteristics F Spe Furthermore, residual connections are introduced to mitigate potential overfitting during training, resulting in the fused feature F. fusion Then, for F respectively Spa F fusion and F Spe The three features are applied to the classifier to output classification prediction probability values. Finally, at the level of the original prediction values, i.e., the decision level, the importance of the three output prediction probability values ​​is adaptively estimated for decision fusion, realizing the optimized integration of multi-task classification results, and ultimately generating a more robust classification map. The fusion process is represented as follows:

[0038] ;

[0039] ;

[0040] ;

[0041] ;

[0042] in, For classifiers, V represents element-wise multiplication; spa and V spe L represents the fusion weights of spatial and spectral features, respectively, both initialized to 0.5 and adaptively updated during model training. Spa L Spe and These are the classification predictions based on the spatial, spectral, and spatial-spectral fusion features of the classifier, respectively; W spa W spe and W F These are the corresponding decision fusion weights, all initialized to 1 / 3; these weights are adaptively updated by the Adam optimizer in each iteration to determine the final fusion weights; This is the fused classification prediction value output by the adaptive feature-decision fusion module.

[0043] In the above scheme, the loss during the model training phase Calculate using the following formula:

[0044] ;

[0045] ;

[0046] Among them, s i and e i These represent the start and end positions of each image data block along the H dimension. N is the number of blocks. is the average height of each image data block, and O is the number of overlapping rows; Each training data block and its corresponding label are represented separately. This is the cross-entropy loss function.

[0047] In the above scheme, the block-based linear weighted average strategy is as follows:

[0048] Weighting function With splicing function Calculate using the following formula:

[0049] ;

[0050] ;

[0051] in, This is the block-by-block output of the model.

[0052] Through the above technical solution, the underwater hyperspectral benthic classification method for coral reefs based on the improved Mamba model provided by this invention has the following beneficial effects:

[0053] 1. This invention proposes the CRUHI-Mamba model, which, using less than 2% of the labeled samples, effectively extracts spatial and spectral features that satisfy fine-grained classification of coral reef benthic types by simulating long-distance interactions of the entire image at the pixel level. This results in high-quality coral reef benthic classification results with complete and clear boundaries. This pioneering technology is applied for the first time to underwater hyperspectral image classification of coral reefs, solving the problem that existing methods cannot effectively extract high-quality coral reef features with limited training samples, and significantly improving the accuracy and efficiency of coral reef classification.

[0054] 2. This invention designs a Spatial-Spectral Mamba Backbone Network (SSMB), which models global spatial dependencies and long-range spectral correlations through dual pooling layers and dual-branch spatial-spectral feature extraction Mamba blocks. This design enhances the model's spatial-spectral feature extraction capability, enabling rapid and efficient extraction of global spatial and fine spectral features of benthic types, thereby significantly improving the accuracy and efficiency of feature extraction and providing higher-quality basic features for subsequent classification tasks.

[0055] 3. This invention proposes an Adaptive Feature-Decision Fusion Module (AFDFM), which dynamically balances spatial and spectral feature information at the feature level to avoid information forgetting; and coordinates the balanced fusion of multi-task classification results at the decision level, effectively enhancing feature utilization and fusion capabilities, and improving model robustness. In this way, this invention can effectively alleviate feature bias and fusion challenges caused by the imbalance of information density in benthic types, significantly improving the accuracy and reliability of classification.

[0056] 4. For underwater hyperspectral image classification of coral reefs under high resolution and large data volumes, this invention designs a Chunked Linear Weighted Averaging (CLWA) strategy. This strategy corrects overlapping areas in the classification results of adjacent image blocks by stitching together, effectively reducing stitching errors between a large number of adjacent image block classification results, improving the prediction consistency of boundary categories, solving the stitching error problem caused by image block segmentation in traditional classification methods, and significantly improving the classification accuracy of coral reefs. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0058] Figure 1 This is a schematic diagram of the overall framework of a CRUHI-Mamba model disclosed in an embodiment of the present invention;

[0059] Figure 2 For the adaptive feature-decision fusion module structure, (a) classifier structure, (b) feature-level fusion weight initialization and update, and (c) decision-level fusion weight initialization and update;

[0060] Figure 3 A schematic diagram of the block-based linear weighted average strategy for model inference;

[0061] Figure 4 The following are the standard RGB composite true-color images of hyperspectral data and their acquisition locations: (a) shows the location of Curaçao in the Caribbean Sea; (b) shows the location of the Carmabi survey station; and (c) shows three hyperspectral images and their magnified local views.

[0062] Figure 5 The spectral curve of the average radiation value of the sample;

[0063] Figure 6For the benthic classification map of UHI (data number 005), (a) standard RGB composite true-color image of UHI data (data number 005); (b) classification ground truth; (c) RF; (d) RRSN; (e) 3-DCNN; (f) FullyContNet; (g) Spectralformer; (h) SSFTT; (i) GSC-ViT; (j) MambaHSI; (k) CRUHI-Mamba of the present invention;

[0064] Figure 7 For the benthic classification map of UHI (data number 006), (a) standard RGB composite true-color image of UHI (data number 006); (b) classification ground truth; (c) RF; (d) RRSN; (e) 3-DCNN; (f) FullyContNet; (g) Spectralformer; (h) SSFTT; (i) GSC-ViT; (j) MambaHSI; (k) CRUHI-Mamba of the present invention;

[0065] Figure 8 For the benthic classification map of UHI (019), (a) standard RGB composite true-color image of UHI (019); (b) classification ground truth; (c) RF; (d) RRSN; (e) 3-DCNN; (f) FullyContNet; (g) Spectralformer; (h) SSFTT; (i) GSC-ViT; (j) MambaHSI; (k) CRUHI-Mamba of the present invention;

[0066] Figure 9 For the comparison of training parameter performance of different models, (a) is 005 data, (b) is 006 data, and (c) is 019 data. Detailed Implementation

[0067] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0068] This invention provides a method for underwater hyperspectral benthic classification of coral reefs based on an improved Mamba model, comprising the following steps:

[0069] Step 1: Construct the underwater hyperspectral image classification model for coral reefs, CRUHI-Mamba;

[0070] 1. Overall Model Architecture

[0071] The framework of the CRUHI-Mamba model proposed in this invention is as follows: Figure 1As shown, it is built on a pure Mamba model. The framework consists of three parts: a pixel-based embedding layer; a Spatial-Spectral Mambabackbone (SSMB); and an Adaptive Feature-Decision Fusion (AFDFM) module.

[0072] Underwater hyperspectral image data of coral reefs are represented as ,in H and W It refers to the height and width of the spatial dimension. C Representing spectral dimensions; principal component analysis was applied to underwater hyperspectral image data of coral reefs to obtain a cube. , V This represents the spectral dimension after dimensionality reduction. H and W The spatial dimensions remain unchanged; the cube X As input data for the CRUHI-Mamba model;

[0073] Pixel-based embedding layers: This process uses a cube-shaped... As input, the pixel-based embedding layer extracts the embedding of each pixel, captures non-linear features, and preserves global detail information. This process can be formulated as follows:

[0074] ;

[0075] in, This represents the generated embedding result, where D is the embedding dimension. 2D GroupNorm (GN) and SiLU represent a 1×1 kernel-sized convolutional layer, a grouped normalized layer, and a SiLU activation function, respectively.

[0076] The Spatial-Spectral Mamba backbone (SSMB) is used for fast and efficient extraction of discriminative spatial-spectral features. It mainly consists of a dual-pooling layer for extracting local salient features while preserving overall information, and connected to it a dual-branch Mamba block for spatial feature extraction and a Mamba block for spectral feature extraction. The feature extraction process is shown below:

[0077] ;

[0078] Among them, LSSM backbone This represents the main feature extraction framework. Indicates LSSM backbone The output features, j = Pool, Spa, Spe, represent the dual pooling layer, the spatial feature extraction Mamba block, and the spectral feature extraction Mamba block, respectively.

[0079] The Adaptive Feature-Decision Fusion (AFDFM) module comprehensively integrates spatial and spectral features, performing adaptive fusion at both the feature level and the classification result level, and outputting the fused predicted probability values. This provides strong support for the final output of accurate and reliable classification results of coral reef benthic types. This process can be formulated as follows:

[0080] ;

[0081] in, The original output predicted probability values ​​are fused. This represents the probability that a pixel at a corresponding location in the image belongs to the k-th class (K represents the total number of classes), and AFDFM represents the adaptive decision fusion module.

[0082] Finally, bilinear interpolation is used to... Adjust the image to the same size as the input image, and apply the argmax function to determine the category of each pixel in the image to obtain the classification results of coral reef benthic types. ;

[0083] .

[0084] 2. Spatial-Spectral Mamba Backbone Network (SSMB)

[0085] Underwater hyperspectral images of coral reefs are characterized by large data volume and high spatial heterogeneity, and the distribution of benthic types is complex. Directly using existing hyperspectral classification methods for benthic classification of coral reefs has the problems of large sample requirements and inability to effectively extract spatial and spectral features that meet the requirements for fine-grained classification of benthic types.

[0086] To address the aforementioned problems, this invention designs an SSMB backbone network comprising a dual-pooling layer and a dual-branch spatial-spectral Mamba feature extraction structure. First, the dual-pooling layer reduces the spatial size of the data and lowers computational complexity while preserving important features. The embedded features F... e The input is a max pooling layer to extract salient features such as edges and textures. Then, an average pooling layer is used to integrate global information and suppress noise. Finally, the extracted pooling features F... Pool The input consists of two Mamba blocks for spatial and spectral feature extraction, capturing global spatial and fine spectral features of coral reef benthic types with linear computational complexity. This lightweight and concise structure efficiently captures discriminative features. The detailed structure of LSSM is as follows: Figure 1 As shown. The forward process can be described as follows:

[0087] ;

[0088] ;

[0089] in, This represents the output characteristics of the dual pooling layer. and These represent max pooling and average pooling, respectively; LSSM Spa and LSSM Spe These are the Mamba blocks for spatial feature extraction and spectral feature extraction, respectively. These are the extracted spatial features and spectral features, respectively.

[0090] (1) Spatial feature extraction Mamba block (LSSM) Spa Block)

[0091] Considering the complex distribution and strong spatial heterogeneity of benthic types in coral reefs, the entire F... Pool As input, Mamba is used as the basic unit to construct a spatial feature extractor, establishing global pixel-level spatial dependencies with linear computational complexity. LSSM Spa Detailed structure as follows Figure 1 As shown. The forward process can be described as follows:

[0092] ;

[0093] in, Indicates the flattened output. ; This represents the spatial features extracted by standard Mamba; To indicate flattening, Represents standard Mamba; , , These represent feature size reshaping, grouped normalization layer, and SiLU activation function, respectively; F Spa This represents the spatial characteristics of the output.

[0094] Studies have shown that Mamba-based classification methods can meet classification performance requirements using only simple flattening methods. This invention employs a row-optimized flattening method, utilizing the spatial features extracted by standard Mamba. The model sequentially performs feature size reshaping, GN layer, and SiLU activation function layer, and uses residual connections to enhance its learning ability. Finally, it outputs spatial features F. Spa .

[0095] (2) Spectral feature extraction Mamba block (LSSM) Spe Block)

[0096] To fully utilize the spectral information in the blue-green spectral regions of underwater hyperspectral images of coral reefs, and to extract fine and separable spectral features, the spectral bands are divided into m groups, as follows: Figure 1 As shown, Mamba is used to model long-range spectral correlations between different spectral groups, and spectral features are updated based on the relationships between the mined spectral groups. LSSM Spe Block's output F Spe It can be obtained from the following formula:

[0097] ;

[0098] in, , These represent the spectral grouping output, the flattened spectral group features, and the spectral feature information mined by Mamba, respectively; BandGroup m,n It is a spectral grouping layer, where m is the number of groups. Indicates the number of channels in each group; F Spe This represents the spectral characteristics of the output. Next, it is compared with LSSM. Spe Similar to the block, the process involves feature size reshaping, GN layers, SiLU activation function layers, and residual connections in sequence. Finally, the LSSM is obtained. Spe The output spectral characteristics of the block F Spe .

[0099] 3. Adaptive Feature-Decision Fusion Module (AFDFM)

[0100] To maximize the utilization of extracted spatial and spectral features without losing beneficial classification information, and to mitigate feature bias and fusion problems caused by uneven information density among benthic species, this invention designs an AFDFM module, such as... Figure 1 As shown. First, in the feature-level adaptive fusion spatial features F Spa and spectral characteristics F Spe To prevent key features from being forgotten, residual connections are introduced to mitigate potential overfitting during training, resulting in the fused feature F. fusion ,like Figure 2 As shown in (b). Next, for F respectively Spa F fusion and F Spe The three features are applied to the classifier, which outputs the classification prediction probability value. The classifier structure is as follows: Figure 2 As shown in (a). Finally, at the level of the original predicted values, i.e. the decision level, the importance of the three output predicted probability values ​​is adaptively estimated for decision fusion, as shown in (a). Figure 2 As shown in (c), the results of multi-task classification are optimized and integrated to generate a more robust classification map. The fusion process is represented as follows:

[0101] ;

[0102] ;

[0103] ;

[0104] ;

[0105] in, For classifiers, V represents element-wise multiplication; spa and V spe L represents the fusion weights of spatial and spectral features, respectively, both initialized to 0.5 and adaptively updated during model training. Spa L Spe and These are the classification predictions based on the spatial, spectral, and spatial-spectral fusion features of the classifier, respectively; W spa W spe and W F These are the corresponding decision fusion weights, all initialized to 1 / 3; these weights are adaptively updated by the Adam optimizer in each iteration to determine the final fusion weights; This is the fused classification prediction value output by the adaptive feature-decision fusion module.

[0106] Step two: Model training and validation;

[0107] A dataset was constructed by acquiring underwater hyperspectral images of coral reefs and dividing it into training and validation sets. The model was trained using the training set data. The underwater hyperspectral images of coral reefs were divided into blocks and input into the model. The model was trained using the labels corresponding to each image data block, and the loss was calculated. After training, the model was validated using the validation set.

[0108] CRUHI-Mamba is a pixel-based classification method that takes the entire hyperspectral image as input, trains it, and then infers (validates and tests) to directly output the classification result of the entire image. To balance high efficiency in model training and inference with high accuracy in classification results, this invention addresses the classification of underwater hyperspectral images of coral reefs with high resolution and large data volumes. Training and inference (i.e., image data blocks) are performed by dividing the data into blocks (hyperspectral image data blocks) along the H dimension. Figure 3 (P1, P2, ...).

[0109] Loss during model training Calculate using the following formula:

[0110] ;

[0111] ;

[0112] Among them, s i and e i These represent the start and end positions of each image data block along the H dimension. N is the number of blocks. is the average height of each image data block, and O is the number of overlapping rows; Each training data block and its corresponding label are represented separately. This is the cross-entropy loss function.

[0113] After the block training is completed, when performing inference (verification, testing), it is necessary to process the results of each block of inference (i.e., (The fused classification predictions output by the AFDFM module are concatenated.)

[0114] Step 3, Model Prediction;

[0115] Underwater hyperspectral images of coral reefs in the area to be classified are collected, segmented, and then input into the model. The predicted values ​​output by the model are then stitched together using a segmented linear weighted average strategy to obtain the final classification result.

[0116] To reduce concatenation errors between adjacent data blocks and improve the prediction consistency of boundary categories, this invention designs a Chunked Linear Weighted Averaging (CLWA) strategy for concatenation. Weighting function... With splicing function Calculate using the following formula:

[0117] ;

[0118] ;

[0119] in, This is the block-by-block output of the model.

[0120] in, The weight of the first block is set so that it linearly decreases from 1 to 0.5 from the starting position s0 to the ending position e0. The middle block... The data is divided into two equal parts. The weight of the first part increases from 0.5 to 1, and the weight of the second part decreases from 1 to 0.5. The weight of the last part is set in the opposite way to the first part, increasing linearly from 0.5 to 1. Figure 3 As shown. The block-by-block output of CRUHI-Mamba is compared with the corresponding weight w. i The results are multiplied and normalized. Finally, continuous and accurate predictions of the entire underwater hyperspectral image of the coral reef are obtained. By employing the CLWA strategy, not only were classification accuracy and model efficiency improved, but stitching artifacts were also effectively reduced, ensuring the accuracy and continuity of prediction results.

[0121] Experimental verification:

[0122] 1. Experimental Dataset

[0123] The underwater hyperspectral images of coral reefs were acquired by divers using a hyperspectral imaging system (HyperDiver) at eight coral reef sites around Curaçao in the Caribbean Sea. The spectral range is 400-800 nm, encompassing 400 bands. The dataset can be downloaded from https: / / doi.org / 10.1594 / PANGAEA.911300. This invention uses underwater hyperspectral images of coral reefs numbered 005, 006, and 019 acquired at the Kamab site, see [link to image]. Figure 4 Simultaneously, infrared bands lacking useful signals were discarded, and only raw radiation data from 350 bands in the 400-750nm range were used for coral reef benthic classification, such as... Figure 5 As shown. More relevant information on underwater hyperspectral images of coral reefs and benthic classification experiments is given in Table 1. The categories and sample statistics of the benthic classification experiments of coral reefs are shown in Table 2. The benthic classification system is consistent with that of Chennu (SCHÜRHOLZ D, CHENNU A. Digitizing the coral reef: Machine learning of underwater spectral images enables dense taxonomic mapping of benthic habitats[J]. Methods in Ecology and Evolution,2023,14(2): 596-613.).

[0124] Table 1. Detailed information on underwater hyperspectral image classification experiments for coral reefs.

[0125]

[0126] Table 2 Benthic types and sample statistics of underwater hyperspectral images of coral reefs, "-" indicates null values;

[0127]

[0128] 2. Experimental Setup

[0129] The experimental environment for CRUHI-Mamba was implemented using the PyTorch deep learning framework, and all experiments were conducted on the same hardware: a Hygon C86-3G (OPN:3350) CPU @ 3.00 GHz, an NVIDIA GeForce RTX4090 D GPU, and 128GB of RAM. During the experiments, 135 and 30 pixels of data randomly selected from UHI were used as the training and validation sets, respectively, with the remaining samples serving as the test set (Table 1). Since the proposed CRUHI-Mamba takes the entire underwater hyperspectral image of a coral reef as input and outputs the prediction results for the entire image, the batch size was set to 1. The experiments used an Adam optimizer with a learning rate of 0.0001 for iterative training for 200 epochs. The number of overlapping rows O was set to 100 rows. LSSM Spe The number of block spectral groups m was set to 8, and the number of GN groups G was set to 16. The Mamba-SSM version used in the experiment was 1.2.0, and the embedding dimension D was set to 64, where LSSM... Spe The block state dimension "d_state", the width of the local convolution "d_conv", and the expansion factor "expand" are set to 16, 4, and 1, respectively. LSSM Spa The corresponding parameters for the block are set to 16, 4, and 4 respectively. All other Mamba block parameters are set to their default values.

[0130] 3. Comparison Methods and Evaluation Indicators

[0131] To evaluate the classification accuracy and computational cost of CRUHI-Mamba, it was compared with eight other hyperspectral image classification methods. Specifically, these included the classic Random Forest (RF) algorithm, CNN-based 3-DCNN, FullyContNet, SSRN, Transformer-based Spectralformer, SSFTT, and GSC-ViT algorithms, and the Mamba-based MambaHSI algorithm. SSRN is currently a deep learning model applied to underwater hyperspectral image classification. All compared methods were configured with hyperparameters according to the values ​​specified in the original literature and were trained and tested using the same dataset, evaluation criteria, and experimental environment as CRUHI-Mamba proposed in this invention.

[0132] The hyperparameter configurations for each algorithm are described in the original literature:

[0133] [1] The classic Random Forest (RF) algorithm:

[0134] HAM J, CHEN Y, CRAWFORD M M, et al. Investigation of the randomforest framework for classification of hyperspectral data[J]. IEEEtransactions on geoscience and remote sensing, 2005,43(3): 492-501;

[0135] [2] 3-D CNN based on CNN:

[0136] BEN HAMIDA A, BENOIT A, LAMBERT P, et al. 3-D Deep Learning Approachfor Remote Sensing Image Classification[J]. IEEE Transactions on Geoscienceand Remote Sensing, 2018,56(8): 4420-4434;

[0137] [3] FullyContNet:

[0138] WANG D, DU B, ZHANG L. Fully Contextual Network for HyperspectralScene Parsing[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022,60: 1-16;

[0139] [4] SSRN:

[0140] ZHONG Z, LI J, LUO Z, et al. Spectral-Spatial Residual Network forHyperspectral Image Classification: A 3-D Deep Learning Framework[J]. IEEEtransactions on geoscience and remote sensing, 2018,56(2): 847-858;

[0141] [5] Spectralformer based on Transformer:

[0142] Hong D, Han Z, Yao J, et al. SpectralFormer: Rethinking Hyperspectral Image Classification With Transformers[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-15;

[0143] [6] SSFTT:

[0144] Sun L, Zhao G, Zheng Y, et al. Spectral–Spatial Feature Tokenization Transformer for Hyperspectral Image Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60({}): 1-14;

[0145] [7] GSC-ViT algorithm:

[0146] Zhao Z, Xu X, Li S, et al. Hyperspectral Image Classification Using Groupwise Separable Convolutional Vision Transformer Network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 1-17;

[0147] [8] Mamba-based MambaHSI algorithm:

[0148] Li Y, Luo Y, Zhang L, et al. MambaHSI: Spatial–Spectral Mamba for Hyperspectral Image Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 1-16.

[0149] Three evaluation metrics widely used in hyperspectral image classification—overall accuracy (OA), average accuracy (AA), and Kappa coefficient—were employed to quantitatively evaluate the classification model's performance. Furthermore, training and inference time were used to represent time complexity, and the number of training parameters was used to represent space complexity. All experiments were randomly performed five times to eliminate the influence of random sample selection and parameter initialization. Higher mean values ​​for these metrics generally indicate better model performance.

[0150] 4. Experimental Results and Comparative Analysis

[0151] Based on underwater hyperspectral images of coral reefs numbered 005, 006, and 019, benthic classification was performed using the CRUHI-Mamba method proposed in this invention and eight comparison methods. The classification results were comprehensively compared and analyzed from the perspectives of quantitative accuracy evaluation indicators and qualitative visualization results, and the computational complexity and efficiency of the models were also compared.

[0152] 4.1 Quantitative Evaluation of Classification Accuracy

[0153] The proposed CRUHI-Mamba method was comprehensively compared with eight state-of-the-art (SOTA) HSI classification methods from both quantitative and qualitative perspectives for the classification results of underwater coral reef hyperspectral images numbered 005, 006, and 019. The quantitative accuracy evaluation results are shown in Tables 3-5; the qualitative visualization evaluation results are shown in... Figure 6-8 .

[0154] Table 3. Classification accuracy of different methods on UHI 005

[0155]

[0156] Table 4. Classification accuracy of different methods on UHI 006

[0157]

[0158] Table 5. Classification accuracy of different methods on UHI 019

[0159]

[0160] As shown in the quantitative results in Table 3-5, the CRUHI-Mamba model proposed in this invention clearly performs best among all methods. OA, AA, and Kappa scores are all highest on the three underwater hyperspectral images of coral reefs, with OA exceeding 99%, indicating superior classification performance. On the 005 hyperspectral image, only 0.79% of the labeled samples were used to achieve the highest classification accuracy for 12 benthic types, with an OA of 99.56%. Compared to RF, CRUHI-Mamba has an OA more than 8.9% higher, mainly due to the limitations of RF in terms of handcrafted feature descriptors and its inability to utilize spatial feature information. Compared to CNN-based and Transformer-based models, the OA of the model proposed in this invention is on average more than 3.3% and 5.9% higher, respectively. These models suffer from low classification accuracy because their limited receptive fields make it difficult to establish long-term dependencies and they do not fully utilize the detailed spectral interaction information and global spatial information of coral reef benthic types.

[0161] For the classification results of the 006 data (Table 4), this invention selected 0.95% of the samples for training and validation, and the proposed model achieved an accuracy of 99.19%. Compared with the second-place Mamba-based model MambaHSI, OA, AA, and Kappa improved by approximately 1.14%, 0.89%, and 1.48%, respectively. The experiments in the 019 images were performed using 1.91% of the labeled samples. The classification results in Table 5 clearly show that CRUHI-Mamba exhibited the highest accuracy across all three metrics. Compared with CNN-based and Transformer-based models, OA achieved significant improvements of 8.98% and 5.81%, respectively, while Kappa exceeded 10.18% and 6.60%.

[0162] Furthermore, both Mamba-based MambaHSI and the proposed CRUHI-Mamba achieve an overall accuracy (OA) of over 98% (Table 3-5), outperforming other methods. In contrast, other methods show significant variations in classification accuracy across the three hyperspectral images, indicating that the Mamba-based model performs better and is more stable. Firstly, this is due to Mamba's powerful ability to model long-range dependencies with linear complexity, providing strong support for efficient extraction of global spatial and detailed spectral information. Secondly, both the proposed CRUHI-Mamba and MambaHSI are based on the entire image, employing pixel-level spatial and spectral feature extraction structures, which fully leverages the advantages of Mamba to achieve high-precision classification of underwater coral reef hyperspectral images. It should be noted that although the proposed CRUHI-Mamba's OA is only 0.29%–1.14% higher than MambaHSI in the classification of the three hyperspectral images, MambaHSI has a more complex structure and lower efficiency, significantly exceeding CRUHI-Mamba in terms of model training and inference time and model parameters, which will be analyzed in detail in the next section.

[0163] 4.2 Qualitative Assessment of Benthic Classification Map

[0164] from Figure 6-8 The visualized qualitative results show that the classification results of the RF method have significant salt-and-pepper noise. The classification map of FullyContNet based on CNN has the worst visual effect, with large misclassified regions and incomplete types. The classification maps of the Transformer-based models have blurry boundaries and contain a large amount of small-region noise. Among them, the Spectralformer model has serious misclassification problems for coral types such as Meandrina meandrites, Porites astreoides, and Millepora in the underwater hyperspectral image of coral reef number 005. MambaHSI, due to its complex model structure and large computational cost, cannot input the entire hyperspectral image for training and inference with fewer blocks. This results in many stitching artifacts in the classification map, affecting the smoothness of the classification map. In contrast, the proposed CRUHI-Mamba has the fewest misclassifications and is almost completely consistent with the ground truth. It presents high-quality coral reef benthic classification maps with complete, smooth, and clear boundaries with minimal noise. These visualized classification maps further reveal the effectiveness and superiority of the proposed method.

[0165] In summary, compared with state-of-the-art models based on CNN, Transformer and Mamba, and traditional RF, the proposed CRUHI-Mamba achieves the best performance in terms of accuracy and visualization for benthic classification of underwater hyperspectral images of coral reefs.

[0166] 4.3 Comparative Analysis of Computational Complexity and Efficiency

[0167] The advantages of CRUHI-Mamba go beyond accuracy. Figure 9 The training parameters of the model are shown, the training and inference times for the entire image are reported in Table 6, and it is compared with other models on three datasets.

[0168] Table 6 Comparison of Model Training and Inference Times

[0169]

[0170] 1) In terms of parameters and operational accuracy (OA), CRUHI-Mamba achieved the best results. On the three datasets, the model parameters were reduced by an average of 64%, 49%, and 15% compared to CNN-based, Transformer-based, and Mamba-based models, respectively. This aligns with our design motivation of achieving high classification accuracy with minimal parameters.

[0171] 2) The CRUHI-Mamba model has the shortest inference time for the entire underwater hyperspectral image of the coral reef, approximately twice as fast as FullyContNet and MambaHSI, both image-level models. Notably, the inference time is reduced by 1 to 3 orders of magnitude compared to other models. A comprehensive analysis of the runtime shows that our model remains competitive in classification efficiency compared to other models.

[0172] 3) It should be noted that on No. 005 UHI, the training time of MambaHSI is 84 times that of CRUHI-Mamba. This is because No. 005 UHI has a large dataset and the MambaHSI structure is complex, exceeding GPU memory usage and consuming RAM during training, thus reducing data reading and computation efficiency and significantly increasing training time. Even for No. 006 and 019 UHI, which have smaller datasets, the training time is more than 7 times that of CRUHI-Mamba. These experimental results confirm that the CRUHI-Mamba model not only has excellent classification performance but also the advantages of being lightweight and efficient.

[0173] 4.4 Ablation Experiment

[0174] This invention conducted an ablation study to analyze the impact of each module, including the spatial feature extraction Mamba block (LSSM_Spa), the spectral feature extraction Mamba block (LSSM_Spe), the spatial-spectral Mamba feature extraction backbone (SSMB), the direct addition of spatial and spectral features, and the adaptive feature-decision fusion module (AFDFM).

[0175] Table 7. OA accuracy (%) of underwater hyperspectral classification and ablation experiments on coral reefs

[0176]

[0177] From the ablation experiment results (Table 7), the following conclusions can be drawn:

[0178] (1) LSSM Spa The block's performance surpasses that of LSSM. Spe The block indicates that spatial features are more discriminative than spectral features;

[0179] (2) Directly adding spatial and spectral information does not always improve model performance. For example, the performance of No. 019 decreased slightly, which indicates that spatial and spectral features are not entirely complementary for underwater hyperspectral classification of coral reefs.

[0180] (3) However, by adaptively fusing the output results of different levels through the proposed AFDFM module, spatial and spectral information are fully integrated, thereby improving the classification performance of the model. In particular, for No. 019, the AFDFM module effectively utilizes spatial and spectral information, resulting in a significant improvement in OA, which demonstrates its effectiveness and high robustness.

[0181] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for underwater hyperspectral benthic classification of coral reefs based on an improved Mamba model, characterized in that, Includes the following steps: Step 1: Construct the underwater hyperspectral image classification model for coral reefs, CRUHI-Mamba; The model includes a pixel-based embedding layer, a spatial-spectral Mamba backbone network, and an adaptive feature-decision fusion module; The pixel-based embedding layer is used to extract the embedding of each pixel, capture non-linear features, and retain global detail information; The spatial-spectral Mamba backbone network is used to extract discriminative spatial-spectral features, including a dual pooling layer for extracting local salient features while preserving overall information, and a dual-branch spatial feature extraction Mamba block and a spectral feature extraction Mamba block connected thereto. The adaptive feature-decision fusion module is used to fully integrate spatial and spectral features, perform adaptive fusion at the feature level and classification result level, and output the fused prediction value. Step two: Model training and validation; A dataset was constructed by acquiring underwater hyperspectral images of coral reefs and dividing it into training and validation sets. The model was trained using the training set data. The underwater hyperspectral images of coral reefs were divided into blocks and input into the model. The model was trained using the labels corresponding to each image data block, and the loss was calculated. After training, the model was validated using the validation set. Step 3, Model Prediction; Underwater hyperspectral images of coral reefs in the area to be classified are collected, divided into blocks, and then input into the model. The predicted values ​​output by the model are then stitched together using a block-based linear weighted average strategy to obtain the final classification result. The feature extraction process of the spatial-spectral Mamba backbone network is as follows: ； Among them, LSSM backbone This represents the main feature extraction framework. Indicates LSSM backbone The output features, j = Pool, Spa, Spe, represent the dual pooling layer, the spatial feature extraction Mamba block, and the spectral feature extraction Mamba block, respectively; The dual pooling layer includes a max pooling layer and an average pooling layer, which embeds the feature F. e The input is a max pooling layer for salient feature extraction, followed by an average pooling layer to integrate global information and suppress noise. Then, the extracted pooling features F are processed. Pool The spatial feature extraction Mamba block and the spectral feature extraction Mamba block are input separately to capture global spatial features and fine spectral features of coral reef benthic types with linear computational complexity. The forward process is described below: ； ； in, This represents the output characteristics of the dual pooling layer. and These represent max pooling and average pooling, respectively; LSSM Spa and LSSM Spe These are the Mamba blocks for spatial feature extraction and spectral feature extraction, respectively. These are the extracted spatial features and spectral features, respectively.

2. The underwater hyperspectral benthic classification method for coral reefs based on the improved Mamba model according to claim 1, characterized in that, Underwater hyperspectral image data of coral reefs are represented as ,in H and W It refers to the height and width of the spatial dimension. C Representing spectral dimensions; principal component analysis was applied to underwater hyperspectral image data of coral reefs to obtain a cube. , V This represents the spectral dimension after dimensionality reduction. H and W The spatial dimensions remain unchanged; the cube X As input data for the model; The pixel-based embedding layer is in the form of a cube. As input, the following processing is performed: ； in, This represents the generated embedding result, where D is the embedding dimension; Conv 2D GN and SiLU represent a 1×1 kernel-sized convolutional layer, a grouped normalized layer, and a SiLU activation function, respectively.

3. The underwater hyperspectral benthic classification method for coral reefs based on the improved Mamba model according to claim 1, characterized in that, The calculation process of the adaptive feature-decision fusion module is as follows: ； in, The original output predicted probability values ​​are fused. This represents the probability that a pixel at a corresponding location in the image belongs to the k-th class, where K represents the total number of classes, and AFDFM represents the adaptive feature decision fusion module.

4. The underwater hyperspectral benthic classification method for coral reefs based on the improved Mamba model according to claim 1, characterized in that, The calculation process for extracting the spatial features from the Mamba block is as follows: ； in, Indicates the flattened output. ; This represents the spatial features extracted by standard Mamba; To indicate flattening, Represents standard Mamba; , , These represent feature size reshaping, grouped normalization layer, and SiLU activation function, respectively; F Spa This represents the spatial characteristics of the output.

5. A method for underwater hyperspectral benthic classification of coral reefs based on an improved Mamba model, as described in claim 1, is characterized in that... The calculation process for the Mamba block for spectral feature extraction is as follows: ； in, , These represent the spectral grouping output, the flattened spectral group features, and the spectral feature information mined by Mamba, respectively; BandGroup m,n It is a spectral grouping layer, where m is the number of groups. Indicates the number of channels in each group; F Spe This indicates the spectral characteristics of the output.

6. A method for underwater hyperspectral benthic classification of coral reefs based on an improved Mamba model, as described in claim 5, is characterized in that... The adaptive feature-decision fusion module first adaptively fuses spatial features F at the feature level. Spa and spectral characteristics F Spe Furthermore, residual connections are introduced to mitigate potential overfitting during training, resulting in the fused feature F. fusion Then, for F respectively Spa F fusion and F Spe The three features are applied to the classifier to output classification prediction probability values. Finally, at the level of the original prediction values, i.e., the decision level, the importance of the three output prediction probability values ​​is adaptively estimated for decision fusion, realizing the optimized integration of multi-task classification results, and ultimately generating a more robust classification map. The fusion process is represented as follows: ； ； ； ； in, For classifiers, V represents element-wise multiplication; spa and V spe L represents the fusion weights of spatial and spectral features, respectively, both initialized to 0.5 and adaptively updated during model training. Spa L Spe and These are the classification predictions based on the spatial, spectral, and spatial-spectral fusion features of the classifier, respectively; W spa W spe and W F These are the corresponding decision fusion weights, all initialized to 1 / 3; these weights are adaptively updated by the Adam optimizer in each iteration to determine the final fusion weights; This is the fused classification prediction value output by the adaptive feature-decision fusion module.

7. A method for underwater hyperspectral benthic classification of coral reefs based on an improved Mamba model according to claim 2, characterized in that, Loss during model training Calculate using the following formula: ； ； Among them, s i and e i These represent the start and end positions of each image data block along the H dimension. N is the number of blocks. is the average height of each image data block, and O is the number of overlapping rows; Each training data block and its corresponding label are represented separately. This is the cross-entropy loss function.

8. A method for underwater hyperspectral benthic classification of coral reefs based on an improved Mamba model according to claim 7, characterized in that, The block-based linear weighted average strategy is as follows: Weighting function With splicing function Calculate using the following formula: ； ； in, This is the block-by-block output of the model.

Images