Coral reef underwater hyperspectral benthic classification method based on improved Mama model

Abstract

The invention relates to the field of image classification, and discloses a coral reef underwater hyperspectral benthic classification method based on an improved Mama model, and the method comprises the following steps: constructing a coral reef underwater hyperspectral image classification model CRUHI-Mama; the model comprises a pixel-based embedded layer, a space-spectrum Mama backbone network and an adaptive feature-decision fusion module. Acquiring a coral reef underwater hyperspectral image to construct a data set, and training the model; verifying by using a verification set after training; model prediction; and collecting a coral reef underwater hyperspectral image of a to-be-classified region, carrying out blocking, inputting the image into the model, and splicing predicted values output by the model by adopting a blocking linear weighted average strategy to obtain a final classification result. According to the method disclosed by the invention, the training data demand quantity of the model can be reduced, the global spatial features and fine spectral features of the benthic type are efficiently extracted, and method support is provided for fine benthic habitat mapping of the coral reef.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image classification, and particularly relates to a method for classifying benthic types of coral reefs under water based on an improved Mamba model. BACKGROUND

[0002] Coral reef ecosystem is one of the most productive and biodiversity ecosystems on earth. Under the global trend of declining health of coral reefs, the demand for fast, efficient and accurate acquisition of information on coral reef benthic habitats, communities and habitats is more urgent than ever. Over the years, many monitoring methods for coral reefs have been developed and implemented. Among them, the traditional surveyor's in-situ observation cannot efficiently map and monitor large areas of coral reefs. Image-based remote sensing technology has brought revolutionary changes to coral reef monitoring. However, water-based coral reef remote sensing monitoring is affected by weather and water, causing image occlusion and distortion. At the same time, due to the limitation of spatial and spectral resolution, only extensive benthic and topographic classification research can be carried out. In addition, it is also difficult to verify the accuracy of water-based coral reef remote sensing derived maps.

[0003] In the past decade, underwater remote sensing imaging technology has become a mature and reliable coral reef monitoring technology. More and more researchers use annotated high spatial and spectral resolution underwater coral reef images to carry out segmentation and classification, benthic habitat mapping and other research. It is worth noting that the key problem to be solved in these studies is how to accurately and efficiently distinguish coral from other benthic elements in underwater images. In order to accurately and efficiently segment or classify coral or other benthic types in underwater coral reef images. At present, based on underwater RGB images of coral reefs, a large number of benthic type segmentation and classification researches have been carried out using traditional machine learning and classic deep learning methods. Traditional machine learning methods such as support vector machine (SVM) and KNN can only achieve good classification results on small spatial size images, and rely on pre-designed texture and color features. The deep learning segmentation method based on CNN and U-Net model for underwater coral reef RGB images outputs only coarse-grained segmentation results of coral types. The pixel-level classification method based on dense annotation can use DeepLabv3+ and Transformer models to perform dense classification on the whole image. However, due to the limited spectral information of underwater RGB images and the high spatial heterogeneity of benthic types, these methods cannot accurately distinguish all benthic types contained in the image, and the classification accuracy is only between 80-90%. The locality of CNN and the secondary computational complexity also lead to low classification accuracy and efficiency. In addition, these methods all use a large number of labeled samples for training, which requires a lot of time and effort to construct the classification sample database, greatly reducing the utilization efficiency of underwater hyperspectral image data.

[0004] It is worth noting that the underwater environment of coral reefs is complex, and compared with high-spectral images on land, underwater high-spectral images have higher spatial heterogeneity, so a stronger spatial and spectral feature extraction model needs to be designed. At the same time, the uneven distribution of information density of the benthic types of coral reefs easily leads to deviation of the extracted feature information, and how to effectively utilize and fuse these features to improve the classification accuracy of the benthic types is a problem to be solved. SUMMARY

[0005] To solve the above technical problems, the present application provides a coral reef underwater hyperspectral benthic classification method based on an improved Mamba model, so as to efficiently extract global spatial features and fine spectral features of benthic types, while reducing the amount of training data required by the model.

[0006] To achieve the above purpose, the technical scheme of the present application is as follows:

[0007] A coral reef underwater hyperspectral benthic classification method based on an improved Mamba model, comprising the following steps: Step 1, constructing a coral reef underwater hyperspectral image classification model CRUHI-Mamba; The model comprises a pixel-based embedding layer, a spatial-spectral Mamba backbone network, and an adaptive feature-decision fusion module. The pixel-based embedding layer is used to extract the embedding of each pixel, capture nonlinear features, and retain global detail information. The spatial-spectral Mamba backbone network is used to extract discriminative spatial-spectral features, including a double-pooling layer for extracting local significant features and retaining overall information, and a double-branch spatial feature extraction Mamba block and a spectral feature extraction Mamba block connected thereto. The adaptive feature-decision fusion module is used to comprehensively integrate spatial and spectral features, and to perform adaptive fusion at the feature level and the classification result level, and to output the fused prediction value. Step 2, model training and verification; A coral reef underwater hyperspectral image dataset is constructed, and a training set and a verification set are divided, and the model is trained using the training set data; the coral reef underwater hyperspectral image is input into the model after being divided into blocks, and the model is trained using the label corresponding to each block of image data, and the loss is calculated; after training, the verification set is used for verification; Step 3, model prediction; The coral reef underwater hyperspectral image of the area to be classified is collected, divided into blocks, and input into the model, and the prediction value output by the model is spliced using a block linear weighted average strategy to obtain the final classification result.

[0008] In the above scheme, the coral reef underwater hyperspectral image data is represented as wherein H and W are height, width, C represent the spectral dimension; applying principal component analysis to the underwater hyperspectral image data of the coral reef, a cube is obtained V represents the spectral dimension after dimension reduction, H and W the spatial dimension is unchanged; the cube X is taken as the input data of the model; The pixel-based embedding layer takes the cube as input and processes as follows: ; wherein, represents the generated embedding result, and D is the embedding dimension; Conv 2D , GN and SiLU represent a convolution layer with a kernel size of 1*1, a group normalization layer and a SiLU activation function, respectively.

[0009] In the above scheme, the feature extraction process of the spatial-spectral Mamba backbone network is represented as follows: ; wherein, LSSM backbone represents a feature extraction backbone, represents the features output by the LSSM backbone , and j= Pool, Spa, Spe represent a double pooling layer, a spatial feature extraction Mamba block and a spectral feature extraction Mamba block, respectively.

[0010] In the above scheme, the calculation process of the adaptive feature-decision fusion module is as follows: ; wherein, is the fused original output prediction probability value, represents the probability that the pixel at the corresponding position in the image belongs to the kth class, K represents the total number of classes, and AFDFM represents the adaptive feature-decision fusion module.

[0011] In the above scheme, the double pooling layer includes a max pooling layer and an average pooling layer, and the embedding feature F e is input into the max pooling layer for significant feature extraction, and then the average pooling layer is used to integrate global information and suppress noise influence; subsequently, the extracted pooling feature F Pool is input into the spatial feature extraction Mamba block and the spectral feature extraction Mamba block, respectively, to capture the global spatial features and fine spectral features of the coral reef benthic types with linear computational complexity; the forward process is described as follows: ; ; wherein, represents the output feature of the double-pooling layer, and respectively represent the max-pooling and average-pooling; LSSM Spa and LSSM Spe respectively represent the spatial feature extraction Mamba block and the spectral feature extraction Mamba block, respectively represent the extracted spatial feature and spectral feature.

[0012] In the above scheme, the calculation process of the spatial feature extraction Mamba block is as follows: ; wherein, represents the flattened output, ; represents the spatial feature extracted by the standard Mamba; represents the flattening, represents the standard Mamba; , , respectively represent the feature size reshaping, group normalization layer and SiLU activation function; F Spa represents the output spatial feature.

[0013] In the above scheme, the calculation process of the spectral feature extraction Mamba block is as follows: ; wherein, , respectively represent the spectral group output, flattened spectral group feature and Mamba mined spectral feature information; BandGroup m,n is a spectral grouping layer, wherein m is the number of groups, represents the number of channels per group; F Spe represents the output spectral feature.

[0014] In the above scheme, the adaptive feature-decision fusion module first adaptively fuses the spatial feature F Spa and the spectral feature F Spe at the feature level, and introduces a residual connection to relieve overfitting that may occur in the training process, to obtain the fused feature F fusion ; Then, F Spa , F fusion and F SpeThe three features are applied to the classifier to output classification prediction probability values. Finally, at the level of the original prediction values, i.e., the decision level, the importance of the three output prediction probability values ​​is adaptively estimated for decision fusion, realizing the optimized integration of multi-task classification results, and ultimately generating a more robust classification map. The fusion process is represented as follows: ; ; ; ; in, For classifiers, V represents element-wise multiplication; spa and V spe L represents the fusion weights of spatial and spectral features, respectively, both initialized to 0.5 and adaptively updated during model training. Spa L Spe and These are the classification predictions based on the spatial, spectral, and spatial-spectral fusion features of the classifier, respectively; W spa W spe and W F These are the corresponding decision fusion weights, all initialized to 1 / 3; these weights are adaptively updated by the Adam optimizer in each iteration to determine the final fusion weights; This is the fused classification prediction value output by the adaptive feature-decision fusion module.

[0015] In the above scheme, the loss during the model training phase Calculate using the following formula: ; ; Among them, s i and e i These represent the start and end positions of each image data block along the H dimension. N is the number of blocks. is the average height of each image data block, and O is the number of overlapping rows; Each training data block and its corresponding label are represented separately. This is the cross-entropy loss function.

[0016] In the above scheme, the block-based linear weighted average strategy is as follows: Weighting function With splicing function Calculate using the following formula: ; ; in, This is the block-by-block output of the model.

[0017] Through the above technical solution, the underwater hyperspectral benthic classification method for coral reefs based on the improved Mamba model provided by this invention has the following beneficial effects: 1. This invention proposes the CRUHI-Mamba model, which, using less than 2% of the labeled samples, effectively extracts spatial and spectral features that satisfy fine-grained classification of coral reef benthic types by simulating long-distance interactions of the entire image at the pixel level. This results in high-quality coral reef benthic classification results with complete and clear boundaries. This pioneering technology is applied for the first time to underwater hyperspectral image classification of coral reefs, solving the problem that existing methods cannot effectively extract high-quality coral reef features with limited training samples, and significantly improving the accuracy and efficiency of coral reef classification. 2. This invention designs a Spatial-Spectral Mamba Backbone Network (SSMB), which models global spatial dependencies and long-range spectral correlations through dual pooling layers and dual-branch spatial-spectral feature extraction Mamba blocks. This design enhances the model's spatial-spectral feature extraction capability, enabling rapid and efficient extraction of global spatial and fine spectral features of benthic types, thereby significantly improving the accuracy and efficiency of feature extraction and providing higher-quality basic features for subsequent classification tasks. 3. This invention proposes an Adaptive Feature-Decision Fusion Module (AFDFM), which dynamically balances spatial and spectral feature information at the feature level to avoid information forgetting; and coordinates the balanced fusion of multi-task classification results at the decision level, effectively enhancing feature utilization and fusion capabilities, and improving model robustness. In this way, this invention can effectively alleviate feature bias and fusion challenges caused by the imbalance of information density in benthic types, significantly improving the accuracy and reliability of classification. 4. For underwater hyperspectral image classification of coral reefs under high resolution and large data volumes, this invention designs a Chunked Linear Weighted Averaging (CLWA) strategy. This strategy corrects overlapping areas in the classification results of adjacent image blocks by stitching together, effectively reducing stitching errors between a large number of adjacent image block classification results, improving the prediction consistency of boundary categories, solving the stitching error problem caused by image block segmentation in traditional classification methods, and significantly improving the classification accuracy of coral reefs. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0019] Figure 1This is a schematic diagram of the overall framework of a CRUHI-Mamba model disclosed in an embodiment of the present invention; Figure 2 For the adaptive feature-decision fusion module structure, (a) classifier structure, (b) feature-level fusion weight initialization and update, and (c) decision-level fusion weight initialization and update; Figure 3 A schematic diagram of the block-based linear weighted average strategy for model inference; Figure 4 The following are the standard RGB composite true-color images of hyperspectral data and their acquisition locations: (a) shows the location of Curaçao in the Caribbean Sea; (b) shows the location of the Carmabi survey station; and (c) shows three hyperspectral images and their magnified local views. Figure 5 The spectral curve of the average radiation value of the sample; Figure 6 For the benthic classification map of UHI (data number 005), (a) standard RGB composite true-color image of UHI data (data number 005); (b) classification ground truth; (c) RF; (d) RRSN; (e) 3-DCNN; (f) FullyContNet; (g) Spectralformer; (h) SSFTT; (i) GSC-ViT; (j) MambaHSI; (k) CRUHI-Mamba of the present invention; Figure 7 For the benthic classification map of UHI (data number 006), (a) standard RGB composite true-color image of UHI (data number 006); (b) classification ground truth; (c) RF; (d) RRSN; (e) 3-DCNN; (f) FullyContNet; (g) Spectralformer; (h) SSFTT; (i) GSC-ViT; (j) MambaHSI; (k) CRUHI-Mamba of the present invention; Figure 8 For the benthic classification map of UHI (019), (a) standard RGB composite true-color image of UHI (019); (b) classification ground truth; (c) RF; (d) RRSN; (e) 3-DCNN; (f) FullyContNet; (g) Spectralformer; (h) SSFTT; (i) GSC-ViT; (j) MambaHSI; (k) CRUHI-Mamba of the present invention; Figure 9 For the comparison of training parameter performance of different models, (a) is 005 data, (b) is 006 data, and (c) is 019 data. Detailed Implementation

[0020] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0021] This invention provides a method for underwater hyperspectral benthic classification of coral reefs based on an improved Mamba model, comprising the following steps:

[0022] Step 1: Construct the underwater hyperspectral image classification model for coral reefs, CRUHI-Mamba;

[0023] 1. Overall Model Architecture

[0024] The framework of the CRUHI-Mamba model proposed in this invention is as follows: Figure 1 As shown, it is built on a pure Mamba model. The framework consists of three parts: a pixel-based embedding layer; a Spatial-Spectral Mambabackbone (SSMB); and an Adaptive Feature-Decision Fusion (AFDFM) module.

[0025] Underwater hyperspectral image data of coral reefs are represented as ,in H and W It refers to the height and width of the spatial dimension. C Representing spectral dimensions; principal component analysis was applied to underwater hyperspectral image data of coral reefs to obtain a cube. , V This represents the spectral dimension after dimensionality reduction. H and W The spatial dimensions remain unchanged; the cube X As input data for the CRUHI-Mamba model;

[0026] Pixel-based embedding layer: This process uses a cube As input, the pixel-based embedding layer extracts the embedding of each pixel, captures non-linear features, and preserves global detail information. This process can be formulated as follows: ; in, This represents the generated embedding result, where D is the embedding dimension. 2D GroupNorm (GN) and SiLU represent a 1×1 kernel-sized convolutional layer, a grouped normalized layer, and a SiLU activation function, respectively.

[0027] The Spatial-Spectral Mamba backbone (SSMB) is used for fast and efficient extraction of discriminative spatial-spectral features. It mainly consists of a dual-pooling layer for extracting local salient features while preserving overall information, and connected to it a dual-branch Mamba block for spatial feature extraction and a Mamba block for spectral feature extraction. The feature extraction process is shown below: ; Among them, LSSM backbone This represents the main feature extraction framework. Indicates LSSM backbone The output features, j = Pool, Spa, Spe, represent the dual pooling layer, the spatial feature extraction Mamba block, and the spectral feature extraction Mamba block, respectively.

[0028] The Adaptive Feature-Decision Fusion (AFDFM) module comprehensively integrates spatial and spectral features, performing adaptive fusion at both the feature level and the classification result level, and outputting the fused predicted probability values. This provides strong support for the final output of accurate and reliable classification results of coral reef benthic types. This process can be formulated as follows: ; in, The original output predicted probability values ​​are fused. This represents the probability that a pixel at a corresponding location in the image belongs to the k-th class (K represents the total number of classes), and AFDFM represents the adaptive decision fusion module.

[0029] Finally, bilinear interpolation is used to... Adjust the image to the same size as the input image and apply the argmax function to determine the category of each pixel in the image to obtain the classification results of coral reef benthic types. ; .

[0030] 2. Spatial-Spectral Mamba Backbone Network (SSMB)

[0031] Underwater hyperspectral images of coral reefs are characterized by large data volume and high spatial heterogeneity, and the distribution of benthic types is complex. Directly using existing hyperspectral classification methods for benthic classification of coral reefs has the problems of large sample requirements and inability to effectively extract spatial and spectral features that meet the requirements for fine-grained classification of benthic types.

[0032] To address the aforementioned problems, this invention designs an SSMB backbone network comprising a dual-pooling layer and a dual-branch spatial-spectral Mamba feature extraction structure. First, the dual-pooling layer reduces the spatial size of the data and lowers computational complexity while preserving important features. The embedded features F... eThe input is a max pooling layer to extract salient features such as edges and textures. Then, an average pooling layer is used to integrate global information and suppress noise. Finally, the extracted pooling features F... Pool The input consists of two Mamba blocks for spatial and spectral feature extraction, capturing global spatial and fine spectral features of coral reef benthic types with linear computational complexity. This lightweight and concise structure efficiently captures discriminative features. The detailed structure of LSSM is as follows: Figure 1 As shown. The forward process can be described as follows: ; ; in, This represents the output characteristics of the dual pooling layer. and These represent max pooling and average pooling, respectively; LSSM Spa and LSSM Spe These are the Mamba blocks for spatial feature extraction and spectral feature extraction, respectively. These are the extracted spatial features and spectral features, respectively.

[0033] (1) Spatial feature extraction Mamba block (LSSM) Spa Block)

[0034] Considering the complex distribution and strong spatial heterogeneity of benthic types in coral reefs, the entire F... Pool As input, Mamba is used as the basic unit to construct a spatial feature extractor, establishing global pixel-level spatial dependencies with linear computational complexity. LSSM Spa Detailed structure as follows Figure 1 As shown. The forward process can be described as follows: ; in, Indicates the flattened output. ; This represents the spatial features extracted by standard Mamba; To indicate flattening, Represents standard Mamba; , , These represent feature size reshaping, grouped normalization layer, and SiLU activation function, respectively; F Spa This represents the spatial characteristics of the output.

[0035] Studies have shown that Mamba-based classification methods can meet classification performance requirements using only simple flattening methods. This invention employs a row-optimized flattening method, utilizing the spatial features extracted by standard Mamba. The model sequentially performs feature size reshaping, GN layer, and SiLU activation function layer, and uses residual connections to enhance its learning ability. Finally, it outputs spatial features F. Spa .

[0036] (2) Spectral feature extraction Mamba block (LSSM) Spe Block)

[0037] To fully utilize the spectral information in the blue-green spectral regions of underwater hyperspectral images of coral reefs, and to extract fine and separable spectral features, the spectral bands are divided into m groups, as follows: Figure 1 As shown, Mamba is used to model long-range spectral correlations between different spectral groups, and spectral features are updated based on the relationships between the mined spectral groups. LSSM Spe Block's output F Spe It can be obtained from the following formula: ; in, , These represent the spectral grouping output, the flattened spectral group features, and the spectral feature information mined by Mamba, respectively; BandGroup m,n It is a spectral grouping layer, where m is the number of groups. Indicates the number of channels in each group; F Spe This indicates the spectral characteristics of the output. Next, it is compared with LSSM. Spe Similar to the block, the process involves feature size reshaping, GN layers, SiLU activation function layers, and residual connections in sequence. Finally, the LSSM is obtained. Spe The output spectral characteristics of the block F Spe .

[0038] 3. Adaptive Feature-Decision Fusion Module (AFDFM)

[0039] To maximize the utilization of extracted spatial and spectral features without losing beneficial classification information, and to mitigate feature bias and fusion problems caused by uneven information density among benthic species, this invention designs an AFDFM module, such as... Figure 1 As shown. First, in the feature-level adaptive fusion spatial features F Spa and spectral characteristics F Spe To prevent key features from being forgotten, residual connections are introduced to mitigate potential overfitting during training, resulting in the fused feature F. fusion ,like Figure 2 As shown in (b). Next, for F respectively Spa F fusion and F Spe The three features are applied to the classifier, which outputs the classification prediction probability value. The classifier structure is as follows:Figure 2 As shown in (a). Finally, at the level of the original predicted values, i.e. the decision level, the importance of the three output predicted probability values ​​is adaptively estimated for decision fusion, as shown in (a). Figure 2 As shown in (c), the results of multi-task classification are optimized and integrated to generate a more robust classification map. The fusion process is represented as follows: ; ; ; ; in, For classifiers, V represents element-wise multiplication; spa and V spe L represents the fusion weights of spatial and spectral features, respectively, both initialized to 0.5 and adaptively updated during model training. Spa L Spe and These are the classification predictions based on the spatial, spectral, and spatial-spectral fusion features of the classifier, respectively; W spa W spe and W F These are the corresponding decision fusion weights, all initialized to 1 / 3; these weights are adaptively updated by the Adam optimizer in each iteration to determine the final fusion weights; This is the fused classification prediction value output by the adaptive feature-decision fusion module.

[0040] Step two: Model training and validation;

[0041] A dataset was constructed by acquiring underwater hyperspectral images of coral reefs and dividing it into training and validation sets. The model was trained using the training set data. The underwater hyperspectral images of coral reefs were divided into blocks and input into the model. The model was trained using the labels corresponding to each image data block, and the loss was calculated. After training, the model was validated using the validation set.

[0042] CRUHI-Mamba is a pixel-based classification method that takes the entire hyperspectral image as input, trains it, and then infers (validates and tests) to directly output the classification result of the entire image. To balance high efficiency in model training and inference with high accuracy in classification results, this invention addresses the classification of underwater hyperspectral images of coral reefs with high resolution and large data volumes. Training and inference (i.e., image data blocks) are performed by dividing the data into blocks (hyperspectral image data blocks) along the H dimension. Figure 3 (P1, P2, ...).

[0043] Loss during model training Calculate using the following formula: ; ; Among them, s i and e i These represent the start and end positions of each image data block along the H dimension. N is the number of blocks. is the average height of each image data block, and O is the number of overlapping rows; Each training data block and its corresponding label are represented separately. This is the cross-entropy loss function.

[0044] After the block training is completed, when performing inference (verification, testing), it is necessary to process the results of each block of inference (i.e., (The fused classification predictions output by the AFDFM module are concatenated.)

[0045] Step 3, Model Prediction;

[0046] Underwater hyperspectral images of coral reefs in the area to be classified are collected, segmented, and then input into the model. The predicted values ​​output by the model are then stitched together using a segmented linear weighted average strategy to obtain the final classification result.

[0047] To reduce concatenation errors between adjacent data blocks and improve the prediction consistency of boundary categories, this invention designs a Chunked Linear Weighted Averaging (CLWA) strategy for concatenation. Weighting function... With splicing function Calculate using the following formula: ; ; in, This is the block-by-block output of the model.

[0048] in, The weight of the first block is set so that it linearly decreases from 1 to 0.5 from the starting position s0 to the ending position e0. The middle block... The data is divided into two equal parts. The weight of the first part increases from 0.5 to 1, and the weight of the second part decreases from 1 to 0.5. The weight of the last part is set in the opposite way to the first part, increasing linearly from 0.5 to 1. Figure 3 As shown. The block-by-block output of CRUHI-Mamba is compared with the corresponding weight w. i The results are multiplied and normalized. Finally, continuous and accurate predictions of the entire underwater hyperspectral image of the coral reef are obtained. By employing the CLWA strategy, not only were classification accuracy and model efficiency improved, but stitching artifacts were also effectively reduced, ensuring the accuracy and continuity of prediction results.

[0049] Experimental verification:

[0050] 1. Experimental Dataset

[0051] The underwater hyperspectral images of coral reefs were acquired by divers using a hyperspectral imaging system (HyperDiver) at eight coral reef sites around Curaçao in the Caribbean Sea. The spectral range is 400-800 nm, encompassing 400 bands. The dataset can be downloaded from https: / / doi.org / 10.1594 / PANGAEA.911300. This invention uses underwater hyperspectral images of coral reefs numbered 005, 006, and 019 acquired at the Kamab site, see [link to image]. Figure 4 Simultaneously, infrared bands lacking useful signals were discarded, and only raw radiation data from 350 bands in the 400-750nm range were used for coral reef benthic classification, such as... Figure 5 As shown. More relevant information on underwater hyperspectral images of coral reefs and benthic classification experiments is given in Table 1. The categories and sample statistics of the benthic classification experiments of coral reefs are shown in Table 2. The benthic classification system is consistent with that of Chennu (SCHÜRHOLZ D, CHENNU A. Digitizing the coral reef: Machine learning of underwater spectral images enables dense taxonomic mapping of benthic habitats[J]. Methods in Ecology and Evolution,2023,14(2): 596-613.).

[0052] Table 1. Detailed information on underwater hyperspectral image classification experiments for coral reefs.

[0053] Table 2. Benthic types and sample statistics of underwater hyperspectral images of coral reefs, "-" indicates null values;

[0054] 2. Experimental Setup

[0055] The experimental environment for CRUHI-Mamba was implemented using the PyTorch deep learning framework, and all experiments were conducted on the same hardware: a Hygon C86-3G (OPN:3350) CPU @ 3.00 GHz, an NVIDIA GeForce RTX4090 D GPU, and 128GB of RAM. During the experiments, 135 and 30 pixels of data randomly selected from UHI were used as the training and validation sets, respectively, with the remaining samples serving as the test set (Table 1). Since the proposed CRUHI-Mamba takes the entire underwater hyperspectral image of a coral reef as input and outputs the prediction results for the entire image, the batch size was set to 1. The experiments used an Adam optimizer with a learning rate of 0.0001 for iterative training for 200 epochs. The number of overlapping rows O was set to 100 rows. LSSM Spe The number of block spectral groups m was set to 8, and the number of GN groups G was set to 16. The Mamba-SSM version used in the experiment was 1.2.0, and the embedding dimension D was set to 64, where LSSM... Spe The block state dimension "d_state", the width of the local convolution "d_conv", and the expansion factor "expand" are set to 16, 4, and 1, respectively. LSSM Spa The corresponding parameters for the block are set to 16, 4, and 4 respectively. All other Mamba block parameters are set to their default values.

[0056] 3. Comparison Methods and Evaluation Indicators

[0057] To evaluate the classification accuracy and computational cost of CRUHI-Mamba, it was compared with eight other hyperspectral image classification methods. Specifically, these included the classic Random Forest (RF) algorithm, CNN-based 3-DCNN, FullyContNet, SSRN, Transformer-based Spectralformer, SSFTT, and GSC-ViT algorithms, and the Mamba-based MambaHSI algorithm. SSRN is currently a deep learning model applied to underwater hyperspectral image classification. All compared methods were configured with hyperparameters according to the values ​​specified in the original literature and were trained and tested using the same dataset, evaluation criteria, and experimental environment as CRUHI-Mamba proposed in this invention.

[0058] The hyperparameter configurations for each algorithm are described in the original literature: [1] The classic Random Forest (RF) algorithm: HAM J, CHEN Y, CRAWFORD M M, et al. Investigation of the randomforest framework for classification of hyperspectral data[J]. IEEEtransactions on geoscience and remote sensing, 2005,43(3): 492-501; [2] 3-D CNN based on CNN: BEN HAMIDA A, BENOIT A, LAMBERT P, et al. 3-D Deep Learning Approachfor Remote Sensing Image Classification[J]. IEEE Transactions on Geoscienceand Remote Sensing, 2018,56(8): 4420-4434; [3] FullyContNet: WANG D, DU B, ZHANG L. Fully Contextual Network for HyperspectralScene Parsing[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022,60: 1-16; [4] SSRN: ZHONG Z, LI J, LUO Z, et al. Spectral-Spatial Residual Network forHyperspectral Image Classification: A 3-D Deep Learning Framework[J]. IEEEtransactions on geoscience and remote sensing, 2018,56(2): 847-858; [5] Spectralformer based on Transformer: HONG D, HAN Z, YAO J, et al. SpectralFormer: Rethinking HyperspectralImage Classification With Transformers[J]. IEEE Transactions on Geoscienceand Remote Sensing, 2022,60: 1-15; [6]SSFTT: SUN L, ZHAO G, ZHENG Y, et al. Spectral–Spatial Feature TokenizationTransformer for Hyperspectral Image Classification[J]. IEEE Transactions onGeoscience and Remote Sensing, 2022,60({}): 1-14; [7]GSC-ViT algorithm: ZHAO Z, XU X, LI S, et al. Hyperspectral Image Classification UsingGroupwise Separable Convolutional Vision Transformer Network[J]. IEEETransactions on Geoscience and Remote Sensing, 2024,62: 1-17; [8]Mamba-based MambaHSI algorithm: LI Y, LUO Y, ZHANG L, et al. MambaHSI: Spatial–Spectral Mamba forHyperspectral Image Classification[J]. IEEE Transactions on Geoscience andRemote Sensing, 2024,62: 1-16。

[0059] Three evaluation metrics widely used in hyperspectral image classification—overall accuracy (OA), average accuracy (AA), and Kappa coefficient—were employed to quantitatively evaluate the classification model's performance. Furthermore, training and inference time were used to represent time complexity, and the number of training parameters was used to represent space complexity. All experiments were randomly performed five times to eliminate the influence of random sample selection and parameter initialization. Higher mean values ​​for these metrics generally indicate better model performance.

[0060] 4. Experimental Results and Comparative Analysis

[0061] Based on underwater hyperspectral images of coral reefs numbered 005, 006, and 019, benthic classification was performed using the CRUHI-Mamba method proposed in this invention and eight comparison methods. The classification results were comprehensively compared and analyzed from the perspectives of quantitative accuracy evaluation indicators and qualitative visualization results, and the computational complexity and efficiency of the models were also compared.

[0062] 4.1 Quantitative Evaluation of Classification Accuracy

[0063] The proposed CRUHI-Mamba method was comprehensively compared with eight state-of-the-art (SOTA) HSI classification methods from both quantitative and qualitative perspectives for the classification results of underwater coral reef hyperspectral images numbered 005, 006, and 019. The quantitative accuracy evaluation results are shown in Tables 3-5; the qualitative visualization evaluation results are shown in... Figures 6-8 .

[0064] Table 3. Classification accuracy of different methods on UHI 005

[0065] Table 4. Classification accuracy of different methods on UHI 006

[0066] Table 5. Classification accuracy of different methods on UHI 019

[0067] As shown in the quantitative results in Table 3-5, the CRUHI-Mamba model proposed in this invention clearly performs best among all methods. OA, AA, and Kappa scores are all highest on the three underwater hyperspectral images of coral reefs, with OA exceeding 99%, indicating superior classification performance. On the 005 hyperspectral image, only 0.79% of the labeled samples were used to achieve the highest classification accuracy for 12 benthic types, with an OA of 99.56%. Compared to RF, CRUHI-Mamba has an OA more than 8.9% higher, mainly due to the limitations of RF in terms of handcrafted feature descriptors and its inability to utilize spatial feature information. Compared to CNN-based and Transformer-based models, the OA of the model proposed in this invention is on average more than 3.3% and 5.9% higher, respectively. These models suffer from low classification accuracy because their limited receptive fields make it difficult to establish long-term dependencies and they do not fully utilize the detailed spectral interaction information and global spatial information of coral reef benthic types.

[0068] For the classification results of the 006 data (Table 4), this invention selected 0.95% of the samples for training and validation, and the proposed model achieved an accuracy of 99.19%. Compared with the second-place Mamba-based model MambaHSI, OA, AA, and Kappa improved by approximately 1.14%, 0.89%, and 1.48%, respectively. The experiments in the 019 images were performed using 1.91% of the labeled samples. The classification results in Table 5 clearly show that CRUHI-Mamba exhibited the highest accuracy across all three metrics. Compared with CNN-based and Transformer-based models, OA achieved significant improvements of 8.98% and 5.81%, respectively, while Kappa exceeded 10.18% and 6.60%.

[0069] Furthermore, both Mamba-based MambaHSI and the proposed CRUHI-Mamba achieve an overall accuracy (OA) of over 98% (Table 3-5), outperforming other methods. In contrast, other methods show significant variations in classification accuracy across the three hyperspectral images, indicating that the Mamba-based model performs better and is more stable. Firstly, this is due to Mamba's powerful ability to model long-range dependencies with linear complexity, providing strong support for efficient extraction of global spatial and detailed spectral information. Secondly, both the proposed CRUHI-Mamba and MambaHSI are based on the entire image, employing pixel-level spatial and spectral feature extraction structures, which fully leverages the advantages of Mamba to achieve high-precision classification of underwater coral reef hyperspectral images. It should be noted that although the proposed CRUHI-Mamba's OA is only 0.29%–1.14% higher than MambaHSI in the classification of the three hyperspectral images, MambaHSI has a more complex structure and lower efficiency, significantly exceeding CRUHI-Mamba in terms of model training and inference time and model parameters, which will be analyzed in detail in the next section.

[0070] 4.2 Qualitative Assessment of Benthic Classification Map

[0071] from Figures 6-8 The visualized qualitative results show that the classification results of the RF method have significant salt-and-pepper noise. The classification map of FullyContNet based on CNN has the worst visual effect, with large misclassified regions and incomplete types. The classification maps of the Transformer-based models have blurry boundaries and contain a large amount of small-region noise. Among them, the Spectralformer model has serious misclassification problems for coral types such as Meandrina meandrites, Porites astreoides, and Millepora in the underwater hyperspectral image of coral reef number 005. MambaHSI, due to its complex model structure and large computational cost, cannot input the entire hyperspectral image for training and inference with fewer blocks. This results in many stitching artifacts in the classification map, affecting the smoothness of the classification map. In contrast, the proposed CRUHI-Mamba has the fewest misclassifications and is almost completely consistent with the ground truth. It presents high-quality coral reef benthic classification maps with complete, smooth, and clear boundaries with minimal noise. These visualized classification maps further reveal the effectiveness and superiority of the proposed method.

[0072] In summary, compared with state-of-the-art models based on CNN, Transformer and Mamba, and traditional RF, the proposed CRUHI-Mamba achieves the best performance in terms of accuracy and visualization for benthic classification of underwater hyperspectral images of coral reefs.

[0073] 4.3 Comparative Analysis of Computational Complexity and Efficiency

[0074] The advantages of CRUHI-Mamba go beyond accuracy. Figure 9 The training parameters of the model are shown, the training and inference times for the entire image are reported in Table 6, and it is compared with other models on three datasets.

[0075] Table 6 Comparison of Model Training and Inference Times

[0076] 1) In terms of parameters and operational accuracy (OA), CRUHI-Mamba achieved the best results. On the three datasets, the model parameters were reduced by an average of 64%, 49%, and 15% compared to CNN-based, Transformer-based, and Mamba-based models, respectively. This aligns with our design motivation of achieving high classification accuracy with minimal parameters. 2) The CRUHI-Mamba model has the shortest inference time for the entire underwater hyperspectral image of the coral reef, approximately twice as fast as FullyContNet and MambaHSI, both image-level models. Notably, the inference time is reduced by 1 to 3 orders of magnitude compared to other models. A comprehensive analysis of the runtime shows that our model remains competitive in classification efficiency compared to other models. 3) It should be noted that on No. 005 UHI, the training time of MambaHSI is 84 times that of CRUHI-Mamba. This is because No. 005 UHI has a large dataset and the MambaHSI structure is complex, exceeding GPU memory usage and consuming RAM during training, thus reducing data reading and computation efficiency and significantly increasing training time. Even for No. 006 and 019 UHI, which have smaller datasets, the training time is more than 7 times that of CRUHI-Mamba. These experimental results confirm that the CRUHI-Mamba model not only has excellent classification performance but also the advantages of being lightweight and efficient.

[0077] 4.4 Ablation Experiment

[0078] This invention conducted an ablation study to analyze the impact of each module, including the spatial feature extraction Mamba block (LSSM_Spa), the spectral feature extraction Mamba block (LSSM_Spe), the spatial-spectral Mamba feature extraction backbone (SSMB), the direct addition of spatial and spectral features, and the adaptive feature-decision fusion module (AFDFM).

[0079] Table 7. OA accuracy (%) of underwater hyperspectral classification and ablation experiments on coral reefs

[0080] From the ablation experiment results (Table 7), the following conclusions can be drawn: (1) LSSM Spa The block's performance surpasses that of LSSM. Spe The block indicates that spatial features are more discriminative than spectral features; (2) Directly adding spatial and spectral information does not always improve model performance. For example, the performance of No. 019 decreased slightly, which indicates that spatial and spectral features are not entirely complementary for underwater hyperspectral classification of coral reefs. (3) However, by adaptively fusing the output results of different levels through the proposed AFDFM module, spatial and spectral information are fully integrated, thereby improving the classification performance of the model. In particular, for No. 019, the AFDFM module effectively utilizes spatial and spectral information, resulting in a significant improvement in OA, which demonstrates its effectiveness and high robustness.

[0081] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for underwater hyperspectral benthic classification of coral reefs based on an improved Mamba model, characterized in that, Includes the following steps: Step 1: Construct the underwater hyperspectral image classification model for coral reefs, CRUHI-Mamba; The model includes a pixel-based embedding layer, a spatial-spectral Mamba backbone network, and an adaptive feature-decision fusion module; The pixel-based embedding layer is used to extract the embedding of each pixel, capture non-linear features, and retain global detail information; The spatial-spectral Mamba backbone network is used to extract discriminative spatial-spectral features, including a dual pooling layer for extracting local salient features while preserving overall information, and a dual-branch spatial feature extraction Mamba block and a spectral feature extraction Mamba block connected thereto. The adaptive feature-decision fusion module is used to fully integrate spatial and spectral features, perform adaptive fusion at the feature level and classification result level, and output the fused prediction value. Step two: Model training and validation; A dataset was constructed by acquiring underwater hyperspectral images of coral reefs and dividing it into training and validation sets. The model was trained using the training set data. The underwater hyperspectral images of coral reefs were divided into blocks and input into the model. The model was trained using the labels corresponding to each image data block, and the loss was calculated. After training, the model was validated using the validation set. Step 3, Model Prediction; Underwater hyperspectral images of coral reefs in the area to be classified are collected, segmented, and then input into the model. The predicted values ​​output by the model are then stitched together using a segmented linear weighted average strategy to obtain the final classification result.

2. The underwater hyperspectral benthic classification method for coral reefs based on the improved Mamba model according to claim 1, characterized in that, Underwater hyperspectral image data of coral reefs are represented as ,in H and W It refers to the height and width of the spatial dimension. C Representing spectral dimensions; principal component analysis was applied to underwater hyperspectral image data of coral reefs to obtain a cube. , V This represents the spectral dimension after dimensionality reduction. H and W The spatial dimensions remain unchanged; the cube X As input data for the model; The pixel-based embedding layer is in the form of a cube. As input, the following processing is performed: ； in, This represents the generated embedding result. D For the embedding dimension; Conv 2D GN and SiLU represent a 1×1 kernel-sized convolutional layer, a grouped normalized layer, and a SiLU activation function, respectively.

3. The underwater hyperspectral benthic classification method for coral reefs based on the improved Mamba model according to claim 1, characterized in that, The feature extraction process of the spatial-spectral Mamba backbone network is as follows: ； Among them, LSSM backbone This represents the main feature extraction framework. Indicates LSSM backbone Output characteristics j = Pool, Spa, Spe These are a dual-pooling layer, a spatial feature extraction Mamba block, and a spectral feature extraction Mamba block, respectively.

4. The underwater hyperspectral benthic classification method for coral reefs based on the improved Mamba model according to claim 1, characterized in that, The calculation process of the adaptive feature-decision fusion module is as follows: ； in, The original output predicted probability values ​​are fused. This indicates that the pixel at the corresponding position in the image belongs to the first... k The probability of a class K This represents the total number of categories, and AFDFM represents the Adaptive Feature Decision Fusion Module.

5. A method for underwater hyperspectral benthic classification of coral reefs based on an improved Mamba model, as described in claim 1, is characterized in that... The dual pooling layer includes a max pooling layer and an average pooling layer, which embeds features. F e The input is a max pooling layer for salient feature extraction, followed by an average pooling layer to integrate global information and suppress noise. Then, the extracted pooling features are... F Pool The spatial feature extraction Mamba block and the spectral feature extraction Mamba block are input separately to capture global spatial features and fine spectral features of coral reef benthic types with linear computational complexity. The forward process is described below: ； ； in, This represents the output characteristics of the dual pooling layer. and These represent max pooling and average pooling, respectively; LSSM Spa and LSSM Spe These are the Mamba blocks for spatial feature extraction and spectral feature extraction, respectively. These are the extracted spatial features and spectral features, respectively.

6. A method for underwater hyperspectral benthic classification of coral reefs based on an improved Mamba model, as described in claim 5, is characterized in that... The calculation process for extracting the spatial features from the Mamba block is as follows: ； in, Indicates the flattened output. ; This represents the spatial features extracted by standard Mamba; To indicate flattening, Represents standard Mamba; , , These represent feature size reshaping, grouping normalization layer, and SiLU activation function, respectively. F Spa This represents the spatial characteristics of the output.

7. A method for underwater hyperspectral benthic classification of coral reefs based on an improved Mamba model, as described in claim 5, is characterized in that... The calculation process for the Mamba block for spectral feature extraction is as follows: ； in, , These represent the spectral grouping output, the flattened spectral group features, and the spectral feature information mined by Mamba, respectively; BandGroup m,n It is a spectral grouping layer, in which m It is the number of groups. Indicates the number of channels in each group; F Spe This indicates the spectral characteristics of the output.

8. A method for underwater hyperspectral benthic classification of coral reefs based on an improved Mamba model, as described in claim 7, is characterized in that... The adaptive feature-decision fusion module first adaptively fuses spatial features at the feature level. F Spa and spectral characteristics F Spe Furthermore, residual connections are introduced to mitigate potential overfitting during training, resulting in fused features. F fusion Then, respectively for F Spa , F fusion and F Spe The three features are applied to the classifier to output classification prediction probability values. Finally, at the level of the original prediction values, i.e., the decision level, the importance of the three output prediction probability values ​​is adaptively estimated for decision fusion, realizing the optimized integration of multi-task classification results, and ultimately generating a more robust classification map. The fusion process is represented as follows: ； ； ； ； in, For classifiers, This represents element-wise multiplication; V spa and V spe These represent the fusion weights for spatial and spectral features, respectively. Both are initialized to 0.5 and are adaptively updated during model training. L Spa , L Spe and These are the classification predictions based on the spatial, spectral, and spatial-spectral fusion features of the classifier, respectively. W spa , W spe and W F These are the corresponding decision fusion weights, all initialized to 1 / 3; these weights are adaptively updated by the Adam optimizer in each iteration to determine the final fusion weights; This is the fused classification prediction value output by the adaptive feature-decision fusion module.

9. A method for underwater hyperspectral benthic classification of coral reefs based on an improved Mamba model according to claim 2, characterized in that, Loss during model training Calculate using the following formula: ； ； in, s i and e i For each image data block H Start and end positions in the dimension , N The number of blocks, The average height of each image data block, O Number of overlapping rows; Each training data block and its corresponding label are represented separately. This is the cross-entropy loss function.

10. A method for underwater hyperspectral benthic classification of coral reefs based on an improved Mamba model according to claim 9, characterized in that, The block-based linear weighted average strategy is as follows: Weighting function With splicing function Calculate using the following formula: ； ； in, This is the block-by-block output of the model.

Images