An image segmentation method based on medical hyperspectral image segmentation network

Abstract

The present application is applicable to the technical field of medical hyperspectral image processing, and discloses an image segmentation method based on a medical hyperspectral image segmentation network: obtaining a medical hyperspectral image sample set, the medical hyperspectral image sample set includes a first image sample set, a second image sample set and a spectral slice; constructing a medical hyperspectral image segmentation network, the medical hyperspectral image segmentation network includes a dual encoder, a jump learning component, a decoder, a first segmentation head, a spatial geometry refinement component and a second segmentation head, the dual encoder includes a spectral encoder and a spatial encoder; constructing a total loss function to optimize the medical hyperspectral image segmentation network to obtain a medical hyperspectral image segmentation optimization network; obtaining the final image segmentation result based on the medical hyperspectral image segmentation optimization network. The method can obtain higher image segmentation accuracy by improving the ability to perceive subtle structures in complex medical hyperspectral images.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical hyperspectral image processing, and in particular relates to an image segmentation method based on a medical hyperspectral image segmentation network. Background Art

[0002] Medical image segmentation involves multiple imaging modalities, including grayscale techniques such as X-rays and CT scans, and RGB imaging for endoscopes and pathology microscopes. Although these traditional techniques have achieved important results in medical applications, they are still limited in capturing the complete spectral information of biological tissues. To address this problem, researchers are increasingly using hyperspectral imaging (HSI). Medical hyperspectral imaging (MHSI) breaks through the three-color limitation of the RGB system and can capture a continuous spectrum from visible light to near-infrared, generating a three-dimensional data cube containing high-resolution spectral features. MHSI can distinguish normal from abnormal tissues through the intensity changes of individual pixels in different spectral bands, showing the potential for enhanced tissue characterization. MHSI has significant advantages. It enables non-invasive, label-free tissue examination, reducing the need for contrast agents and invasive biopsies. The rich spectral information can detect and characterize tiny tissue abnormalities that cannot be identified by conventional imaging. In addition, the multidimensional data of MHSI is suitable for complex computational analysis, which can extract complex patterns and biomarkers and improve the accuracy of diagnosis and prognosis. Experimental results show that MHSI is superior to traditional RGB imaging in pathological image segmentation, so it has broad application prospects.

[0003] In recent years, a new two-stream architecture design has been developed in this field, aiming to handle spatial and spectral dimensions independently. This design extracts and learns corresponding representations through different structural components, thereby capturing spatial and spectral information more effectively while reducing the overall network complexity. However, these methods usually rely on simple convolutional layers and linear layers for spectral feature extraction, which may not fully exploit the complex spectral relationships in hyperspectral images. This simplified approach may ignore or weaken important spectral information, thus affecting the overall effectiveness and performance of the model. A recent study proposed a new module that uses an efficient matrix factorization method to learn multi-granular spectral features. This innovative approach maintains computational efficiency while capturing and exploiting complex spectral information, thereby significantly improving the performance of hyperspectral image processing tasks. Despite these improvements, the integration of spatial and spectral features is still relatively rudimentary, usually relying on simply concatenating the two modalities. This oversimplified approach ignores the intrinsic relationship between spatial structure and spectral response, as well as the dependencies at different scales and levels. This simplification may weaken the model's ability to handle complex spatial-spectral structured targets, especially in scenarios where it is necessary to accurately distinguish similar spectral features but with different spatial distributions.

[0004] Meanwhile, U-Net architectures have become the dominant research paradigm, demonstrating excellent performance and diversity. The efficiency of these architectures stems from their unique encoder-decoder structure and skip connection mechanism, which can integrate multi-scale feature information, allowing the U-Net to capture global semantic context while retaining local details. This is crucial for accurately segmenting complex anatomical structures and pathological features in medical images. However, despite its widespread success, the U-Net architecture still faces some inherent limitations that deserve further exploration. These limitations are mainly reflected in its adaptability to heterogeneous data, the effectiveness of feature representation and integration, and the utilization of geometric information. Traditional U-Net models often show insufficient adaptation in dealing with the high variability inherent in medical images, especially in multimodal inputs or diverse pathological conditions. Although the skip connection mechanism is effective in information propagation, its design is relatively simple and fails to fully utilize the complex relationship between features at different scales, which may lead to suboptimal feature fusion and selective propagation, thereby affecting the model's ability to perceive subtle structures in complex medical images. In addition, the insufficient utilization of geometric information in existing methods also poses significant challenges. The lack of an effective mechanism to incorporate geometric prior knowledge may lead to poor performance in boundary localization and shape preservation, especially when dealing with pathological structures with high variability or irregular morphology. This limitation affects the segmentation accuracy and increases the possibility of false positive or false negative results in images with blurred boundaries or complex backgrounds.

[0005] Therefore, most of the existing technical methods have the following problems: 1. Insufficient use of spectral information: Hyperspectral images contain dozens or hundreds of spectral bands, with a huge amount of information. How to efficiently extract and utilize this spectral information is a major challenge. Traditional methods rely on simple convolutional layers and linear layers for spectral feature extraction, which may not fully utilize the complex spectral relationships in hyperspectral images. 2. Low efficiency of fusion of spectral and spatial information: The integration method of spatial and spectral features is still relatively simple, usually relying only on simply splicing the two modalities. This oversimplified approach ignores the intrinsic association between spatial structure and spectral response, as well as the dependencies at different scales and levels. 3. Insufficient use of geometric information. The lack of an effective mechanism to integrate geometric prior knowledge, especially when dealing with pathological structures with high variability or irregular morphology, may lead to poor performance in boundary positioning and shape retention. As a result, it is difficult to improve the perception of subtle structures in complex medical hyperspectral images, making the accuracy of medical image segmentation still challenging. Summary of the invention

[0006] Based on this, this application proposes an image segmentation method based on a medical hyperspectral image segmentation network, aiming to achieve higher image segmentation accuracy by improving the ability to perceive subtle structures in complex medical hyperspectral images.

[0007] The first aspect of the present application provides an image segmentation method based on a medical hyperspectral image segmentation network, the method comprising:

[0008] Acquire a medical hyperspectral image sample set, wherein the medical hyperspectral image sample set includes a first image sample set, a second image sample set, and a spectral slice;

[0009] A medical hyperspectral image segmentation network is constructed, wherein the medical hyperspectral image segmentation network includes a dual encoder, a jump learning component, a decoder, a first segmentation head, a spatial geometry refinement component, and a second segmentation head, wherein the dual encoder includes a spectral encoder and a spatial encoder, wherein:

[0010] The spectral encoder is used to obtain spectral features according to the first image sample set,

[0011] The spatial encoder is used to obtain spatial features according to the second image sample set,

[0012] The jump learning component is used to obtain a first spectral spatial feature according to the spectral feature and the spatial feature,

[0013] The decoder is used to obtain a second spectral spatial feature according to the first spectral spatial feature,

[0014] The first segmentation head is used to obtain an initial segmentation mask according to the second spectral spatial feature,

[0015] The spatial geometry refinement component is used to obtain a refined feature according to the spectral slice, the second spectral spatial feature and the initial segmentation mask,

[0016] The second segmentation head is used to obtain a refined segmentation mask according to the refined features;

[0017] Constructing a total loss function to optimize the medical hyperspectral image segmentation network to obtain a medical hyperspectral image segmentation optimization network;

[0018] The final image segmentation result is obtained based on the medical hyperspectral image segmentation optimization network.

[0019] As an optional implementation of the first aspect, the step of acquiring a medical hyperspectral image sample set, wherein the medical hyperspectral image sample set includes a first image sample set, a second image sample set, and a spectral slice includes:

[0020] Get a sample set of medical hyperspectral images , divide X into the first image sample set , the second image sample set and spectral slices , and then input three branches respectively;

[0021] Among them, the first image sample set For input to the spectral encoder, the second image sample set For input to spatial encoder, spectral slice Used to refine the input spatial geometry components.

[0022] As an optional implementation manner of the first aspect, the spectral encoder includes multiple levels of spectral Mamba enhancement layers, each spectral Mamba enhancement layer includes a spectral Mamba enhancement module, the spectral Mamba enhancement module includes a serialization block and a spectral Mamba enhancement block, and the processing process of the spectral Mamba enhancement module includes:

[0023] The first image sample set Subset features of Input the serialized block and convert it into a feature sequence , the feature sequence The coding sequence is obtained by position encoding ;

[0024] The coding sequence Input the spectral Mamba enhancement block to get the feature enhancement sequence ;

[0025] The feature enhancement sequence Perform feature reconstruction to obtain a three-dimensional tensor .

[0026] As an optional implementation manner of the first aspect, the first image sample set Subset features of Input the serialized block and convert it into a feature sequence , the feature sequence The coding sequence is obtained by position encoding The steps include:

[0027] Subset Features Adjust the shape and perform spectral dimension mapping to obtain ;

[0028] Will Divide into non-overlapping blocks, each denoted by ,in Indicates the size of the block;

[0029] Block Linear projection to the preset dimension to generate feature sequence ,definition , represents the spectral mapping function;

[0030] Encode the position Add to feature sequence The coding sequence was obtained .

[0031] As an optional implementation manner of the first aspect, the spectral Mamba enhancement block includes a Mamba block and a spectral channel attention block, wherein the processing process of the Mamba block is expressed as follows:

[0032] ,

[0033] in, and Represents the input and output tags of the Mamba block, Represents the mapping function of the Mamba block;

[0034] The processing process of the spectral channel attention block is expressed as:

[0035] Input Tag , using compression operations to aggregate features and applying global average pooling to obtain the spectral vector , and then through one-dimensional convolution projection, we get ;

[0036] Introducing a spectral memory unit, which includes a global memory bank ,from Retrieve the most relevant low-rank spectral vector from To adjust ,

[0037] Through one-dimensional convolution Scaling is performed to generate low-rank constrained spectral attention weights, and the spectral attention weights are combined with the input Multiply element by element, and the final output is .

[0038] As an optional implementation of the first aspect, the jump learning component includes a dual-scale channel attention layer, a dual-scale spatial attention layer, and a residual connection layer, and the step of obtaining a first spectral spatial feature according to the spectral feature and the spatial feature includes:

[0039] The spectral feature and the spatial feature are fused to obtain a spectral spatial fusion feature. ;

[0040] Generate channel attention weights based on the dual-scale channel attention layer , by element-wise multiplication Applied to spectral spatial fusion features , to obtain ;

[0041] Generate spatial attention weights from the dual-scale spatial attention layer , by element-wise multiplication Application , to get ;

[0042] According to the residual connection layer, the spectral space fusion features and Combined, the first spectral spatial feature is obtained .

[0043] As an optional implementation manner of the first aspect, the step of obtaining a refined feature according to the spectral slice, the second spectral spatial feature and the initial segmentation mask includes:

[0044] Determine the initial segmentation mask Relative to spectral slice The geodesic distance , the geodesic distance Normalize and obtain the geometric prior graph G;

[0045] The second spectral spatial feature is fused through the output of the first hybrid convolution unit and the geometric prior graph G through the output of the second hybrid convolution unit to obtain a refined fusion feature. ;

[0046] Introducing expansion feature aggregation to refine fusion features Enhance key features and introduce residual connections to obtain refined features .

[0047] As an optional implementation manner of the first aspect, the total loss function is ,in, and is the weight coefficient that balances the contribution of initial segmentation and final fine segmentation loss, and Represent the loss terms of initial segmentation and final refined segmentation respectively;

[0048] ,in ,

[0049] represents the Dice similarity coefficient loss, represents the binary cross entropy loss;

[0050] ,

[0051] ,

[0052] in, and Represent the predicted probability map and the true label respectively, is the total number of pixels.

[0053] A second aspect of an embodiment of the present application provides a computer device, which includes a memory, a processor, and a processing program stored in the memory and executable on the processor, wherein the processing program implements the above-mentioned image segmentation method when executed by the processor.

[0054] A third aspect of an embodiment of the present application provides a storage medium, on which a processing program is stored. When the processing program is executed by a processor, the above-mentioned image segmentation method is executed.

[0055] Compared with the prior art, the technical effects of the present invention include: 1. Dual encoder structure and spectral-spatial information fusion: A dual encoder structure is introduced, which is used to extract spectral and spatial information respectively. Through a carefully designed feature fusion mechanism, the organic combination of the corresponding layers of the spectral encoder and the spatial encoder is realized, making full use of the heterogeneity of MHSI data and effectively improving the segmentation performance. 2. Improved jump learning module and adaptive feature enhancement: On the basis of the traditional U-shaped network jump connection, the dual-scale channel attention and dual-scale spatial attention are integrated into the introduced dual-scale convolutional block attention module for multi-scale feature selection and enhancement, which greatly improves the information flow efficiency and optimizes the segmentation model's use of complex information. 3. Geometric optimization and boundary refinement: By introducing geometric prior knowledge in the spatial geometry refinement component, especially for complex anatomical structures and irregular lesions, the boundary accuracy and shape fidelity of the segmentation results are significantly improved, ensuring the high-quality output of the segmentation results.

[0056] Additional aspects and advantages of the present application will be given in part in the following description, and in part will become apparent from the following description, or will be understood through the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A flowchart of an image segmentation method based on a medical hyperspectral image segmentation network proposed in the first embodiment of the present application;

[0058] Figure 2 This is a diagram showing the medical hyperspectral image segmentation network structure framework proposed in the first embodiment of the present application;

[0059] Figure 3 This is a network architecture diagram of the spectral Mamba enhancement module and its submodules in the first embodiment of the present application;

[0060] Figure 4This is a framework diagram of the jump learning component in the first embodiment of the present application;

[0061] Figure 5 This is a network structure diagram of dual-scale channel attention in the first embodiment of this application;

[0062] Figure 6 This is a network structure diagram of dual-scale spatial attention in the first embodiment of the present application;

[0063] Figure 7 A single spectral band slice, segmentation mask, Euclidean distance map and geodesic distance map of MHSI in the first embodiment of the present application;

[0064] Figure 8 This is a flowchart of an experimental comparison implementation of an image segmentation method based on a medical hyperspectral image segmentation network proposed in the second embodiment of the present application;

[0065] Fig. 9 Three sample examples from the MDC dataset in the second implementation of this application are shown in the figure;

[0066] Fig.10 This is a diagram showing basic information and sample examples of five organs selected from the HeiPorSPECTRAL dataset in the second implementation of this application;

[0067] Fig.11 A visualization result display diagram of the original output features of the corresponding layers of the medical hyperspectral image segmentation network proposed in the second implementation of the present application and the fused features;

[0068] Fig.12 The figure shows the segmentation results of the existing method and the present method on 10 representative samples in the second implementation of the present application;

[0069] Fig.13 This is a schematic diagram of the structure of an image segmentation system based on a medical hyperspectral image segmentation network proposed in the third embodiment of the present application.

[0070] The following specific implementation methods will further illustrate the present application in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0072] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here. In addition, the "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated with each other are in an "or" relationship.

[0073] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.

[0074] See also Figure 1 , is a flow chart of an image segmentation method based on a medical hyperspectral image segmentation network proposed in the first embodiment of the present application. Figure 2 As shown, this is the network structure framework diagram of the medical hyperspectral image segmentation proposed in this application.

[0075] In order to facilitate the understanding of this solution, we can describe the problem in mathematical terms before implementing the steps: represents the three-dimensional volume of MHSI, where represents the number of spectral bands, Represents the spatial resolution; assuming that the Ground Truth (dataset used for segmentation training) segmentation mask is , where 1 represents foreground and 0 represents background; the target segmentation task of this scheme is to predict pixel-level annotation masks , so that Very close .

[0076] The proposed method includes steps S01 to S04.

[0077] S01: Acquire a medical hyperspectral image sample set, wherein the medical hyperspectral image sample set includes a first image sample set, a second image sample set, and spectral slices.

[0078] Specifically, obtain a medical hyperspectral image sample set , divide X into the first image sample set , the second image sample set and spectral slices , and then input three branches respectively;

[0079] Among them, the first image sample set For input to the spectral encoder, the second image sample set For input to spatial encoder, spectral slice Used to refine the input spatial geometry components.

[0080] For example, the input medical hyperspectral image is divided into multiple streams: Spectral encoder input , spatial encoder input , and spatial geometry refinement component input ,in represents the number of groups the spectral bands are evenly divided into, and the first spectral band of MHSI is selected as the input for geometry refinement.

[0081] S02: construct a medical hyperspectral image segmentation network, wherein the medical hyperspectral image segmentation network includes a dual encoder, a jump learning component, a decoder, a first segmentation head, a spatial geometry refinement component, and a second segmentation head, wherein the dual encoder includes a spectral encoder and a spatial encoder, wherein:

[0082] The spectral encoder is used to obtain spectral features according to the first image sample set,

[0083] The spatial encoder is used to obtain spatial features according to the second image sample set,

[0084] The jump learning component is used to obtain a first spectral spatial feature according to the spectral feature and the spatial feature,

[0085] The decoder is used to obtain a second spectral spatial feature according to the first spectral spatial feature,

[0086] The first segmentation head is used to obtain an initial segmentation mask according to the second spectral spatial feature,

[0087] The spatial geometry refinement component is used to obtain a refined feature according to the spectral slice, the second spectral spatial feature and the initial segmentation mask,

[0088] The second segmentation head is used to obtain a refined segmentation mask according to the refined features.

[0089] In the dual encoder architecture, which consists of a spectral encoder and a spatial encoder, it aims to decouple spectral and spatial information in the learning process. Specifically, the spectral encoder focuses on capturing subtle changes and features in high-dimensional spectral data, while the spatial encoder is dedicated to extracting spatial structures and contextual information in images. Through an efficient fusion mechanism, these complementary features are integrated together to generate a more comprehensive and robust target representation.

[0090] In the spectral encoder, such as Figure 2 As shown, in this embodiment, the spectral encoder includes four levels of spectral Mamba enhancement layers, each spectral Mamba enhancement layer (SME) includes a spectral Mamba enhancement module (SMEM), and the spectral Mamba enhancement module includes a serialization block and a spectral Mamba enhancement block (SMEB).

[0091] It should be noted that Transformer performs well in sequence learning tasks, but its quadratic computational complexity reduces efficiency when processing long sequences. To solve this problem, this application uses the more advanced Mamba architecture to replace Transformer. While maintaining powerful sequence modeling capabilities, the Mamba architecture reduces computational complexity to a linear level, significantly improving computational efficiency without sacrificing performance. Based on the Mamba architecture, this application proposes a spectral Mamba enhancement layer (SME).

[0092] Taking the first spectral Mamba enhancement layer as an example, input First, the spatial resolution is reduced by spatial downsampling. This step is implemented by depthwise separable convolution, and the number of convolution groups is equal to the number of input channels, as shown in formula (1). The downsampled features are then passed through the Spectral Mamba Enhancement Module (SMEM) and the intermediate output is generated through residual connection, as shown in formula (2). Finally, the features are further processed by the convolution block and the final output is obtained through another residual connection, as shown in formula (3). This design improves the expressiveness of the features and ensures the effective transmission of information.

[0093]

[0094]

[0095]

[0096] in, represents a depth-wise separable convolution operation, It is the Spectrum Mamba Enhancement Module Operation, Represents a normal convolution operation, and F represents Subset features for depthwise separable convolution , Represents a three-dimensional volume tensor , Represents the output of a normal convolution operation.

[0097] Furthermore, if Figure 3 This is the network architecture diagram of the Spectrum Mamba enhancement module and its submodules. Figure 3 (a) in the figure is the overall framework diagram of the spectral Mamba enhancement module (SMEM). The spectral Mamba enhancement module (SMEM) includes a serialization block and a spectral Mamba enhancement block (SMEB). The spectral Mamba enhancement block (SMEB) includes a Mamba block and a spectral channel attention block. Figure 3(b) is the overall framework diagram of the Mamba block. Figure 3 (c) is the overall framework diagram of the serialization block. Figure 3 (d) in FIG. 1 is an overall framework diagram of the spectral channel attention block; the processing process of the spectral Mamba enhancement module (SMEM) includes steps S0211 to S0213.

[0098] S0211: The first image sample set Subset features of Input the serialized block and convert it into a feature sequence , the feature sequence The coding sequence is obtained by position encoding ;

[0099] In the serialization block, the data processing process includes:

[0100] Subset Features Adjust the shape and perform spectral dimension mapping to obtain ;

[0101] Will Divide into non-overlapping blocks, each denoted by ,in Indicates the size of the block;

[0102] Block Linear projection to the preset dimension to generate feature sequence ,definition , represents the spectral mapping function;

[0103] Encode the position Add to feature sequence The coding sequence was obtained .

[0104] Specifically, in order to fully utilize the sequence modeling capabilities of Mamba, this application designs an input preprocessing pipeline for generating spectral sequences. This process includes three main steps: spectral dimension mapping, spatial partitioning, and dimension mapping.

[0105] First, input subset features is reshaped and spectrally mapped, producing ,in represents the transition dimension of the spectral mapping process, which can be expressed as:

[0106]

[0107] in Represents the spectral mapping function, a simple multi-layer perceptron implementation, Represents a reshape operation.

[0108] then, Divided into space non-overlapping blocks ,in Indicates the size of the block.

[0109] Finally, these blocks are linearly projected to the specified dimension to generate the input tokens , which is defined as Considering the importance of position information in accurate sequence modeling, position encoding Added to middle:

[0110]

[0111] This preprocessing process provides structured input to the Spectral Mamba enhancement block while retaining key information of the original data, thereby maximizing its advantages in sequence modeling.

[0112] S0212: The coding sequence Input the spectral Mamba enhancement block to get the feature enhancement sequence ;

[0113] In the spectral Mamba enhancement block, the spectral Mamba enhancement block (SMEB) includes a Mamba block and a spectral channel attention block.

[0114] The processing of Mamba blocks is expressed as:

[0115] ,

[0116] in, and Represents the input and output tags of the Mamba block, Represents the mapping function of the Mamba block;

[0117] The processing of the spectral channel attention block is expressed as:

[0118] Input Tag , using compression operations to aggregate features and applying global average pooling to obtain the spectral vector , and then through one-dimensional convolution projection, we get ;

[0119] Introducing a spectral memory unit, which includes a global memory bank ,from Retrieve the most relevant low-rank spectral vector from To adjust ,

[0120] Through one-dimensional convolution Scaling is performed to generate low-rank constrained spectral attention weights, and the spectral attention weights are combined with the input Multiply element by element, and the final output is .

[0121] It can be understood that the spectral Mamba enhancement block (SMEB) is the core component of spectral feature extraction, which contains the Mamba block and the spectral channel attention block. The Mamba block mainly captures the long-range dependencies in the spectral sequence, while the spectral channel attention block enhances the representation ability of key spectral channels. The iterative application of SMEB gradually refines and enhances the spectral features, building a richer and more discriminative feature representation.

[0122] In the In a SMEB, the processing of the Mamba block is expressed as:

[0123]

[0124] in, and Respectively represent The input and output tags of each block, Represents the mapping function for a Mamba block.

[0125] In view of the inherent low-rank characteristics of the spectral dimension of medical hyperspectral imaging, this application introduces a memory unit mechanism in the spectral channel attention block. Inspired by spectral matrix decomposition and low-rank decomposition and memory units for storing low-rank statistical information of hyperspectral image cubes, this design aims to capture and utilize the low-rank structure of spectral data and enhance the model's sensitivity to key spectral features.

[0126] After the Mamba block feature extraction, the obtained spectral tokens are passed to the spectral channel attention block, and the input tokens ,in Corresponding to the number of blocks after the cube feature is divided, it aims to explore the correlation between spectral bands. The compression operation is used to aggregate the features, and the global average pooling is applied to obtain the aggregated spectral vector . This vector is then projected onto :

[0127]

[0128] in, represents a one-dimensional convolution operation, Represents a global average pooling operation.

[0129] In order to deeply mine the spectral correlation and enhance the expression of low-rank spectral vectors, a spectral memory unit is introduced. This unit maintains a global memory bank , as a learnable parameter. For the spectral vector ,from Retrieve the most relevant low-rank spectral vector from . and The correlation coefficient between the low-rank vectors stored in The calculation method is . Through the correlation coefficient matrix ,from The required low-rank vector is retrieved from :

[0130]

[0131] Finally, through one-dimensional convolution Rescale to generate low-rank constrained spectral attention weights. This attention weight is consistent with the input Multiply element by element to produce the final output :

[0132]

[0133] where ⊙ represents the Hadamard product.

[0134] S0213: Enhance the feature sequence Perform feature reconstruction to obtain a three-dimensional tensor .

[0135] The above are all the contents that need to be explained in the spectral encoder part.

[0136] In addition, between the spectral encoder and the spatial encoder, the features extracted by the spectral encoder interact and fuse with the output of the spatial encoder through a specially designed fusion module. The fusion module is designed to facilitate the information exchange between spectral and spatial features and adopts a complex strategy. After the features are concatenated, the module uses parallel average pooling and maximum pooling paths for channel selection to fully capture feature statistics. This design ensures that key information from both encoders is preserved, while complementing and enhancing the model's overall understanding of MHSI data.

[0137] In the jump learning component, e.g. Figure 4 Shown is a framework diagram of the skip learning component, which includes a dual-scale channel attention (DSCA) layer, a dual-scale spatial attention (DSSA) layer, and a residual connection layer.

[0138] It should be noted that the jump learning component effectively promotes the multi-scale integration and transmission of feature information by establishing connections between the encoder and the decoder. To further optimize this process, the dual-scale convolutional block attention module (DS-CBAM) is introduced, which is an advanced extension of the classic CBAM. This module is designed to address the unique challenges in medical imaging, such as geometric diversity, boundary ambiguity, and scale variability. DS-CBAM uses a dual-scale attention mechanism to capture both macroscopic and microscopic features, thereby enhancing the model's understanding and representation of complex medical images. This improvement enables the model to accurately locate and segment lesions of different sizes and shapes, thereby improving the accuracy and reliability of diagnosis.

[0139] The step of obtaining a first spectral spatial feature according to the spectral feature and the spatial feature comprises:

[0140] S0221: Fusing the spectral feature and the spatial feature to obtain a spectral spatial fusion feature ;

[0141] S0222: Generate channel attention weights based on dual-scale channel attention layer , by element-wise multiplication Applied to spectral spatial fusion features , to obtain ;

[0142] S0223: Generate spatial attention weights from a dual-scale spatial attention layer , by element-wise multiplication Application , to get .

[0143] like Figure 5 and Figure 6 As shown in the figure, the network structure diagrams of dual-scale channel attention (DSCA) and dual-scale spatial attention (DSSA) are respectively. Dual-scale channel attention (DSCA) and dual-scale spatial attention (DSSA) are designed to decompose the attention mechanism into macro and micro levels, thereby achieving comprehensive multi-scale feature capture. In each attention layer, the upper branch is responsible for macro feature learning, while the lower branch focuses on micro feature extraction. The macro branch retains global feature information through maximum pooling and average pooling operations, and then learns global attention of channels and spaces through a series of convolutions. This method is inspired by the design of traditional CBAM and effectively captures large-scale structures and global semantic information, but may ignore local details.

[0144] In the micro-branch, fine-grained features are gradually extracted through continuous depth-wise convolution. For channel attention, the input features Generated by multiple layers of deep convolution :

[0145]

[0146] in, represents the weight of the nth layer of depth convolution, represents the convolution operation. The micro-branch of spatial attention is applied to , add point convolution (1×1 convolution) after each depth convolution to reduce the channel dimension. The final output The calculation is:

[0147]

[0148] in, Represents the weight of the n-th layer of point convolution.

[0149] It is worth noting that the DSSA layer contains two key optimizations. In the macro branch, parallel convolution operations with different convolution kernel sizes are used to enhance the model's perception of multi-scale features and address the complexity of medical images. Second, in order to capture a wider range of contextual information, this application introduces a dilated feature aggregation (DFA) module at the end of the DSSA layer. DFA is implemented through four different dilation rates ( ), effectively expanding the receptive field. Feature fusion and dimensionality reduction are achieved through channel-level concatenation followed by 1×1 convolution, which significantly improves the model's ability to capture long-range dependencies while maintaining computational efficiency.

[0150] Therefore, the designed dual-scale attention mechanism enables DS-CBAM to focus on global semantic information and local detail features at the same time, significantly improving the performance of the model in complex medical image analysis tasks.

[0151] S0224: Introduce residual connection to fuse spectral and spatial features and Combined, the first spectral spatial feature is obtained .

[0152] In the spatial geometry refinement component, it should be noted that the segmentation mask is refined by geometric priors, which significantly improves the accuracy and robustness of the segmentation results, especially when dealing with complex medical hyperspectral images. To achieve this goal, this application proposes a spatial geometry refinement component, which combines spatial geometry information with semantic features to accurately depict segmentation boundaries and preserve complex details.

[0153] The core of the spatial geometry refinement component lies in the SGR module, which uses the geodesic distance as a key geometric prior. The geodesic distance not only covers spatial proximity, but also includes local contextual information (such as intensity similarity), and performs well in distinguishing neighboring pixels that are similar in appearance but belong to different semantic categories. This property is particularly important in medical image segmentation, where accurate depiction of anatomical structures is crucial. To demonstrate the effectiveness of the geodesic distance, Figure 7 A comparative analysis was conducted in Figure 7, where (a) in Figure 7 is a single spectral band slice of MHSI, (b) in Figure 7 is a segmentation mask, (c) in Figure 7 is a Euclidean distance map, and (d) in Figure 7 is a geodesic distance map. The geodesic distance map clearly shows its advantages in capturing location priors and neighborhood similarity information based on image context, which is crucial for achieving accurate target localization in medical image segmentation tasks.

[0154] In the implementation process of the spatial geometry refinement component, the refined features are obtained according to the spectral slice, the second spectral spatial features and the initial segmentation mask, including steps S0231 to S0233.

[0155] S0231: Determine initial segmentation mask Relative to spectral slice The geodesic distance , the geodesic distance Normalize and obtain the geometric prior graph G;

[0156] Specifically, the initial segmentation mask generated by the first segmentation head By calculating the spectral slice relative to the input MHSI This process generates a significant geometric prior map to guide subsequent segmentation refinement. and Respectively represent masks The set of foreground and background pixels in the image. , defined to the foreground set The unsigned geodesic distance is as follows:

[0157]

[0158] in Indicates that from the pixel To Pixel The geodesic distance is calculated as follows:

[0159]

[0160] here, Represents pixels and The set of all possible paths between is a feasible path, with parameters Represents. Vector is the tangent vector in the path direction, represents the image gradient, ensuring that the path optimizes the arrive The strength similarity.

[0161] Then, the geodesic distance is normalized to:

[0162]

[0163] S0232: Fuse the second spectral spatial feature through the output of the first hybrid convolution unit with the geometric prior graph G through the output of the second hybrid convolution unit to obtain a refined fusion feature ;

[0164] Specifically, in the implementation of geometric refinement SGR, the feature map of the final decoder layer as input. This feature map is processed by a hybrid convolutional unit (HCU), which consists of two cascaded convolutions with different kernel sizes. At the same time, the geometric prior map generated by the geodesic distance transform is also processed by a similar HCU. These processed features are then fused through the aforementioned fusion module to obtain the fused feature , which is expressed in the following form:

[0165]

[0166] in represents the mixed convolution unit operation, It is a fusion module.

[0167] S0233: Introduce expansion feature aggregation to refine fusion features Enhance key features to obtain refined features .

[0168] Specifically, in order to further enhance the feature representation, a dilated feature aggregation (DFA) module is introduced, which uses multi-way parallel dilated convolution to capture multi-scale context information. These features are concatenated in the channel dimension and reduced to a single channel through a 1×1 convolution to generate a spatial attention map. By seamlessly integrating local and global information, the model's ability to adapt to objects of different scales is greatly improved. With attention map Perform element-by-element multiplication to selectively enhance key features and generate enhanced features :

[0169]

[0170] Given the feature map from the final decoder layer Contains rich high-level semantic information and introduces residual connections to preserve these key details during refinement:

[0171]

[0172] Finally, the refined feature E is passed through the segmentation head to generate a refined segmentation mask. This design not only effectively combines geometric priors with semantic information, but also maintains end-to-end trainability, thereby improving segmentation performance while ensuring computational efficiency.

[0173] S03: Constructing a total loss function to optimize the medical hyperspectral image segmentation network to obtain a medical hyperspectral image segmentation optimization network.

[0174] It should be noted that a combined loss function is defined for each probability map, which is implemented by calculating the sum of the Dice similarity coefficient (DSC) loss and the binary cross entropy (BCE) loss. The combined loss function of a given probability map is expressed as ,in , respectively represent the loss terms of initial segmentation and final fine segmentation. The formula of DSC loss is ,in and Represent the predicted probability map and the true label respectively, is the total number of pixels. The BCE loss is defined as The total loss function of the entire segmentation task is expressed as the weighted sum of each loss term:

[0175]

[0176] in, and is a weight coefficient that balances the contribution of the initial segmentation loss and the final segmentation loss. This formula allows the integration of multi-stage supervision, thereby improving the robustness and accuracy of the segmentation results.

[0177] S04: Obtaining a final image segmentation result based on the medical hyperspectral image segmentation optimization network.

[0178] In summary, based on the improved U-shaped network structure and dual-stream strategy, this application proposes an innovative MHSI segmentation framework that aims to utilize the rich information content of hyperspectral data while addressing the inherent challenges in traditional segmentation methods. The framework contains the following core components:

[0179] The dual encoder, consisting of a dedicated spectral encoder and a spatial encoder, effectively captures the heterogeneity of MHSI data and overcomes the limitations of the traditional single encoder method. The feature fusion mechanism of each layer is designed to organically combine the output of the spectral encoder with the output of the corresponding layer of the spatial encoder, thus realizing the fusion of spectral-spatial information;

[0180] The improved skip learning component incorporates an adaptive learning mechanism for feature selection and enhancement based on the traditional U-shaped network skip connection, which significantly improves the efficiency of information flow and the utilization of multi-scale features;

[0181] The decoder part is symmetrical with the encoder structure. It gradually restores the spatial resolution through layer-by-layer upsampling and feature fusion, and finally generates a high-quality segmentation mask.

[0182] The spatial geometry refinement component plays an important role in this framework. It optimizes the initial segmentation results by introducing geometric prior knowledge, especially for complex anatomical structures and irregular lesions, significantly improving the boundary accuracy and shape fidelity of the segmentation results.

[0183] To further optimize the performance, this application integrates three key modules: Spectral Mamba Enhancement Layer (SME), Dual-Scale Convolutional Block Attention Module (DS-CBAM), and Spatial Geometry Refinement (SGR). SME is integrated in the spectral encoder to enhance and extract high-dimensional spectral features. DS-CBAM is integrated in the jump learning component to improve selective information transmission through a multi-scale feature attention mechanism. SGR is applied in the spatial geometry refinement component stage to fine-tune the segmentation results using spatial geometry information.

[0184] See also Figure 8 , shown is an experimental comparison implementation flow chart of an image segmentation method based on a medical hyperspectral image segmentation network proposed in the second embodiment of the present application, including:

[0185] S001: Obtain a multidimensional bile duct dataset for cholangiocarcinoma diagnosis and a Heidelberg pig hyperspectral imaging dataset for simulation experiments;

[0186] To demonstrate the effectiveness of this scheme, a large number of experiments were conducted on two types of datasets covering six different tissues or organs. The results showed that this scheme is competitive with the most advanced methods in multiple evaluation indicators, effectively verifying its effectiveness in overcoming the limitations of existing MHSI segmentation techniques. All experiments were conducted under the PyTorch framework using an NVIDIA RTX 3090 GPU (24GB). The AdamW algorithm was used for optimization, training for 100 cycles, and the cosine annealing scheduler was used to dynamically adjust the learning rate. The training set, validation set, and test set were divided into a ratio of 3:1:1, and the model weights with the best performance on the validation set were used for testing to reduce the risk of overfitting and improve generalization ability. The key information and result analysis of the experiment will be described in detail below.

[0187] To validate the proposed method, two publicly available datasets were used:

[0188] Multidimensional bile duct dataset (MDC) for cholangiocarcinoma diagnosis: This dataset contains 538 high-quality annotated hyperspectral images collected specifically for cholangiocarcinoma diagnosis. The image acquisition process is performed using a hyperspectral imaging system combined with a microscope, through the transmission light capture of bile duct tissue sections, the spectral range is 550nm to 1000nm, and 60 independent spectral bands are generated for each scene. The single-band image size of the original microscopic hyperspectral data cube is 1024×1280 pixels. In order to be consistent with previous studies and optimize computational efficiency, preprocessed (calibrated and downsampled) images were used for experimental analysis, and finally a data cube with a size of 256×320×60 (height×width×number of channels) was generated. Fig. 9 Three sample examples from the MDC dataset are shown, including grayscale images in the 550nm, 700nm, and 850nm spectral bands and the target’s true position mask (true annotation).

[0189] Heidelberg pig hyperspectral imaging dataset (HeiPorSPECTRAL): The HeiPorSPECTRAL dataset contains 5756 hyperspectral data cubes of 20 types of physiological organs from 11 pigs, which are accurately annotated by a panel of three medical experts to ensure high-quality real labels. Each hyperspectral data cube in this dataset has a size of 480×640×100 (height×width×number of channels). The spectral information contains 100 non-overlapping bands covering a wavelength range of 500 nm to 1000 nm, providing high-resolution spectral features for each spatial location. In the experimental analysis, 5 specific organs in the HeiPorSPECTRAL dataset were selected to ensure the diversity and manageability of the dataset. Fig.10 Basic information and sample examples of 5 organs selected from HeiPorSPECTRAL are shown.

[0190] S002: Evaluate the performance of image segmentation methods based on medical hyperspectral image segmentation networks on key indicators such as IoU, DSC and HD;

[0191] The proposed method is comprehensively evaluated on two representative datasets covering six different tissues and organs. The evaluation metrics include Dice Similarity Coefficient (DSC), Intersection over Union (IoU) and Hausdorff Distance (HD), which evaluate the accuracy, overlap and boundary error of segmentation, respectively. To verify its effectiveness, the proposed method is compared with eight popular existing segmentation methods, including 2D and 3D frameworks such as PCA-UNet, CGRU-UNet, Dual-Stream-MHSI, 3D-UNet, nnUNet, HyperNet, Swin-UNETR and SpecTr.

[0192] Table 1 Performance comparison with common methods on the MDC dataset

[0193]

[0194] As shown in Table 1, our method achieves significant improvements in three key metrics: IoU reaches 64.75%, DSC increases to 77.32%, and HD decreases to 74.00 pixels. In particular, in terms of throughput, our method achieves 9.32 images per second, balancing computational efficiency with high accuracy. These results show that our method is able to effectively manage the trade-off between accuracy and efficiency when faced with complex medical image segmentation tasks. To further verify the generalization ability of our method, we directly compare it with the current best performing Dual-Stream-MHSI method on the HeiPorSPECTRAL dataset.

[0195] Table 2 Comparison with Dual-Stream-MHSI method on HeiPorSPECTRAL dataset

[0196]

[0197] As shown in Table 2, our method performs well in the segmentation tasks of five different organs. In terms of the two key indicators, IoU and DSC, our method achieves excellent results on all organs, especially in the segmentation of the spleen and small intestine, where the performance improvement is particularly significant. For the HD indicator, our method achieves lower errors on all organs except the great veins. It is worth noting that the complexity of the HeiPorSPECTRAL dataset is lower than that of the MDC dataset, and the target organs have less variation in shape, size, and position, and the boundaries are clearer, making the segmentation task relatively simple. However, our method still significantly improves the performance under these conditions, indicating its good adaptability and robustness on datasets of different difficulty levels.

[0198] S003: Evaluate the visual performance of image segmentation methods based on medical hyperspectral image segmentation networks;

[0199] The features extracted by the spectral encoder and the spatial encoder show their own unique characteristics, highlighting the complementarity of multimodal information processing. The spectral encoder mainly captures subtle changes and spectral features in high-dimensional spectral data, while the spatial encoder focuses on extracting structural and texture information. Through the feature fusion module, the outputs of the corresponding layers of the two encoders are effectively integrated, significantly improving the representation ability of the model. Fig.11 The visualization results of the original output features of the corresponding layer and the fused features are shown. The comparative analysis clearly shows the complementarity of the features emphasized by the two encoders. Specifically, the spectral encoder shows higher sensitivity to subtle spectral features, enabling it to effectively distinguish targets with similar spatial structures but different spectral features. In contrast, the spatial encoder performs well in capturing the geometric shape, texture pattern, and spatial distribution of the target, and can well identify targets with significant spatial structural differences.

[0200] In order to more comprehensively evaluate the advantages of our method, several existing algorithms with better performance were selected from the comparison methods for qualitative analysis. Fig.12 The segmentation results of the existing methods and this method on 10 representative samples are shown. For the sake of demonstration, the input is a grayscale image in a single band. Comparing these visual results, it can be seen that the segmentation quality of this method has been significantly improved. Compared with other methods, this algorithm shows stronger robustness and accuracy in processing complex medical images, especially in reducing over-segmentation and under-segmentation problems, and significantly reduces the incidence of false positives and false negatives. This progress is mainly attributed to the excellent performance of this method in dealing with fuzzy boundaries between target and non-target areas. This method performs well in edge preservation and detail capture, and can accurately depict complex anatomical structures, especially when segmenting targets with irregular shapes or low contrast.

[0201] S004: Conduct ablation research on image segmentation methods based on medical hyperspectral image segmentation networks;

[0202] In order to comprehensively evaluate the effectiveness of the key components in the proposed MHSI segmentation method, a series of detailed ablation studies are conducted. These studies analyze the independent impact of each module and explore the synergy between them. The following in-depth analysis is based on the experimental results in Table 3:

[0203] 1. Effectiveness of spectral encoder: Comparing Experiment 2 with Experiment 1 clearly shows the importance of the spectral encoder. Its introduction improves IoU from 62.04% to 63.25%, DSC from 75.23% to 76.23%, and HD is significantly reduced by 1.55 pixels. These improvements show that the spectral encoder can effectively extract and enhance high-dimensional spectral features, making full use of the rich spectral information inherent in MHSI data, thereby improving segmentation accuracy and boundary positioning accuracy.

[0204] 2. Contribution of DS-CBAM: Comparison between Experiment 3 and Experiment 1 reveals the performance improvement brought by DS-CBAM. Its introduction improves IoU from 62.04% to 62.89% and DSC from 75.23% to 76.10%. This highlights the effectiveness of DS-CBAM as a key component in skip learning, indicating that it can optimize selective information transfer through a multi-scale attention mechanism and enhance the model's perception of complex anatomical structures and pathological features.

[0205] 3. Impact of SGR: Comparing Experiment 4 with Experiment 1, the key role of SGR is highlighted. Its introduction improves IoU from 62.04% to 63.29% and DSC from 75.23% to 76.18%. These results emphasize the key function of SGR in the geometric refinement stage, which can effectively integrate spatial geometric information, especially when dealing with complex anatomical structures and irregular lesions, to improve the boundary accuracy and shape consistency of segmentation results.

[0206] 4. Synergistic effect of dual module combination:

[0207] (a) Spectral Encoder + DS-CBAM (Experiment 5): This combination fully exploits the advantages of spectral feature extraction and multi-scale attention mechanism and significantly improves all indicators.

[0208] (b) Spectral Encoder + SGR (Experiment 7): This combination shows improvements in all three metrics, especially in IoU and DSC, indicating that the combination of high-dimensional spectral features and geometric optimization comprehensively improves the segmentation performance.

[0209] (c) DS-CBAM + SGR (Experiment 6): Although there is an improvement over the use of a single module, the effect of this combination is not as significant as the other combinations. This may indicate that the synergy between the attention mechanism and geometric optimization is limited in the absence of dedicated spectral processing.

[0210] 5. Comprehensive effect of three-module integration: Experiment 8 shows the best performance achieved when combining all modules (backbone network, spectral encoder, DS-CBAM and SGR). The peak IoU is 64.75% and DSC is 77.32%, which shows that the synergy between the modules achieves the best effect. These results verify that our proposed comprehensive framework maximizes the potential of MHSI data and effectively copes with the inherent complexity in high-dimensional, multimodal medical imaging.

[0211] This ablation study provides empirical evidence for the effectiveness and complementarity of each component in the MHSI segmentation method. The spectral encoder plays an important role in utilizing spectral information, DS-CBAM enhances feature selection and attention mechanism, and SGR improves geometric accuracy.

[0212] Table 3 Ablation study on the MDC dataset to evaluate the impact of backbone network, spectral encoder, DS-CBAM and spatial geometry refinement on segmentation performance indicators

[0213]

[0214] See also Fig.13 , which is a schematic diagram of the structure of an image segmentation system based on a medical hyperspectral image segmentation network proposed in the third embodiment of the present application, the system includes:

[0215] A data acquisition module 100 is used to acquire a medical hyperspectral image sample set, wherein the medical hyperspectral image sample set includes a first image sample set, a second image sample set and a spectral slice;

[0216] The network construction module 200 is used to construct a medical hyperspectral image segmentation network, and the network construction module includes:

[0217] A dual encoder unit 210, a skip learning component unit 220, a decoder unit 230, a first segmentation head unit 240, a spatial geometry refinement component unit 250, and a second segmentation head unit 260, wherein the dual encoder unit 210 includes a spectral encoder subunit 211 and a spatial encoder subunit 212, wherein:

[0218] The spectral encoder subunit 211 is used to obtain spectral features according to the first image sample set.

[0219] The spatial encoder subunit 212 is configured to obtain spatial features according to the second image sample set,

[0220] The jump learning component unit 220 is used to obtain a first spectral spatial feature according to the spectral feature and the spatial feature,

[0221] The decoder unit 230 is configured to obtain a second spectral spatial feature according to the first spectral spatial feature,

[0222] The first segmentation head unit 240 is used to obtain an initial segmentation mask according to the second spectral spatial feature,

[0223] The spatial geometry refinement component unit 250 is used to obtain a refined feature according to the spectral slice, the second spectral spatial feature and the initial segmentation mask,

[0224] The second segmentation head unit 260 is used to obtain a refined segmentation mask according to the refined features;

[0225] A function construction module 300 is used to construct a total loss function to optimize the medical hyperspectral image segmentation network to obtain a medical hyperspectral image segmentation optimization network;

[0226] The image segmentation result module 400 is used to obtain the final image segmentation result based on the medical hyperspectral image segmentation optimization network.

[0227] On the other hand, the present application also proposes a computer device, which includes a memory, a processor, and a processing program stored in the memory and executable on the processor. When the processing program is executed by the processor, the above-mentioned image segmentation method based on the medical hyperspectral image segmentation network is implemented.

[0228] On the other hand, the present application further proposes a storage medium, on which a processing program is stored. When the processing program is executed by a processor, the above-mentioned image segmentation method based on a medical hyperspectral image segmentation network is executed.

[0229] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0230] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0231] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An image segmentation method based on a medical hyperspectral image segmentation network, characterized in that: The method comprises: Acquire a medical hyperspectral image sample set, wherein the medical hyperspectral image sample set includes a first image sample set, a second image sample set, and a spectral slice; A medical hyperspectral image segmentation network is constructed, wherein the medical hyperspectral image segmentation network includes a dual encoder, a jump learning component, a decoder, a first segmentation head, a spatial geometry refinement component, and a second segmentation head, wherein the dual encoder includes a spectral encoder and a spatial encoder, wherein: The spectral encoder is used to obtain spectral features according to the first image sample set. The spectral encoder includes multiple levels of spectral Mamba enhancement layers. Each spectral Mamba enhancement layer includes a spectral Mamba enhancement module. The spectral Mamba enhancement module includes a serialization block and a spectral Mamba enhancement block. The processing process of the spectral Mamba enhancement module includes: converting the first image sample set Subset features of Input the serialization block and convert it into a feature sequence , the feature sequence The coding sequence is obtained by position encoding ; The coding sequence Input the spectral Mamba enhancement block to get the feature enhancement sequence ; The feature enhancement sequence Perform feature reconstruction to obtain a three-dimensional tensor ; The spatial encoder is used to obtain spatial features according to the second image sample set, The jump learning component is used to obtain a first spectral spatial feature according to the spectral feature and the spatial feature. The jump learning component includes a dual-scale channel attention layer, a dual-scale spatial attention layer and a residual connection layer. The step of obtaining the first spectral spatial feature according to the spectral feature and the spatial feature includes: performing feature fusion on the spectral feature and the spatial feature to obtain a spectral spatial fusion feature. ; Generate channel attention weights based on the dual-scale channel attention layer , by element-wise multiplication Applied to spectral spatial fusion features , to obtain ; Generate spatial attention weights based on the dual-scale spatial attention layer , by element-wise multiplication Application , to get ; According to the residual connection layer, the spectral space fusion features and Combined, the first spectral spatial feature is obtained ; The decoder is used to obtain a second spectral spatial feature according to the first spectral spatial feature, The first segmentation head is used to obtain an initial segmentation mask according to the second spectral spatial feature, The spatial geometry refinement component is used to obtain a refined feature according to the spectral slice, the second spectral spatial feature and the initial segmentation mask, including: determining the initial segmentation mask Relative to spectral slice The geodesic distance , the geodesic distance Normalize to obtain a geometric prior graph G; fuse the second spectral spatial feature through the output of the first hybrid convolution unit with the geometric prior graph G through the output of the second hybrid convolution unit to obtain a refined fusion feature ; Introduce expansion feature aggregation to refine fusion features Enhance key features and introduce residual connections to obtain refined features ; The second segmentation head is used to obtain a refined segmentation mask according to the refined features; Constructing a total loss function to optimize the medical hyperspectral image segmentation network to obtain a medical hyperspectral image segmentation optimization network; The final image segmentation result is obtained based on the medical hyperspectral image segmentation optimization network.

2. The image segmentation method according to claim 1, characterized in that: The steps of acquiring a medical hyperspectral image sample set, wherein the medical hyperspectral image sample set includes a first image sample set, a second image sample set and a spectral slice include: Get a sample set of medical hyperspectral images , divide X into the first image sample set , the second image sample set and spectral slices , and then input three branches respectively; Among them, the first image sample set For input to the spectral encoder, the second image sample set For input to spatial encoder, spectral slice Used to refine the input spatial geometry components.

3. The image segmentation method according to claim 2, characterized in that: The first image sample set Subset features of Input the serialization block and convert it into a feature sequence , the feature sequence The coding sequence is obtained by position encoding The steps include: Subset Features Adjust the shape and perform spectral dimension mapping to obtain ; Will Divide into non-overlapping blocks, each denoted by ,in Indicates the size of the block; Block Linear projection to the preset dimension to generate feature sequence ,definition , represents the spectral mapping function; Encode the position Add to feature sequence The coding sequence was obtained .

4. The image segmentation method according to claim 1, characterized in that: The spectral Mamba enhancement block includes a Mamba block and a spectral channel attention block, wherein the processing process of the Mamba block is expressed as: ， in, and Represents the input and output tags of the Mamba block, Represents the mapping function of the Mamba block; The processing process of the spectral channel attention block is expressed as: Input Tag , using compression operations to aggregate features and applying global average pooling to obtain the spectral vector , and then through one-dimensional convolution projection, we get ; Introducing a spectral memory unit, which includes a global memory bank ,from Retrieve the most relevant low-rank spectral vector from To adjust , Through one-dimensional convolution Scaling is performed to generate low-rank constrained spectral attention weights, and the spectral attention weights are combined with the input Multiply element by element, and the final output is .

5. The image segmentation method according to claim 1, characterized in that: The total loss function is ,in, and is the weight coefficient that balances the contribution of initial segmentation and final fine segmentation loss, and Represent the loss terms of initial segmentation and final refined segmentation respectively; ,in , represents the Dice similarity coefficient loss, represents the binary cross entropy loss; ， ， in, and Represent the predicted probability map and the true label respectively, is the total number of pixels.

6. A computer device, characterized in that: The computer device includes a memory, a processor, and a processing program stored in the memory and executable on the processor. When the processing program is executed by the processor, the image segmentation method according to any one of claims 1 to 5 is implemented.

7. A storage medium, characterized in that: The storage medium stores a processing program, and when the processing program is executed by a processor, the image segmentation method according to any one of claims 1 to 5 is implemented.

Images