Optical coherence tomography image segmentation method and system based on state space model

Abstract

The invention relates to an optical coherence tomography image segmentation method and system based on a state space model. The method comprises the following steps: inputting an optical coherence tomography image into a state space model comprising a dual-path encoder, a feature fusion module and a multi-scale dynamic fusion decoder; extracting a first feature and a second feature from a first coding path and a second coding path in the dual-path encoder; the feature fusion module fuses the first feature and the second feature and inputs the fused features into a multi-scale dynamic fusion decoder to obtain an optical coherence tomography image segmentation prediction map; and taking the cross entropy and the Dice loss as loss functions to obtain an optical coherence tomography image segmentation result. A double-path encoder is combined with global semantic capture capability and local detail and long-range dependence modeling capability, and high integration of different scale features is realized through progressive feature aggregation and up-sampling, so that the accuracy of optical coherence tomography image segmentation can be improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for optical coherence tomography image segmentation based on a state-space model. Background Technology

[0002] In modern medical diagnostic systems, medical image segmentation technology plays an indispensable and crucial role. Accurate image segmentation is a vital foundation for assisting doctors in disease diagnosis, treatment planning, and monitoring treatment effectiveness. Medical images possess unique characteristics, with target regions often exhibiting blurred boundaries and significant structural variations. Specifically, in skin lesion images, lesions exhibit strong heterogeneity in size, shape, and texture, making it difficult to define segmentation boundaries; in fundus optical coherence tomography (OCT) images, the gray-level contrast between pathological areas and adjacent normal tissues is extremely low, making it difficult to clearly distinguish lesion areas; in histopathological images, the boundaries of lesions such as polyps often exhibit blurred features, further increasing the difficulty of accurate segmentation. UNet++ and UNetV2 have improved the performance of medical image segmentation tasks by modifying the original structure of UNet; Attention UNet introduces an attention gate mechanism, improving the DSC coefficients in pancreas segmentation tasks. However, CNNs are limited by their inherent local receptive field and insufficient ability to model long-distance dependencies, resulting in limitations in the representation of blurred boundaries when processing complex anatomical structures.

[0003] To address the limitations of convolutional neural networks, the Transformer architecture, which achieves long-range dependency learning through self-attention, has been integrated into medical image segmentation systems. TransUNet, by combining a Transformer encoder for global context and a UNet decoder for preserving spatial resolution, demonstrates superior performance. However, the quadratic complexity of Transformer layers remains a bottleneck for real-time medical image analysis systems. In recent years, state-space models (SSMs) have emerged as a novel sequence modeling approach, offering new insights for medical image segmentation. Vision Mamba, a typical state-space model, improves computational efficiency while maintaining long-range dependency modeling capabilities through selective state-space operations with linear complexity.

[0004] Recently, many studies have improved upon Mamba and applied it to computer vision tasks. VM-UNet integrates Vision Mamba into the UNet architecture using an asymmetric decoder-encoder structure and applies it to medical image segmentation tasks; its performance in skin lesion segmentation surpasses that of the traditional CNN structure used by UNet. Meanwhile, HMT-UNet, by integrating the complementary advantages of Mamba and Transformer architectures, demonstrates superior performance in segmenting colonic tumors. Mamba was originally tailored for sequential data processing, and its inherent unidirectional architecture limits its ability to fully utilize the complex spatial dependencies in medical imaging.

[0005] However, although the Mamba series models have shown good application prospects in the field of optical coherence tomography (OCT) image segmentation, the Mamba models cannot simultaneously utilize the contextual information of the target region for bidirectional feature interaction, and existing Mamba models generally lack efficient and powerful positional encoding mechanisms. Therefore, traditional use of Mamba models for OCT image segmentation often suffers from a lack of accuracy and low practicality. Summary of the Invention

[0006] Therefore, in order to solve the above-mentioned technical problems, a method and system for optical coherence tomography (OCT) image segmentation based on a state-space model is provided, which can improve the accuracy and practicality of OCT image segmentation.

[0007] A method for segmenting optical coherence tomography images based on a state-space model, the method comprising:

[0008] An optical coherence tomography (OCT) image to be segmented is acquired, and the OCT image is input into a trained state space model; the state space model includes a dual-path encoder, a feature fusion module, and a multi-scale dynamic fusion decoder.

[0009] The optical coherence tomography (OCT) image is captured using the first encoding path in the dual-path encoder to capture contextual semantics and global features, and the first feature is extracted. The OCT image is then processed using the second encoding path in the dual-path encoder to extract the second feature. The first encoding path contains a hybrid module of residual network and Transformer, and the second encoding path contains a sequence of Natten visual state space (NVSS) modules.

[0010] The feature fusion module is used to receive and fuse the first feature and the second feature to generate a fused feature.

[0011] The fused features are input into the multi-scale dynamic fusion decoder, and the multi-scale dynamic fusion module in the multi-scale dynamic fusion decoder performs progressive feature aggregation and upsampling processing on the fused features to obtain an optical coherence tomography image segmentation prediction map.

[0012] Cross-entropy and Dice loss are used as loss functions to calculate the loss of the optical coherence tomography (OCT) image segmentation prediction map, and the state space model is optimized to obtain the final OCT image segmentation result.

[0013] In one embodiment, the first encoding path in the dual-path encoder is used to capture contextual semantics and global features of the optical coherence tomography image, and the first features are extracted, including:

[0014] The optical coherence tomography image is input into the residual network in the first encoding path, and convolution and downsampling are performed through each layer of the residual network to obtain the feature sequence.

[0015] The feature sequence is input into the Transformer encoder, and the global dependency between all features in the feature sequence is modeled through the self-attention mechanism in the Transformer encoder, and the feature sequence with global context enhancement is output.

[0016] The feature sequence enhanced by global context is reconstructed into a feature map, which is used as the first feature.

[0017] In one embodiment, the optical coherence tomography image is processed using the second encoding path in the dual-path encoder to extract a second feature, including:

[0018] The optical coherence tomography image is input into the second encoding path, and spatial downsampling and channel transformation are performed through the initial convolutional layer in the second encoding path to obtain initial intermediate features;

[0019] The initial intermediate features are input into at least one cascaded NVSS module, where local details are enhanced and preserved by residual blocks containing grouped convolutions. The local details are then input into the local neighborhood attention Natten layer, where adaptive local context information is aggregated through a sliding window attention mechanism.

[0020] The aggregated adaptive local context information is input into the visual state space (VSS) block, and the sequence of each path is determined by scanning the path to obtain the long-distance dependency between features.

[0021] Based on the aforementioned long-distance dependency, deep features are determined as the second feature.

[0022] In one embodiment, the scan path includes an original scan path and an extended scan path;

[0023] The original scanning path includes scanning paths in the horizontal, vertical, anti-horizontal, and anti-vertical directions;

[0024] The extended scan path includes diagonal and spiral scan paths.

[0025] In one embodiment, the feature fusion module receives and fuses the first feature and the second feature to generate a fused feature, including:

[0026] The first feature is subjected to batch normalization, activation, and convolution, and the processed feature is subjected to channel attention operation to obtain the first attention feature;

[0027] The second feature is batch normalized, activated, and convolved, and then spatial attention is applied to the processed feature to obtain the second attention feature.

[0028] The first attention feature and the second attention feature are fused together, and the channel distribution is adjusted by convolution to generate the fused feature.

[0029] In one embodiment, the fused features are progressively aggregated and upsampled by the multi-scale dynamic fusion module in the multi-scale dynamic fusion decoder to obtain an optical coherence tomography (OCT) image segmentation prediction map, including:

[0030] The multi-scale dynamic fusion module receives hierarchical features from different levels and performs channel attention and spatial attention processing on the hierarchical features respectively.

[0031] The processed features are concatenated and aligned, and spatial adaptive weights corresponding to the number of layers are generated through convolution and activation functions.

[0032] The spatial adaptive weights are used to perform a weighted summation of the hierarchical features at each level to obtain preliminary fused features;

[0033] The preliminary fused features are input into two cascaded deformable convolutional layers for deformation modeling and feature calibration. The calibrated features are added to the anchor features to output the fused features of the current layer.

[0034] The fused features are progressively upsampled to restore the image resolution and obtain an optical coherence tomography (OCT) image segmentation prediction map.

[0035] In one embodiment, the cross-entropy loss is used to measure the difference between the predicted result and the true label; the Dice loss is used to measure the degree of overlap between the predicted region and the true region.

[0036] An optical coherence tomography image segmentation system based on a state-space model, the system comprising:

[0037] The image acquisition module is used to acquire the optical coherence tomography (OCT) image to be segmented and input the OCT image into the trained state space model; the state space model includes a dual-path encoder, a feature fusion module, and a multi-scale dynamic fusion decoder.

[0038] The feature extraction module is used to capture contextual semantics and global features of the optical coherence tomography image using the first encoding path in the dual-path encoder, and extract the first feature; and to process the optical coherence tomography image using the second encoding path in the dual-path encoder, and extract the second feature; wherein the first encoding path includes a hybrid module of residual network and Transformer, and the second encoding path includes a sequence of Natten visual state space (NVSS) modules;

[0039] The feature fusion module is used to receive and fuse the first feature and the second feature to generate a fused feature.

[0040] The pre-segmentation module is used to input the fused features into the multi-scale dynamic fusion decoder. The multi-scale dynamic fusion module in the multi-scale dynamic fusion decoder performs progressive feature aggregation and upsampling processing on the fused features to obtain the optical coherence tomography image segmentation prediction map.

[0041] The model optimization and image segmentation module is used to calculate the loss of the optical coherence tomography (OCT) image segmentation prediction map by using cross-entropy and Dice loss as loss functions, and to optimize the state space model to obtain the final OCT image segmentation result.

[0042] In one embodiment, the feature extraction module is further configured to input the optical coherence tomography image into a residual network in the first encoding path, perform convolution and downsampling processing through each layer of the residual network to obtain a feature sequence; input the feature sequence into a Transformer encoder, model the global dependencies between all features in the feature sequence through the self-attention mechanism in the Transformer encoder, and output a feature sequence enhanced by global context; and reassemble the feature sequence enhanced by global context into a feature map as the first feature.

[0043] In one embodiment, the feature extraction module is further configured to input the optical coherence tomography (OCT) image into a second encoding path, perform spatial downsampling and channel transformation through an initial convolutional layer in the second encoding path to obtain initial intermediate features; input the initial intermediate features into at least one cascaded NVSS module, enhance and preserve local detail information through residual blocks containing grouped convolutions, input the local detail information into a local neighborhood attention (NATEN) layer, aggregate adaptive local context information through a sliding window attention mechanism; input the aggregated adaptive local context information into a visual state space (VSS) block, determine the sequence of each path through the scanning path to obtain long-distance dependencies between features; and determine deep features based on the long-distance dependencies as second features.

[0044] The aforementioned optical coherence tomography (OCT) image segmentation method and system based on the state-space model utilizes a dual-path encoder that combines the global semantic capture capabilities of residual networks and Transformers with the local detail and long-range dependency modeling capabilities of the NVSS module, thus addressing the insufficient utilization of spatial dependencies in the traditional Mamba unidirectional architecture. The multi-scale dynamic fusion module achieves efficient integration of features at different scales through progressive feature aggregation and upsampling, resolving the issues of large target scale differences and blurred boundaries in OCT images. The weighted hybrid loss function of cross-entropy and Dice loss balances pixel-level classification accuracy and overall lesion contour matching, effectively avoiding overfitting and ensuring the accuracy of lesion region prediction, thereby improving the precision of OCT image segmentation. Attached Figure Description

[0045] Figure 1 This is an application environment diagram of an optical coherence tomography image segmentation method based on a state-space model in one embodiment;

[0046] Figure 2 This is a flowchart illustrating an optical coherence tomography image segmentation method based on a state-space model in one embodiment.

[0047] Figure 3 This is a schematic diagram illustrating the application process of an optical coherence tomography image segmentation method based on a state-space model in one embodiment.

[0048] Figure 4 A visualization of experimental results on the ISIC17 and OCT datasets in one embodiment;

[0049] Figure 5 The figure shows the experimental results of an optical coherence tomography image segmentation method based on a state-space model in one embodiment at different scales.

[0050] Figure 6 This is a structural block diagram of an optical coherence tomography image segmentation system based on a state-space model in one embodiment;

[0051] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] It is understood that the terms "first," "second," etc., used in this application may be used herein to describe encoding paths, but these encoding paths are not limited by these terms. These terms are only used to distinguish one encoding path from another. For example, without departing from the scope of this application, the first encoding path may be referred to as the second encoding path, and similarly, the second encoding path may be referred to as the first encoding path. Both the first and second encoding paths are encoding paths, but they are not the same encoding path.

[0054] The optical coherence tomography image segmentation method based on a state-space model provided in this application can be applied to, for example... Figure 1 The application environment shown. For example... Figure 1As shown, the application environment includes a computer device 110. The computer device 110 can acquire an optical coherence tomography (OCT) image to be segmented and input the OCT image into a trained state space model. The state space model includes a dual-path encoder, a feature fusion module, and a multi-scale dynamic fusion decoder. The computer device 110 can use the first encoding path in the dual-path encoder to capture contextual semantics and global features of the OCT image and extract the first feature; it can also use the second encoding path in the dual-path encoder to process the OCT image and extract the second feature. The first encoding path includes a hybrid module of residual network and Transformer, and the second encoding path includes a Natten visual state space... The sequence of NVSS modules; the computer device 110 can use the feature fusion module to receive and fuse the first feature and the second feature to generate fused features; the computer device 110 can input the fused features into the multi-scale dynamic fusion decoder, and the multi-scale dynamic fusion module in the multi-scale dynamic fusion decoder performs progressive feature aggregation and upsampling processing on the fused features to obtain the optical coherence tomography (OCT) image segmentation prediction map; the computer device 110 can use cross-entropy and Dice loss as loss functions, use the loss functions to calculate the loss of the OCT image segmentation prediction map, and optimize the state space model to obtain the final OCT image segmentation result. The computer device 110 can be, but is not limited to, various personal computers, laptops, smartphones, robots, tablets, etc.

[0055] In one embodiment, such as Figure 2 As shown, an optical coherence tomography (OCT) image segmentation method based on a state-space model is provided, including the following steps:

[0056] Step 202: Obtain the optical coherence tomography (OCT) image to be segmented, and input the OCT image into the trained state space model; the state space model includes a dual-path encoder, a feature fusion module, and a multi-scale dynamic fusion decoder.

[0057] The optical coherence tomography (OCT) image segmentation method based on a state-space model provided in this embodiment can be implemented using a trained state-space model MF-UNet, which consists of a novel feature encoder and decoder. Specifically, as... Figure 3 As shown in (a), this embodiment proposes a dual-branch encoder utilizing ResNet and Mamba, and a multi-scale dynamic fusion module (MMSF) built on top of Mamba in the decoder.

[0058] like Figure 3As shown in (a), the state-space model MF-UNet initially extracts features through a dual-path encoder: one path uses a hybrid module combining ResNet34 and Transformer to capture contextual semantics and global features, while the other path utilizes the NVSS (Natten Visual State Space) model with integrated local attention (Natten), thus capturing long-range dependency features while preserving local spatial details; these two paths are fused through a DAG module to obtain rich feature information, which allows the network to focus more on lesion generation; the decoder adopts a multi-scale feature pyramid structure, where the MMSF module uses SS2D to adaptively fuse feature maps of four different scales; in the feature reconstruction process, residual connections are introduced to preserve low-level semantic information of the original features, and deformable convolutions are used to optimize spatial alignment; subsequently, progressive upsampling is used to restore resolution, gradually generating the final output.

[0059] Step 204: Use the first encoding path in the dual-path encoder to capture contextual semantics and global features of the optical coherence tomography image and extract the first feature; use the second encoding path in the dual-path encoder to process the optical coherence tomography image and extract the second feature; wherein, the first encoding path contains a hybrid module of residual network and Transformer, and the second encoding path contains a sequence of Natten visual state space NVSS modules.

[0060] The NVSS module can be used to build encoder networks, such as Figure 3 As shown in (b), each NVSS block contains a residual block, a local self-attention (Natten), and a state-space model (VSS), which are stacked sequentially and connected by residual connections to capture local details and global context.

[0061] To achieve linear time complexity, Mamba compresses the global context of each channel into a fixed-dimensional hidden state. However, this compression inevitably sacrifices fine-grained spatial information, which is crucial for distinguishing region boundaries during segmentation. To address this, the architecture can be enhanced by appending a residual block containing two local convolutional residuals before the VSS block, while simultaneously incorporating local self-attention (Natten) to improve context capture.

[0062] Specifically, in one embodiment, the optical coherence tomography (OCT) image segmentation method based on a state-space model may further include the process of extracting a first feature using a first encoding path. The specific process includes: inputting the OCT image into a residual network in the first encoding path; performing convolution and downsampling processing through each layer of the residual network to obtain a feature sequence; inputting the feature sequence into a Transformer encoder; modeling the global dependencies between all features in the feature sequence using the self-attention mechanism in the Transformer encoder; and outputting a feature sequence enhanced with global context; and reconstructing the feature sequence enhanced with global context into a feature map as the first feature.

[0063] The optical coherence tomography (OCT) images to be segmented are first fed into the first encoding path and processed by a ResNet residual neural network of predefined depth, where residual blocks are stacked. Specifically, ResNet gradually reduces the spatial resolution of the feature maps through stride convolutions or pooling layers while continuously increasing the number of channels. Each residual block performs convolution operations in its local neighborhood using a 3×3 convolution kernel, effectively capturing local patterns such as edges, textures, and shapes. Through deep stacking, it indirectly obtains increasingly larger receptive fields, enabling the understanding of more complex local contexts. Next, the deepest layer or multiple layers of feature maps with the richest semantic information from the ResNet backbone are selected as input to the Transformer, flattening the spatial dimension of the feature maps into a sequence. Since the Transformer itself does not have the ability to perceive positional information, learnable positional encodings need to be added to these sequential feature embeddings. The positional encodings are added to the feature embeddings to provide the model with the absolute or relative positional information of each feature point in the original two-dimensional space.

[0064] Next, the sequence with embedded location information is fed into the Transformer encoder. By calculating attention weights, the model can dynamically identify other feature blocks that are most semantically related to the current feature block. The output of the self-attention layer is transformed non-linearly by a feedforward neural network, and each sub-layer contains residual connections and layer normalization. The output sequence after processing by the Transformer layer is restored to the two-dimensional feature map format, which is the first feature output by the first encoding path.

[0065] Specifically, in this embodiment, the process of extracting the first feature using the first encoding path is as follows: given an input feature Where B is the batch size. Given the spatial resolution and C as the number of channels, the calculation process for the residual block is as follows: ; ;in, For module input, , Both represent intermediate outputs of the residual block. The final output of the residual block is represented by Conv1, which indicates a 3×3 grouped convolution, and GroupNorm, which indicates grouped normalization. The data then enters the neighborhood attention mechanism, where the sliding window attention mechanism restricts the attention range of each pixel to its immediate neighbor.

[0066] The operation of neighborhood attention (Natten) is as follows: [The input is obtained...] , ( For the number of tokens, For the embedded dimension, its linear projection query Q, key K, value V: in, These are learnable parameters; for each position i, the attention weights are... and The definition of is: , ,in, This represents the j-th nearest neighbor of position i. It represents the relative position offset, where T denotes matrix transpose. The first feature of the final output is: , where NA stands for Xnattennatten.

[0067] In one embodiment, the optical coherence tomography (OCT) image segmentation method based on a state-space model may further include a process of extracting a second feature using a second encoding path. Specifically, this process includes: inputting the OCT image into the second encoding path; performing spatial downsampling and channel transformation through an initial convolutional layer in the second encoding path to obtain initial intermediate features; inputting the initial intermediate features into at least one cascaded NVSS module; enhancing and preserving local detail information through residual blocks containing grouped convolutions; inputting the local detail information into a local neighborhood attention Natten layer; aggregating adaptive local context information through a sliding window attention mechanism; inputting the aggregated adaptive local context information into a visual state space (VSS) block; determining the sequence of each path through the scanning path to obtain long-distance dependencies between features; and determining deep features based on the long-distance dependencies as the second feature.

[0068] Long-range dependency modeling enables comprehensive context learning and scene understanding, as demonstrated by the success of the Transformer-based optical coherence tomography state-space model. Therefore, to enable the model to efficiently learn the global context, this embodiment uses the VSS block from VMamba. The VSS block employs four different and complementary scan paths, allowing each token to integrate information from all other tokens from four different directions.

[0069] Specifically, the computer device can construct a sequence modeling path centered on enhanced Mamba, i.e., a second encoding path. Through carefully designed module combinations and scan path extensions, the computer device can efficiently model long-distance dependencies while resolutely preserving and enhancing its ability to capture local details and spatial location information, thereby compensating for the inherent shortcomings of the original state-space model (SSM) in visual tasks.

[0070] In this embodiment, the optical coherence tomography (OCT) image to be segmented can be input into the second encoding path. The image first undergoes preliminary spatial downsampling and channel dimension enhancement through a convolutional layer to obtain an initial intermediate feature map. Then, the initial intermediate feature map can be input into a cascaded Natten visual state space (NVSS) module for deep feature transformation. Each NVSS module enhances the initial intermediate feature map using a sub-module consisting of two consecutive residual blocks before inputting it into a local neighborhood attention (Natten) layer. Through learnable query (Q), key (K), and value (V) projections, the importance of features within the neighborhood is dynamically assigned, thereby obtaining features carrying rich local information. The input is finally scanned into the visual state space block. For the sequence obtained from each scanning path, a SSM with a selection mechanism is applied for computation to capture the long-distance semantic dependencies between organs, lesions, and overall anatomical structures. The output of the VSS block is added to the original input of the module through a residual connection after passing through a final projection layer to form the final output of the NVSS module. Multiple NVSS modules are usually organized into multiple stages. The spatial resolution is reduced and the number of channels is increased through downsampling operations between stages to form a multi-level feature pyramid of the second path itself, which retains more spatial details. Then, the feature map corresponding to the level of the feature layer to be fused in the first path is selected from the feature pyramid constructed by the second path as the second feature.

[0071] In one embodiment, the scanning path includes an original scanning path and an extended scanning path; wherein, the original scanning path includes scanning paths in horizontal, vertical, anti-horizontal, and anti-vertical directions; and the extended scanning path includes scanning paths in diagonal and spiral directions.

[0072] In other words, to enhance spatial information capture capabilities and enrich long-range dependency modeling, this embodiment adds a new four-directional scan, two diagonal scans, and two spiral scans to the original four-directional scan. The calculation process can be expressed as follows: ;in, ; The different scanning directions are represented as: vertical scan, horizontal scan, reverse vertical, reverse horizontal, counterclockwise scan, clockwise scan, main diagonal scan, and secondary diagonal scan. The final output can be obtained from this calculation formula, where Z represents the input projection. ;in, This represents the calculation process for the scanning method. Indicates Xnatten receives The output generated after input; .

[0073] Step 206: Use the feature fusion module to receive and fuse the first feature and the second feature to generate a fused feature.

[0074] After the two decoder branches have finished processing, the feature fusion module DAG is used to fuse and enhance the features obtained from the two encoder branches, such as... Figure 3 As shown in (c).

[0075] In one embodiment, the optical coherence tomography image segmentation method based on a state-space model may further include a process of fusing a first feature and a second feature. The specific process includes: performing batch normalization, activation, and convolution processing on the first feature, and performing channel attention operation on the processed feature to obtain a first attention feature; performing batch normalization, activation, and convolution processing on the second feature, and performing spatial attention operation on the processed feature to obtain a second attention feature; fusing the first attention feature and the second attention feature, and adjusting the channel distribution through convolution to generate a fused feature.

[0076] The DAG receives two sets of input feature maps (denoted as X and G, respectively, for the NVSS branch and the Resblock branch). The spatial attention branch performs batch normalization, ReLU activation, and 3×3 convolution on the main feature X in sequence, compressing the channel dimension to 1 / 4 of the original input before passing it into the positional attention branch. The spatial attention branch performs the same preprocessing on the guiding feature G before passing it into the channel attention branch. The output features of the two branches are then restored to the original channel dimension after batch normalization, ReLU activation, and 3×3 convolution. The results of the two branches are added pixel-wise, and then the channel distribution is adjusted by 1×1 pointwise convolution to integrate cross-modal semantic information and generate the final fused feature.

[0077] Step 208: Input the fused features into the multi-scale dynamic fusion decoder. The multi-scale dynamic fusion module in the multi-scale dynamic fusion decoder performs progressive feature aggregation and upsampling processing on the fused features to obtain the optical coherence tomography image segmentation prediction map.

[0078] The Multi-Scale Dynamic Fusion Module (MMSF) integrates feature maps of different resolutions and combines dynamic attention mechanisms with deformable convolutions to address the problem of insufficient feature representation caused by target scale differences in optical coherence tomography image segmentation.

[0079] In one embodiment, the optical coherence tomography (OCT) image segmentation method based on a state-space model may further include a process for processing fused features. Specifically, this process includes: a multi-scale dynamic fusion module receiving hierarchical features from different levels and performing channel attention and spatial attention processing on the hierarchical features respectively; concatenating and aligning the processed features, and generating spatial adaptive weights corresponding to the number of levels through convolution and activation functions; using the spatial adaptive weights to perform weighted summation of the hierarchical features at each level to obtain preliminary fused features; inputting the preliminary fused features into a two-level cascaded deformable convolutional layer for deformation modeling and feature calibration, adding the calibrated features to the anchor point features, and outputting the fused features of the current level; progressively upsampling the fused features to restore the image resolution and obtain an OCT image segmentation prediction map.

[0080] Specifically, the encoder generates hierarchical feature maps as follows: Where i represents the i-th level. Before the feature input multi-scale dynamic fusion module, the four branches need to go through a dual attention mechanism to extract features and align feature projections. The specific process can be represented as follows: ; ; ;in, This refers to the number of intermediate channels; For the output generated by the corresponding encoder, 'i' represents the first to fourth branches. The three formulas sequentially pass data to produce the final output. After the features enter the multi-scale dynamic fusion module, they first undergo multi-branch heterogeneous processing, generating spatially adaptive attention weights through convolutional layers. The subsequent two-stage deformable convolutions, through staged deformation modeling and progressive feature calibration, significantly improve the model's robustness to complex geometric deformations. The specific process can be represented as follows: ; ; ; ;in, This indicates the output generated by the corresponding branch; express The inputs correspond to the four branches; This represents the output calculated based on the four inputs.

[0081] The multi-scale dynamic fusion module will output four sets of features. A progressive feature aggregation strategy is employed to achieve the decoding process and form the final output. The four sets of features in the output can be represented as follows: That is, based on four inputs The first one in MSF was calculated The corresponding D() is added to the first input to produce the first fusion result. ,Then Bundle In Replaced as Then calculate again to produce a second output. Then replace them one by one in this way.

[0082] Step 210: Cross-entropy and Dice loss are used as loss functions. The loss functions are used to calculate the loss of the optical coherence tomography (OCT) image segmentation prediction map and optimize the state space model to obtain the final OCT image segmentation result.

[0083] In one embodiment, cross-entropy loss is used to measure the difference between the predicted result and the true label; Dice loss is used to measure the degree of overlap between the predicted region and the true region.

[0084] Since all datasets use two classes of masks (target and background), cross-entropy and Dice loss are primarily used as loss functions. The loss function can be expressed as: ; ; ;in, Represents cross-entropy loss, This indicates Dice's loss. N is a constant, where N = batch size × image height × image width, and these three parameters are input by the user. This represents the true label of the i-th pixel; This represents the original output of the model for the i-th pixel; This represents the Sigmoid activation function; This is a very small smoothing term to prevent the denominator from being zero. To avoid overfitting and to accurately predict lesion areas, the constant can be set to (0.4, 0.6).

[0085] In one embodiment, to verify the optical coherence tomography (OCT) image segmentation method based on the state-space model in this application, an OCT image segmentation task was completed on the mCNV OCT image dataset. The state-space model MF-UNet in this application achieved superior segmentation performance, resulting in clearer boundaries. The experimental results of the state-space model MF-UNet on the OCT dataset are visualized as follows: Figure 4 As shown.

[0086] Specifically, the comparison of experimental results for different methods on the OCT dataset is shown in the table below:

[0087]

[0088] The experimental results of MF-UNet at different scales are shown in the figure below. Figure 5 As shown, the experimental results of the state-space model MF-UNet in this application on the dataset demonstrate that MF-UNet ensures the accuracy of lesion region prediction and improves the precision of optical coherence tomography image segmentation.

[0089] It should be understood that although the steps in the flowchart above are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart above may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0090] In one embodiment, such as Figure 6 As shown, an optical coherence tomography (OCT) image segmentation system based on a state-space model is provided, comprising: an image acquisition module 610, a feature extraction module 620, a feature fusion module 630, a pre-segmentation module 640, and a model optimization and image segmentation module 650, wherein:

[0091] The image acquisition module 610 is used to acquire the optical coherence tomography (OCT) image to be segmented and input the OCT image into the trained state space model. The state space model includes a dual-path encoder, a feature fusion module, and a multi-scale dynamic fusion decoder.

[0092] The feature extraction module 620 is used to capture contextual semantics and global features of the optical coherence tomography image using the first encoding path in the dual-path encoder and extract the first feature; and to process the optical coherence tomography image using the second encoding path in the dual-path encoder and extract the second feature; wherein the first encoding path contains a hybrid module of residual network and Transformer, and the second encoding path contains a sequence of Natten visual state space NVSS modules.

[0093] The feature fusion module 630 is used to receive and fuse the first feature and the second feature to generate a fused feature.

[0094] The pre-segmentation module 640 is used to input the fused features into the multi-scale dynamic fusion decoder. The multi-scale dynamic fusion module in the multi-scale dynamic fusion decoder performs progressive feature aggregation and upsampling processing on the fused features to obtain the optical coherence tomography image segmentation prediction map.

[0095] The model optimization and image segmentation module 650 is used to calculate the loss of the optical coherence tomography (OCT) image segmentation prediction map by using cross-entropy and Dice loss as loss functions, and to optimize the state space model to obtain the final OCT image segmentation result.

[0096] In one embodiment, the feature extraction module 620 is further configured to input the optical coherence tomography image into the residual network in the first encoding path, perform convolution and downsampling processing through each layer of the residual network to obtain a feature sequence; input the feature sequence into the Transformer encoder, model the global dependency relationship between all features in the feature sequence through the self-attention mechanism in the Transformer encoder, and output the feature sequence enhanced by global context; and reassemble the feature sequence enhanced by global context into a feature map as the first feature.

[0097] In one embodiment, the feature extraction module 620 is further configured to input the optical coherence tomography (OCT) image into a second encoding path, perform spatial downsampling and channel transformation through an initial convolutional layer in the second encoding path to obtain initial intermediate features; input the initial intermediate features into at least one cascaded NVSS module, enhance and preserve local detail information through residual blocks containing grouped convolutions, input the local detail information into a local neighborhood attention Natten layer, aggregate adaptive local context information through a sliding window attention mechanism; input the aggregated adaptive local context information into a visual state space (VSS) block, determine the sequence of each path through the scanning path, and obtain long-distance dependencies between features; and determine deep features based on long-distance dependencies as second features.

[0098] In one embodiment, the scanning path includes an original scanning path and an extended scanning path; wherein, the original scanning path includes scanning paths in horizontal, vertical, anti-horizontal, and anti-vertical directions; and the extended scanning path includes scanning paths in diagonal and spiral directions.

[0099] In one embodiment, the feature fusion module 630 is further configured to perform batch normalization, activation, and convolution processing on the first feature, and perform channel attention operation on the processed feature to obtain the first attention feature; perform batch normalization, activation, and convolution processing on the second feature, and perform spatial attention operation on the processed feature to obtain the second attention feature; fuse the first attention feature and the second attention feature, and adjust the channel distribution through convolution to generate the fused feature.

[0100] In one embodiment, the pre-segmentation module 640 is further configured to receive hierarchical features from different levels by the multi-scale dynamic fusion module, and perform channel attention and spatial attention processing on the hierarchical features respectively; concatenate and align the processed features, and generate spatial adaptive weights corresponding to the number of levels through convolution and activation functions; use the spatial adaptive weights to perform weighted summation on the hierarchical features of each level to obtain preliminary fused features; input the preliminary fused features into a two-level cascaded deformable convolutional layer for deformation modeling and feature calibration, add the calibrated features to the anchor features, and output the fused features of the current level; perform progressive upsampling on the fused features to restore the image resolution and obtain the optical coherence tomography image segmentation prediction map.

[0101] In one embodiment, cross-entropy loss is used to measure the difference between the predicted result and the true label; Dice loss is used to measure the degree of overlap between the predicted region and the true region.

[0102] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements an optical coherence tomography (OCT) image segmentation method based on a state-space model. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0103] Those skilled in the art will understand that Figure 7The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0104] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of an optical coherence tomography image segmentation method based on a state-space model.

[0105] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of an optical coherence tomography image segmentation method based on a state-space model.

[0106] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0107] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0108] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for segmenting optical coherence tomography images based on a state-space model, characterized in that, The method includes: An optical coherence tomography (OCT) image to be segmented is acquired, and the OCT image is input into a trained state space model; the state space model includes a dual-path encoder, a feature fusion module, and a multi-scale dynamic fusion decoder. The optical coherence tomography (OCT) image is captured using the first encoding path in the dual-path encoder to capture contextual semantics and global features, and the first feature is extracted. The OCT image is then processed using the second encoding path in the dual-path encoder to extract the second feature. The first encoding path contains a hybrid module of residual network and Transformer, and the second encoding path contains a sequence of Natten visual state space (NVSS) modules. The feature fusion module is used to receive and fuse the first feature and the second feature to generate a fused feature. The fused features are input into the multi-scale dynamic fusion decoder, and the multi-scale dynamic fusion module in the multi-scale dynamic fusion decoder performs progressive feature aggregation and upsampling processing on the fused features to obtain an optical coherence tomography image segmentation prediction map. Cross-entropy and Dice loss are used as loss functions to calculate the loss of the optical coherence tomography (OCT) image segmentation prediction map, and the state space model is optimized to obtain the final OCT image segmentation result.

2. The optical coherence tomography image segmentation method based on a state-space model according to claim 1, characterized in that, The first encoding path in the dual-path encoder is used to capture contextual semantics and global features of the optical coherence tomography image, and the first feature is extracted, including: The optical coherence tomography image is input into the residual network in the first encoding path, and convolution and downsampling are performed through each layer of the residual network to obtain the feature sequence. The feature sequence is input into the Transformer encoder, and the global dependency between all features in the feature sequence is modeled through the self-attention mechanism in the Transformer encoder, and the feature sequence with global context enhancement is output. The feature sequence enhanced by global context is reconstructed into a feature map, which is used as the first feature.

3. The optical coherence tomography image segmentation method based on a state-space model according to claim 1, characterized in that, The optical coherence tomography image is processed using the second encoding path in the dual-path encoder to extract a second feature, including: The optical coherence tomography image is input into the second encoding path, and spatial downsampling and channel transformation are performed through the initial convolutional layer in the second encoding path to obtain initial intermediate features; The initial intermediate features are input into at least one cascaded NVSS module, where local details are enhanced and preserved by residual blocks containing grouped convolutions. The local details are then input into the local neighborhood attention Natten layer, where adaptive local context information is aggregated through a sliding window attention mechanism. The aggregated adaptive local context information is input into the visual state space (VSS) block, and the sequence of each path is determined by scanning the path to obtain the long-distance dependency between features. Based on the aforementioned long-distance dependency, deep features are determined as the second feature.

4. The optical coherence tomography image segmentation method based on a state-space model according to claim 3, characterized in that, The scanning path includes the original scanning path and the extended scanning path; The original scanning path includes scanning paths in the horizontal, vertical, anti-horizontal, and anti-vertical directions; The extended scan path includes diagonal and spiral scan paths.

5. The optical coherence tomography image segmentation method based on a state-space model according to claim 1, characterized in that, The feature fusion module receives and fuses the first feature and the second feature to generate a fused feature, including: The first feature is subjected to batch normalization, activation, and convolution, and the processed feature is subjected to channel attention operation to obtain the first attention feature; The second feature is batch normalized, activated, and convolved, and then spatial attention is applied to the processed feature to obtain the second attention feature. The first attention feature and the second attention feature are fused together, and the channel distribution is adjusted by convolution to generate the fused feature.

6. The optical coherence tomography image segmentation method based on a state-space model according to claim 1, characterized in that, The multi-scale dynamic fusion module in the multi-scale dynamic fusion decoder performs progressive feature aggregation and upsampling on the fused features to obtain an optical coherence tomography (OCT) image segmentation prediction map, including: The multi-scale dynamic fusion module receives hierarchical features from different levels and performs channel attention and spatial attention processing on the hierarchical features respectively. The processed features are concatenated and aligned, and spatial adaptive weights corresponding to the number of layers are generated through convolution and activation functions. The spatial adaptive weights are used to perform a weighted summation of the hierarchical features at each level to obtain preliminary fused features; The preliminary fused features are input into two cascaded deformable convolutional layers for deformation modeling and feature calibration. The calibrated features are added to the anchor features to output the fused features of the current layer. The fused features are progressively upsampled to restore the image resolution and obtain an optical coherence tomography (OCT) image segmentation prediction map.

7. The optical coherence tomography image segmentation method based on a state-space model according to claim 1, characterized in that, The cross-entropy loss is used to measure the difference between the predicted result and the true label; the Dice loss is used to measure the degree of overlap between the predicted region and the true region.

8. A state-space model-based optical coherence tomography image segmentation system, characterized in that, The system includes: The image acquisition module is used to acquire the optical coherence tomography (OCT) image to be segmented and input the OCT image into the trained state space model; the state space model includes a dual-path encoder, a feature fusion module, and a multi-scale dynamic fusion decoder. The feature extraction module is used to capture contextual semantics and global features of the optical coherence tomography image using the first encoding path in the dual-path encoder, and extract the first feature; and to process the optical coherence tomography image using the second encoding path in the dual-path encoder, and extract the second feature; wherein the first encoding path includes a hybrid module of residual network and Transformer, and the second encoding path includes a sequence of Natten visual state space (NVSS) modules; The feature fusion module is used to receive and fuse the first feature and the second feature to generate a fused feature. The pre-segmentation module is used to input the fused features into the multi-scale dynamic fusion decoder. The multi-scale dynamic fusion module in the multi-scale dynamic fusion decoder performs progressive feature aggregation and upsampling processing on the fused features to obtain the optical coherence tomography image segmentation prediction map. The model optimization and image segmentation module is used to calculate the loss of the optical coherence tomography (OCT) image segmentation prediction map by using cross-entropy and Dice loss as loss functions, and to optimize the state space model to obtain the final OCT image segmentation result.

9. The optical coherence tomography image segmentation system based on a state-space model according to claim 8, characterized in that, The feature extraction module is further configured to input the optical coherence tomography image into a residual network in the first encoding path, perform convolution and downsampling processing through each layer of the residual network to obtain a feature sequence; input the feature sequence into a Transformer encoder, model the global dependencies between all features in the feature sequence through the self-attention mechanism in the Transformer encoder, and output a feature sequence enhanced by global context; and reassemble the feature sequence enhanced by global context into a feature map as the first feature.

10. The optical coherence tomography image segmentation system based on a state-space model according to claim 8, characterized in that, The feature extraction module is further configured to input the optical coherence tomography (OCT) image into a second encoding path, perform spatial downsampling and channel transformation through an initial convolutional layer in the second encoding path to obtain initial intermediate features; input the initial intermediate features into at least one cascaded NVSS module, enhance and preserve local detail information through residual blocks containing grouped convolutions, input the local detail information into a local neighborhood attention (NATEN) layer, aggregate adaptive local context information through a sliding window attention mechanism; input the aggregated adaptive local context information into a visual state space (VSS) block, determine the sequence of each path through the scanning path, and obtain long-distance dependencies between features; and determine deep features based on the long-distance dependencies as the second feature.

Images