Dynamic guided gland instance segmentation method and system based on state space model

Abstract

The application provides a kind of dynamic guiding gland instance segmentation method and system based on state space model, it is related to medical image processing technical field, including: obtaining colon histology image, and extracting the multiscale feature of image;Receive user interactive click sequence, generate space-time guide tensor based on click sequence;Fusion is obtained to multiscale feature with space-time guide tensor, and get fusion feature;Again input state space model Mamba, carry out dynamic state regulation, obtain interactive driven feature representation;Further input hierarchical encoder, utilize mask fusion Mamba module to carry out boundary refinement with prediction mask, obtain refined feature;Finally input decoder, and fuse with global feature, output gland instance segmentation mask through mask head.The application realizes the joint modeling and boundary accurate refinement of multiple gland instances in shared latent space, guarantees the segmentation accuracy while significantly improves the interaction efficiency.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a dynamic guided gland instance segmentation method and system based on a state-space model. Background Technology

[0002] Colorectal cancer is a prevalent malignant tumor worldwide, with its main pathological type being colorectal adenocarcinoma. Originating in the lining of the colon or rectum, it is identified by the formation of glandular structures. Glands perform protein and carbohydrate secretion functions in various organ systems; therefore, glandular morphology is a crucial basis for pathologists to assess tumor grade and differentiation. In routine diagnosis, pathologists typically rely on hematoxylin and eosin (H&E) stained sections for glandular histological examination, where the integrity and morphological characteristics of gland formation directly influence the assessment of tumor malignancy. However, precise segmentation of glands on a case-by-case basis on histological images is challenging; manual annotation is not only time-consuming and laborious but also requires a high level of expertise.

[0003] Interactive segmentation methods offer an effective solution to the aforementioned problems by introducing expert click guidance. Existing techniques such as ITIS and RITM typically convert user clicks into guidance maps and fuse them with the image to predict segmentation masks. However, these methods suffer from a key technical drawback when dealing with densely distributed and morphologically varied glandular instances in colon histology images: they struggle to accurately distinguish adjacent glands and maintain clear boundaries while iterating efficiently, leading to mask overlap and blurred boundaries. This is primarily due to their predominantly single-instance segmentation paradigm, requiring complete feature recalculation after each interaction, resulting in high computational costs, and their inability to effectively model long-range dependencies between multiple instances in a shared space. Summary of the Invention

[0004] To address the key technical problem of existing interactive segmentation methods struggling to balance segmentation accuracy and computational efficiency in high-density gland scenarios, this invention introduces the Mamba state-space model to construct a dynamic guidance mechanism. This mechanism enables joint modeling and efficient iterative optimization of multiple gland instances within a shared latent space, thereby significantly reducing computational overhead while maintaining boundary discriminability.

[0005] To achieve the above objectives, a first aspect of the present invention provides a dynamic guided gland instance segmentation method based on a state-space model, comprising: Acquire colonic histological images and extract multi-scale features from the images; Receive a user interaction click sequence and generate a spatiotemporal guidance tensor based on the click sequence; The spatiotemporal guiding tensor is fused with the multi-scale features to obtain the fused features; The fused features are input into the state space model Mamba for dynamic state control, resulting in an interactively driven feature representation. The interactively driven feature representation is input into the hierarchical encoder, and the boundary is refined by combining the prediction mask with the mask fusion Mamba module to obtain the refined features. The refined features are input into the decoder and fused with global features to output a segmentation mask for gland instances via a masking mechanism.

[0006] Furthermore, extracting multi-scale features of the image includes: using a ResNet network to extract multi-layer convolutional features of the colon histology image; The feature decoding module decodes the multi-layer convolutional features to generate a multi-scale feature set, which includes features at different resolutions.

[0007] Furthermore, the generation of the spatiotemporal guiding tensor includes: Extract local features at the user interaction click locations; The click location and interaction steps are encoded as a spatiotemporal guidance tensor, wherein the spatiotemporal guidance tensor contains spatial location and time step information.

[0008] Furthermore, the fused features are input into the Mamba state space model, which includes: Jointly model multiple gland instances in a shared potential space; Interaction-driven feature regulation is achieved through dynamic state updates, where new user interaction clicks only update the interaction guidance branch without re-extracting global features.

[0009] Furthermore, boundary refinement is achieved using the mask fusion Mamba module in conjunction with the predicted mask, including: In hierarchical encoders, predictive masks are used to spatially constrain features; By refining gland instance boundaries layer by layer using a state-space model and a feedforward network, instance distinguishability and boundary accuracy are enhanced.

[0010] Furthermore, the segmentation mask for outputting gland instances via the masking terminal includes: The features output by the decoder are fused with global multi-scale features to obtain the predicted gland instance features; Pixel-level classification is performed using a masking tool, outputting a mask for each gland instance, and the final segmentation result is generated using the maximum channel index.

[0011] A second aspect of the present invention provides a dynamic guided gland instance segmentation system based on a state-space model, comprising: A feature extraction module is used to acquire colon histological images and extract multi-scale features from the images; An interactive guidance generation module is used to receive user interaction click sequences and generate a spatiotemporal guidance tensor based on the click sequences. The feature fusion module is used to fuse the spatiotemporal guiding tensor with the multi-scale features to obtain fused features; The dynamic state control module is used to input the fused features into the state space model Mamba, perform dynamic state control, and obtain interactively driven feature representations. The boundary refinement module is used to input the interactively driven feature representation into the hierarchical encoder, and use the mask fusion Mamba module to combine the prediction mask to refine the boundary and obtain the refined features. The mask generation module is used to input the refined features into the decoder and fuse them with global features, and output the segmentation mask of the gland instance through the mask generator.

[0012] A third aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and running on the processor, wherein the processor executes the program to implement the steps of the dynamic guided gland instance segmentation method based on a state-space model as described in the first aspect of the present invention.

[0013] A fourth aspect of the present invention provides a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the steps of the dynamic guided gland instance segmentation method based on a state-space model as described in the first aspect of the present invention.

[0014] A fifth aspect of the present invention provides a computer program product comprising software code, wherein the program in the software code performs the steps of the state-space model-based dynamically guided gland instance segmentation method described in the first aspect of the present invention.

[0015] Compared with existing technologies, the dynamic guided gland instance segmentation method and system based on a state-space model provided by this invention has the following beneficial effects: (1) This invention is based on the state-space model Mamba and dynamic state regulation. It utilizes the state-space model to jointly model multiple gland instances in a shared latent space and achieves dynamic state updates through its selective mechanism. This design allows the model to update only a lightweight interaction guidance branch when it receives a new user interaction, without repeating forward propagation and global feature extraction. This enables efficient interactive reasoning in complex scenarios, significantly reduces computational costs, and solves the technical problems of multi-instance mask conflicts and high computational overhead in high-density gland scenarios.

[0016] (2) This invention uses a mask fusion Mamba module combined with a predicted mask for boundary refinement. In the hierarchical encoder, the predicted mask from the previous layer is used as a spatial constraint to guide the feature fusion and state update of the current layer. This mask-constrained attention mechanism enables the model to focus on the target boundary region and gradually sharpens the boundary and enhances the feature separability between different gland instances through layer-by-layer iteration of the state space model and feedforward network. This effectively improves the segmentation accuracy of the boundary and the discriminability of the instances, solving the technical problems of blurred gland boundaries and insufficient discriminability. Attached Figure Description

[0017] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute an undue limitation of this disclosure.

[0018] Figure 1 This is a flowchart of the dynamic guided gland instance segmentation method based on a state-space model provided in Embodiment 1 of the present invention; Figure 2 This is a flowchart illustrating the overall process of the dynamic guided gland instance segmentation method based on a state-space model provided in Embodiment 1 of the present invention. Figure 3 This is a schematic diagram of the encoder module operation provided in Embodiment 1 of the present invention; Figure 4 This is a schematic diagram of the interactive gland instance segmentation method provided in Embodiment 1 of the present invention; Figure 5 This is a diagram showing the final result provided in Embodiment 1 of the present invention; Figure 6 This is a schematic diagram of a typical complex scene segmentation result provided in Embodiment 1 of the present invention; Figure 7 This is an architecture diagram of a dynamic guided gland instance segmentation system based on a state-space model provided in Embodiment 2 of the present invention. Detailed Implementation

[0019] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0020] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0021] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0022] All data acquisition in this embodiment is carried out in accordance with laws and regulations and with user consent, and the data is used legally.

[0023] Before describing the embodiments of the present invention in detail, the technical terms involved in the present invention are explained and defined as follows: 1. State Space Model (SSM): This refers to a class of frameworks for sequence modeling that captures long-range dependencies through the recursive updating of hidden states. In this invention, it specifically refers to the Mamba state space model, which, through its selective mechanism, can dynamically focus on relevant information based on input (such as user clicks), and is a core component for achieving efficient long-range dependency modeling.

[0024] 2. Mamba Model: This refers to a state-space model with a selective update mechanism. Compared to the static updates of traditional SSM, Mamba dynamically adjusts the state and output projection weights based on the input semantics. In the interactive multi-instance segmentation of this invention, Mamba jointly represents all gland instances within a shared latent space. User clicks are encoded as spatiotemporal guidance vectors, directly modulating the state and output mappings of Mamba. This allows the model to refine the instance-level mask only through internal state reorganization after each interaction, thereby achieving unified multi-instance modeling and efficient inference.

[0025] 3. Multi-scale features: These refer to the set of features extracted from the input image that contains information at different resolutions (or different receptive fields). In this invention, they refer to the set of representations at different resolutions obtained through different layers of backbone networks such as ResNet, combined with feature decoding and fusion modules. These representations are used to simultaneously carry global semantics and local boundary details, aiming to preserve both the global semantic context and local detail information of the image, thus providing a comprehensive feature foundation for accurate segmentation.

[0026] 4. Spatiotemporal Guiding Tensor: This is a data structure used to encode user interaction click information on an image. It contains not only the spatial location of the click (such as image coordinates) but also the time step (or sequence) information of the interaction. This tensor transforms discrete click sequences into continuous guiding signals rich in spatiotemporal context, dynamically guiding the model to perform segmentation optimization.

[0027] 5. Masked Fusion Mamba Module: This is a core component of the hierarchical encoder in this invention. This module uses the segmentation mask predicted by the previous layer as a spatial constraint (or attention map), fuses it with the multi-scale features of the current layer, and processes it using the Mamba model. Its role is to guide the model to focus computational resources on the target region and uncertain boundaries, achieving round-by-round boundary refinement and instance separation.

[0028] 6. Channel Maximum Value Index (argmax): This refers to an operation used in the model output layer. After the masking tool generates an independent probability map for each gland instance candidate in the channel dimension, this operation selects the channel with the largest response for each pixel position as the final instance label of the pixel. This mechanism avoids overlapping conflicts of multiple instance masks and facilitates the discretization of the multi-instance probability stack into an instance segmentation map.

[0029] Example 1 like Figure 1 This embodiment provides a dynamic guided gland instance segmentation method based on a state-space model, including: Acquire colonic histological images and extract multi-scale features from the images; Receive a user interaction click sequence and generate a spatiotemporal guidance tensor based on the click sequence; The spatiotemporal guiding tensor is fused with the multi-scale features to obtain the fused features; The fused features are input into the state space model Mamba for dynamic state control, resulting in an interactively driven feature representation. The interactively driven feature representation is input into the hierarchical encoder, and the boundary is refined by combining the prediction mask with the mask fusion Mamba module to obtain the refined features. The refined features are input into the decoder and fused with global features to output a segmentation mask for gland instances via a masking mechanism.

[0030] This method constructs a complete technical chain from image feature extraction and user intent encoding to dynamic state regulation and boundary iterative optimization. First, colonic histology images are acquired and their multi-scale features are extracted to preserve rich information from global semantics to local details. Then, user interaction click sequences are received and a spatiotemporal guidance tensor is generated, transforming discrete user commands into continuous guidance signals containing spatiotemporal context. Next, the spatiotemporal guidance tensor and multi-scale features are fused and input into the state space model Mamba. This model is used to jointly model multiple gland instances in a shared latent space and achieves interaction-driven dynamic state regulation through its selective mechanism. Subsequently, the regulated features are input into a hierarchical encoder, and the boundary is refined using a mask fusion Mamba module combined with a prediction mask, achieving iterative optimization of prediction, guidance, and refinement. Finally, the refined features are fused with global features by a decoder, and the final segmentation mask is output through a masking module. Through the synergy of the above steps, this method solves the technical problem of existing technologies struggling to balance segmentation accuracy and interaction efficiency in high-density gland scenarios, achieving efficient and accurate instance segmentation.

[0031] Specifically, extracting multi-scale features of the image includes: using a ResNet network to extract multi-layer convolutional features of the colon histology image; The feature decoding module decodes the multi-layer convolutional features to generate a multi-scale feature set, which includes features at different resolutions.

[0032] The step of extracting multi-layer convolutional features using a ResNet network and generating a multi-scale feature set through a feature decoding module leverages the hierarchical structure of deep convolutional networks to capture information from different abstract levels in an image, ranging from low-level details to high-level semantics. This aims to fully utilize the strong semantic representation capabilities of deep networks and the rich spatial details of shallow networks, providing a robust and comprehensive feature foundation for subsequent accurate segmentation. It effectively solves the technical problem of boundary information loss caused by single or insufficient feature representation capabilities.

[0033] Specifically, the generation of the spatiotemporal guiding tensor includes: Extract local features at the user interaction click locations; The click location and interaction steps are encoded as a spatiotemporal guidance tensor, wherein the spatiotemporal guidance tensor contains spatial location and time step information.

[0034] This design extracts local features at the click location and encodes the location and step into a spatiotemporal guidance tensor. It not only focuses on the local image context at the user's click point but also records the spatial order and temporal evolution of the interaction. This transformation converts a series of discrete click events into a structured guidance signal containing spatiotemporal dimensions, enabling the model to dynamically perceive and understand the user's corrective intent sequence. This allows for targeted optimization of specific instances, avoiding confusion and loss of interaction information.

[0035] Specifically, inputting the fused features into the state space model Mamba includes: Jointly model multiple gland instances in a shared potential space; Interaction-driven feature regulation is achieved through dynamic state updates, where new user interaction clicks only update the interaction guidance branch without re-extracting global features.

[0036] This method employs joint modeling in a shared latent space and dynamic state updates to achieve feature modulation. The underlying principle is that the Mamba state-space model treats fused features as a sequence, and its internal states constitute a global memory of the segmented scene. When a new interaction arrives, the model incrementally updates its internal state using only the new click information as input, without recalculating global image features. The advantage of this mechanism lies in separating computationally expensive global feature extraction from lightweight interaction-guided updates, thereby significantly reducing the computational cost per interaction and solving the technical problems of high interaction latency and large computational redundancy in existing methods.

[0037] Specifically, boundary refinement using the mask fusion Mamba module in conjunction with the predicted mask includes: In hierarchical encoders, predictive masks are used to spatially constrain features; By refining gland instance boundaries layer by layer using a state-space model and a feedforward network, instance distinguishability and boundary accuracy are enhanced.

[0038] A mask-constrained attention mechanism is introduced by combining the predicted mask with the Mamba module for boundary refinement. This module uses the predicted mask from the previous layer as a spatial weight map to selectively fuse multi-scale features of the current layer, focusing the model's attention on the foreground target and uncertain boundary regions. Deep processing is then performed via a state-space model and a feedforward network. This design achieves iterative boundary sharpening and instance separation layer by layer, effectively solving the mask adhesion and conflict problems caused by blurred boundaries and similar features between dense glands, significantly improving the visual accuracy and instance discrimination of the segmentation results.

[0039] Specifically, the segmentation mask for outputting gland instances via the masking terminal includes: The features output by the decoder are fused with global multi-scale features to obtain the predicted gland instance features; Pixel-level classification is performed using a masking tool, outputting a mask for each gland instance, and the final segmentation result is generated using the maximum channel index.

[0040] After fusing the decoded features with global features, a masking mechanism is used to output the result indexed by the maximum channel value. The working principle is that the decoder is responsible for restoring the spatial resolution of the feature map, while the final masking mechanism generates an independent probability map for each candidate instance along the channel dimension. The maximum channel value indexing operation selects the channel with the highest probability for each pixel as its assigned instance. This output method mathematically guarantees that each pixel has exactly one instance label, avoiding overlap of multiple instance masks on the same pixel, and ensuring strict separation and topological correctness of the instance segmentation results.

[0041] In one specific embodiment, this embodiment provides a dynamic guided gland instance segmentation method based on a state-space model, comprising: Input colon histology images I And extract multi-scale features , used for subsequent segmentation processing.

[0042] Receive user interaction click sequence The local features at the click location are sampled and converted into a guiding vector.

[0043] The click location and interaction steps are encoded into a spatiotemporal guidance tensor and combined with multi-scale features. After fusion, the input is the Mamba state space model, enabling interactive-driven dynamic state control.

[0044] The Masked Fusion Mamba module is used in the hierarchical encoder to progressively refine the gland boundaries by using prediction masks to constrain the attention region.

[0045] By fusing global features and guiding tensors through a decoder, a set of segmentation masks for multiple instances is output. .

[0046] When a new click input occurs, only the interactive guidance branch is updated without re-extracting global features, achieving efficient interactive iterative segmentation.

[0047] During training, the original histological image and its corresponding gland instance ground truth mask are first obtained. The original image is then input into the ResNet network to extract multi-layer convolutional features, and a multi-scale feature set is obtained through the feature decoding module. Simultaneously, simulated user clicks are received as input, local features are sampled at the click locations, and a spatiotemporal guidance tensor with spatial location and time step information is constructed. This guidance tensor is then fused with multi-scale features and input into the Mamba state space model as interactive guidance features during the training phase.

[0048] During the inference process, the original image is directly input into the ResNet network and feature decoding module to obtain fixed multi-scale features. User interaction clicks dynamically generate a spatiotemporal guidance tensor, and it is then compared with... The merged data is then fed into the Mamba state space model. During the segmentation iteration, new clicks only update the interaction guidance branch without needing to re-extract global features, thus achieving efficient interaction inference.

[0049] The fused features are input into the trained Mamba state space model, and then sequentially pass through Masked Fusion Mamba, State Space Model (SSM), and Feedforward Network (FFN) in the hierarchical encoder. The attention region is constrained by the prediction mask, which gradually refines the boundaries of gland instances and enhances instance discrimination.

[0050] The features output by the encoder are input into the decoding module, which decodes the fused interactive features and global features to obtain the predicted gland instance features. Then, the mask of the gland instance is output through the Mask Head, and the final segmentation result is generated in argmax mode.

[0051] In this embodiment, the training process for gland instance segmentation of colon histology images based on the state-space model Mamba dynamic guidance model is as follows: Before performing gland instance mask prediction, the entire neural network needs to be trained. During training, multi-scale features of the original image are first acquired. It receives user click sequences, extracts local features of the click locations, and encodes them into a spatiotemporal guidance tensor that combines spatial and temporal dimensions. This represents the guiding feature generated at the current time step t.

[0052] Next, the spatiotemporal guiding tensor needs to be fused with multi-scale features and input into the Mamba state space model. In the encoder stage, the Masked Fusion Mamba module first uses a prediction mask to perform spatial constraint fusion of features, and then, combined with the results of the previous layer's state update, gradually refines the boundaries of gland instances; where t represents the current interaction time step. This represents the prediction mask obtained at time step t.

[0053] The features output from the encoder are input into the decoder module, and under the combined effect of global features and interactive guidance, predicted gland instance features are generated; whereby, This represents the interactive update feature at time step t. This indicates the segmentation prediction result for the corresponding time period.

[0054] The multi-scale features of the original image are decoded and fused, and then input together with the predicted interaction features into the Mask Head to obtain the mask for each gland instance. This represents the final output set of instance features, which includes the segmentation results of gland instances.

[0055] In this embodiment, the process of obtaining the gland instance segmentation result is as follows: the training objective is to accurately identify the location and boundary of gland instances based on the joint modeling of user interaction click sequences and image features. Unlike existing static segmentation methods, the DG-Mamba model proposed in this embodiment introduces a dynamic guidance mechanism, that is, only the interaction branch is updated during inference without repeating global calculations, thereby effectively reducing computational overhead. Through a hierarchical encoder and a Masked FusionMamba module, the model can progressively refine the boundaries, avoiding mask conflicts between adjacent glands.

[0056] Therefore, the DG-Mamba network model proposed in this embodiment can achieve efficient interactive segmentation of glands in colon histological images, generate segmentation results with multiple instances and clear boundaries, and significantly improve interactive efficiency and robustness while ensuring accuracy.

[0057] Figure 2 The model framework provided in this embodiment includes: Image representing the input raw histological image; UserClick Extraction representing the user click extraction module, used to convert foreground or background clicks into guiding signals; Backbone representing the ResNet feature extraction network, used to perform convolutional encoding on the raw image; Feature Decoder representing the feature decoding of the backbone network into multi-scale representations of different resolutions; Multi-Scale Features representing the multi-scale feature set generated by the feature decoder; and Feature Fusion representing the feature fusion module, used to fuse multi-scale features with click guidance information to obtain fused features. .

[0058] In the core modeling part, the Masked Fusion Mamba module, together with the SSM (State Space Model) and FFN (Feedforward Network), constitutes the Encoder, which is used to dynamically update features and refine boundaries in a shared latent space with user guidance. Fusion Mamba and FFN constitute the Decoder, which is used to progressively restore spatial resolution and generate segmentation representations. Finally, the Mask Head module outputs gland instance masks based on the decoded features, realizing the interactively optimized segmentation result.

[0059] First, the original image, a colon histology image, is initialized. Then, convolutional features are extracted from the original image, and multi-scale features are generated using a feature decoding module. Simultaneously, user interaction clicks are received, and local features at the click locations are extracted and encoded into a spatiotemporal guidance tensor. Next, the guidance tensor is fused with the multi-scale features and input into the Mamba state-space model, such as... Figure 3 The segmentation is performed using the method proposed in this disclosure. The specific steps of the segmentation method are as follows: (1) Input the original image into the trained ResNet network to obtain the original image features, and generate a multi-scale feature set through the feature decoding module. (2) Receive user interaction clicks, extract and encode the local features of the click location into a spatiotemporal guidance tensor that combines space and time, and input the guidance tensor into the Mamba state space model after fusing it with multi-scale features to obtain the predicted gland instance features; (3) In the hierarchical encoder, use the MaskedFusion Mamba module to perform spatial constraints and dynamic updates on the interaction features, gradually refine the gland boundary and enhance instance discrimination; (4) Input the encoded gland instance features into the decoding module and fuse them with global multi-scale features, and output the mask information of the gland instance and the final instance segmentation result through MaskHead.

[0060] The steps described above are explained in detail below. During training, the multi-scale convolutional features of the original colon histological image are first obtained. (Obtained from the ResNet backbone and feature decoding / fusion process), and simultaneously, based on simulated user clicks (foreground / background), local semantics are sampled at the corresponding locations and encoded as a space-time joint spatiotemporal guided tensor. ; then Multiscale features After fusion, the data is fed into the Mamba state-space model as an interaction guidance signal during the training phase. This drives the network to jointly model multiple gland instances within a shared latent space and refine the boundaries layer by layer (the hierarchical encoder includes Masked-Fusion Mamba, SSM, and FFN). During the interaction iteration process, only the guidance branch is updated, and cached global image features are reused, thus avoiding the need to re-extract features for each click and significantly reducing computational redundancy. The inference phase follows the same mechanism, using measured click sequences instead of simulated clicks for gradual optimization (the paper describes a maximum of 3 training steps per instance and a minimum testing click budget). The network ultimately outputs a multi-instance probability map by the decoder, which combines global multi-scale features with interactive guidance, and generates a discrete instance mask (assigning pixels to the most probable instance) using channel argmax.

[0061] This embodiment uses the selective state-space model (Mamba) as the core dynamics, and its basic form is as follows: (1) (2) This represents the hidden state vector at time step t. This represents the projection matrix that is dynamically updated based on user interaction. This is the predicted output for the corresponding time step. This represents the state transition matrix, used to retain and filter historical information. This represents the input projection matrix, which will project the current... The role of input is mapped to state update. To establish instance-aware global guidance information during the interaction process, click features are aggregated to obtain a click-related contextual representation, as shown in the following formula: (3) Indicates the first Each user click corresponds to a guidance vector. The scale number represents the feature set at multiple scales. Refers to a feature map at a certain scale. Indicates the first The location index of each user's click. Through this design, the user's click interaction can directly affect the mapping process from state to output, thereby achieving fine-grained control over the gland segmentation results.

[0062] Output projection matrix Dynamically related to interaction This is used to inject user click intent into the state-to-output mapping process, and its mathematical expression is as follows: (4) As clicks accumulate, Iterative updates are performed to focus on the foreground region indicated by the user. Meanwhile, Masked-FusionMamba uses the upper-layer prediction mask. As a spatial constraint, it is used for multi-scale fusion and state updating, thereby improving instance separability and boundary sensitivity. Its basic form is as follows: (5) (6) (7) in, Indicates the first The layer's guiding vector set. During the decoding phase, using... Indicates the first The foreground mask features of each layer are used to gradually recover the spatial resolution through hierarchical processing with Fusion Mamba and a feedforward network (FFN), and to generate the corresponding instance probability map. This process can be expressed by the following formula: (8) (9) The above design ensures that: (1) long-range dependencies and multi-instance relationships are modeled uniformly in the shared potential space; (2) each interaction only updates the lightweight guide branch instead of the entire network forward, satisfying the real-time interaction; (3) cross-scale fusion of mask constraints improves the boundary differentiation and robustness of crowded / adjacent glands.

[0063] In summary, this embodiment employs multi-scale features, spatiotemporal guided tensors, dynamic state updates of the Mamba state-space model, cross-scale fusion of mask constraints, and an overall process of decoding and channel argmax output. This achieves interactive controllability and multi-instance consistency, while avoiding the high computational overhead caused by repeated full-network inference. Its advantages in accuracy and efficiency have been verified on the GlaS and CRAG datasets.

[0064] In this embodiment, the DG-Mamba model was tested using the GlaS and CRAG datasets and compared with other state-of-the-art segmentation methods on these datasets. Three categories of metrics were used to evaluate segmentation performance: (1) Object F1, which measures the detection accuracy of a single gland instance; (2) Object Dice, which evaluates the volumetric overlap accuracy of gland segmentation results; and (3) Mean Intersection over Union (mIoU), which evaluates the degree of overlap between the segmentation mask and the ground truth. In addition, the model's inference time and memory consumption were recorded to comprehensively compare efficiency and practicality.

[0065] The GlaS dataset contains a total of 165 H&E-stained colon histological images, with 85 used for training and 80 for testing. The test set is further divided into Test A (60 images) and Test B (20 images). Each image provides accurate gland boundary instance annotations. The CRAG dataset contains a total of 213 H&E-stained histological images, with 173 used for training and 40 for testing. Each image is equipped with instance-level segmentation Ground Truth annotations.

[0066] All experiments were conducted on an NVIDIA RTX 3090 GPU. DG-Mamba was implemented using PyTorch 2.0.0 with the SGD optimizer and a learning rate of 2.5 × 10⁻⁶. -5 The training process was iterated for 100 cycles. During training, the number of simulated clicks was 3, and during testing, the interaction click budget was 10.

[0067] The proposed method was compared with other state-of-the-art methods on the GlaS and CRAG datasets. The experimental results of interactive and non-interactive segmentation methods are shown in Table 1, and the ablation results of key modules are shown in Table 2.

[0068] Table 1. Comparative experimental results of interactive and non-interactive segmentation methods

[0069] Table 2. Ablation results of key modules

[0070] Compared with other algorithms, the scheme described in this embodiment can achieve more accurate segmentation results.

[0071] Compared with existing interactive gland segmentation methods, the DG-Mamba model proposed in this invention has significant advantages in terms of parameter scale, inference speed, and memory usage. As shown in Table 3, the method of this invention can significantly improve inference efficiency while ensuring segmentation accuracy and greatly reduce computational resource consumption, thus making it more suitable for practical applications in clinical interactive scenarios.

[0072] Table 3. Efficiency Comparison Results of Different Models

[0073] Example 2 like Figure 7 As shown, this embodiment provides a dynamic guided gland instance segmentation system based on a state-space model, including: A feature extraction module is used to acquire colon histological images and extract multi-scale features from the images; An interactive guidance generation module is used to receive user interaction click sequences and generate a spatiotemporal guidance tensor based on the click sequences. The feature fusion module is used to fuse the spatiotemporal guiding tensor with the multi-scale features to obtain fused features; The dynamic state control module is used to input the fused features into the state space model Mamba, perform dynamic state control, and obtain interactively driven feature representations. The boundary refinement module is used to input the interactively driven feature representation into the hierarchical encoder, and use the mask fusion Mamba module to combine the prediction mask to refine the boundary and obtain the refined features. The mask generation module is used to input the refined features into the decoder and fuse them with global features, and output the segmentation mask of the gland instance through the mask generator.

[0074] In one specific embodiment, this embodiment provides a dynamic guided gland instance segmentation system based on a state-space model, comprising: The ResNet network module is used to extract features from input histological images and obtain convolutional features at different resolutions.

[0075] The feature decoding and fusion module is used to decode the multi-layer features of ResNet into multi-scale representations and perform feature fusion between the three scales {f32, f16, f8} to enhance the representation of complex glandular structures.

[0076] The Mamba state-space model module is used to combine the spatiotemporal guidance tensor generated by user interaction clicks to jointly model multiple gland instances in a shared latent space, and achieve efficient interactive segmentation through dynamic state updates.

[0077] The hierarchical encoder module employs Masked Fusion Mamba, State Space Model (SSM), and Feedforward Network (FFN) to refine gland instance boundaries layer by layer on multi-scale features, thereby improving instance discriminability and boundary accuracy.

[0078] The Decoding and Mask Head module is used to fuse the encoded interactive features with global features, output the mask of each gland instance, and generate the final segmentation result through pixel-level classification.

[0079] Example 3 Embodiment 3 of the present invention provides an electronic device.

[0080] An electronic device includes a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, it implements the steps in the dynamic guided gland instance segmentation method based on a state-space model as described in Embodiment 1 of the present invention.

[0081] The detailed steps are the same as those of the dynamic guided gland instance segmentation method based on the state space model provided in Example 1, and will not be repeated here.

[0082] Example 4 Embodiment 4 of the present invention provides a computer-readable storage medium.

[0083] A computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of the dynamic guided gland instance segmentation method based on a state-space model as described in Embodiment 1 of the present invention.

[0084] The detailed steps are the same as those of the dynamic guided gland instance segmentation method based on the state space model provided in Example 1, and will not be repeated here.

[0085] Example 5 Embodiment 5 of the present invention provides a computer program product.

[0086] A computer program product includes software code, wherein the program in the software code performs the steps of the dynamic guided gland instance segmentation method based on a state-space model as described in Embodiment 1 of the present invention.

[0087] The detailed steps are the same as those of the dynamic guided gland instance segmentation method based on the state space model provided in Example 1, and will not be repeated here.

[0088] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0089] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0090] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0091] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0092] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0093] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0094] The above description is merely a preferred embodiment of this practice and is not intended to limit the scope of this practice. Various modifications and variations can be made to this practice by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this practice should be included within the protection scope of this practice.

Claims

1. A dynamic guided gland instance segmentation method based on a state-space model, characterized in that, include: Acquire colonic histological images and extract multi-scale features from the images; Receive a user interaction click sequence and generate a spatiotemporal guidance tensor based on the click sequence; The spatiotemporal guiding tensor is fused with the multi-scale features to obtain the fused features; The fused features are input into the state space model Mamba for dynamic state control, resulting in an interactively driven feature representation. The fusion feature input to the state space model Mamba includes: jointly modeling multiple gland instances in a shared latent space; and achieving interaction-driven feature regulation through dynamic state updates. When a new interaction arrives, the model only takes the new click information as input and incrementally updates its internal state without recalculating the global multi-scale features. The interactively driven feature representation is input into the hierarchical encoder, and the boundary is refined by combining the prediction mask with the mask fusion Mamba module to obtain the refined features. The refined features are input into the decoder, and the features output by the decoder are fused with global multi-scale features. The segmentation mask of the gland instance is output through the masking terminal. The generated spatiotemporal guiding tensor includes: Extract local features at the user interaction click locations; The local features of the click location are extracted and encoded into a spatiotemporal guidance tensor that combines spatial and temporal information, wherein the spatiotemporal guidance tensor contains spatial location and time step information.

2. The method as described in claim 1, characterized in that, Extracting multi-scale features from the image includes: using a ResNet network to extract multi-layer convolutional features from the colon histology image; The feature decoding module decodes the multi-layer convolutional features to generate a multi-scale feature set, which includes features at different resolutions.

3. The method as described in claim 1, characterized in that, Boundary refinement using the Mamba module for mask fusion combined with the predicted mask includes: In hierarchical encoders, predictive masks are used to spatially constrain features; By refining gland instance boundaries layer by layer using a state-space model and a feedforward network, instance distinguishability and boundary accuracy are enhanced.

4. The method as described in claim 1, characterized in that, The segmentation mask for outputting gland instances via the masking terminal includes: The features output by the decoder are fused with global multi-scale features to obtain the predicted gland instance features; Pixel-level classification is performed using a masking tool, outputting a mask for each gland instance, and the final segmentation result is generated using the maximum channel index.

5. A dynamic guided gland instance segmentation system based on a state-space model, characterized in that, include: A feature extraction module is used to acquire colon histological images and extract multi-scale features from the images; An interactive guidance generation module is used to receive user interaction click sequences and generate a spatiotemporal guidance tensor based on the click sequences. The feature fusion module is used to fuse the spatiotemporal guiding tensor with the multi-scale features to obtain fused features; The dynamic state control module is used to input the fused features into the state space model Mamba, perform dynamic state control, and obtain interactively driven feature representations. The fusion feature input to the state space model Mamba includes: jointly modeling multiple gland instances in a shared latent space; and achieving interaction-driven feature regulation through dynamic state updates. When a new interaction arrives, the model only takes the new click information as input and incrementally updates its internal state without recalculating the global multi-scale features. The boundary refinement module is used to input the interactively driven feature representation into the hierarchical encoder, and use the mask fusion Mamba module to combine the prediction mask to refine the boundary and obtain the refined features. The mask generation module is used to input the refined features into the decoder, fuse the features output by the decoder with global multi-scale features, and output the segmentation mask of the gland instance through the mask generator. The generated spatiotemporal guiding tensor includes: Extract local features at the user interaction click locations; The local features of the click location are extracted and encoded into a spatiotemporal guidance tensor that combines spatial and temporal information, wherein the spatiotemporal guidance tensor contains spatial location and time step information.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the steps of the dynamic guided gland instance segmentation method based on the state space model as described in any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the dynamic guided gland instance segmentation method based on the state space model as described in any one of claims 1 to 4.

8. A computer program product, comprising software code, characterized in that, The program in the software code performs the steps of the dynamic guided gland instance segmentation method based on the state space model as described in any one of claims 1 to 4.

Images