A method and system for extracting regions of interest in images based on the Mamba architecture

Abstract

This invention provides a method and system for extracting regions of interest (ROIs) in images based on the Mamba architecture, belonging to the field of image processing technology. The method includes: acquiring an image to be extracted; performing multi-scale feature extraction on the image through a local enhancement module; generating multi-scale semantic enhancement features through a cross-scale self-attention module and performing dimensionality reduction processing through a feature transformation module; generating local detail features through an adaptive detail enhancement module; performing global context enhancement processing on the local detail features through a pyramid pooling module to generate context enhancement features; upsampling the context enhancement features; fusing the upsampled context enhancement features through an adaptive global-local fusion gating module; and extracting the image based on the decoded fused features. This invention improves image segmentation accuracy, features a lightweight model with fast inference, and effectively solves the problems of blurred image boundaries and varying scales.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for extracting regions of interest in images based on the Mamba architecture. Background Technology

[0002] Image region of interest (ROI) extraction is a key task in computer vision, widely used in fields such as medical imaging. However, ROIs generally suffer from common problems such as blurred boundaries and varying scales, leading to reliance on manual intervention in existing methods. This results in limitations such as inaccurate localization, low processing efficiency, and coarse boundaries, making it difficult to achieve high-precision automated processing. Taking colon polyp extraction as an example, colon cancer is one of the most common malignant tumors, and its early diagnosis and treatment are of great significance to patients' survival rate and quality of life. However, in the fields of examination and surgery, the automatic localization of lesions based on image feedback remains a pressing problem, mainly relying on the doctor's experience and visual observation, which suffers from inaccurate localization, long processing times, and inability to accurately delineate boundaries.

[0003] In the early exploratory phase, methods such as Canny edge extraction and fuzzy C-means clustering were used. Subsequently, deep learning methods such as UNet, selective feature aggregation network, parallel back attention network, and Transformer-based architecture were widely used in image segmentation. Recently, the Mamba model has also been introduced into this field due to its linear time series modeling and selective state space mechanism.

[0004] However, CNN-based models are insufficient in long-range modeling capabilities, while Transformers are limited by secondary computational complexity, making it difficult to achieve a balance between segmentation accuracy and processing efficiency, especially for images with blurred boundaries and large scale variations. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide an image region of interest extraction method based on the Mamba architecture, which can solve the problems that the existing CNN-based models are insufficient in long-distance modeling capabilities, while the feature transformation module is constrained by the complexity of secondary computation, making it difficult to achieve a balance between segmentation accuracy and processing efficiency, especially for images with blurred boundaries and large scale changes.

[0006] A first aspect of this invention proposes a method for extracting regions of interest (ROIs) from images based on the Mamba architecture, comprising:

[0007] S1: Obtain the image to be extracted.

[0008] S2: Through the local enhancement module, multi-scale feature extraction is performed on the image to be extracted to obtain a multi-scale feature map.

[0009] The multi-scale feature map includes shallow detail features and deep semantic features.

[0010] S3: Through the cross-scale self-attention module, the shallow detail features are optimized layer by layer by using deep semantic features to generate multi-scale semantic enhancement features, and then dimensionality reduction is performed through the feature transformation module.

[0011] S4: Through the adaptive detail enhancement module, semantic enhancement features at a certain size are used as anchors to fuse semantic enhancement features at other sizes to generate local detail features.

[0012] S5: Through the pyramid pooling module, global context enhancement processing is performed on local detailed features to generate context-enhanced features.

[0013] S6: Upsample the feature map corresponding to the context-enhanced features to gradually restore the feature map resolution.

[0014] S7: By using an adaptive global-local fusion gating module, skip connections are strengthened, and the upsampled context enhancement features are fused to obtain decoded fusion features.

[0015] S8: Image extraction is performed based on decoding and fusion features.

[0016] A second aspect of this invention provides an image region of interest extraction system based on the Mamba architecture, comprising: a processor and a memory;

[0017] The memory stores a program or instructions that can run on the processor, and when the program or instructions are executed by the processor, they implement the steps of the image region of interest extraction method based on the Mamba architecture of the first aspect.

[0018] A third aspect of the present invention provides a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the image region of interest extraction method based on the Mamba architecture of the first aspect.

[0019] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0020] In this embodiment of the invention, a feature extraction system constructed using multi-layered local enhancement modules overcomes the bottleneck of CNNs in long-range dependency modeling and fully leverages the linear computational complexity advantage of the Mamba architecture in sequence modeling, fundamentally laying the foundation for high accuracy and efficiency. Based on this, the synergistic effect of cross-scale attention mechanisms and adaptive detail enhancement modules achieves accurate perception of multi-scale semantics and detailed reconstruction of blurred boundaries. The adaptive global-local fusion gating module further enhances the feature representation capability for images with blurred boundaries and varying scales, thus achieving the optimal balance between segmentation accuracy and processing efficiency. Simultaneously, based on the efficient sequence modeling capabilities of the Mamba architecture, the fusion advantages of combined local perturbation feature enhancement, multi-scale attention mechanisms, and multi-level multi-scale feature extraction enable precise localization of image regions. Attached Figure Description

[0021] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0022] Figure 1 This is a flowchart illustrating an image region of interest extraction method based on the Mamba architecture provided in an embodiment of the present invention.

[0023] Figure 2 This is a schematic diagram of an image region of interest extraction model based on the Mamba architecture provided in an embodiment of the present invention.

[0024] Figure 3 This is a schematic diagram of the structure of a local enhancement module provided in an embodiment of the present invention.

[0025] Figure 4 This is a schematic diagram of the structure of a cross-scale self-attention module provided in an embodiment of the present invention.

[0026] Figure 5 This is a schematic diagram of the structure of an adaptive detail enhancement module provided in an embodiment of the present invention.

[0027] Figure 6 This is a schematic diagram of the structure of an adaptive global-local fusion gating module provided in an embodiment of the present invention.

[0028] Figure 7 This is a schematic diagram of an image region of interest extraction system based on the Mamba architecture provided in an embodiment of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] The image region of interest extraction method based on the Mamba architecture provided by the present invention will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0031] This invention provides a method for extracting regions of interest (ROI) from images based on the Mamba architecture. First, the architecture of the Mamba-based ROI extraction model is introduced.

[0032] Reference manual attached Figure 2 The diagram illustrates an image region of interest extraction model based on the Mamba architecture provided by an embodiment of the present invention.

[0033] It should be noted that, in Figure 2 In this diagram, the original image serves as the network's input data; patch embedding is the image block embedding module, used to convert the original image into initial feature blocks; the local enhancement module x2 represents a module group containing two local enhancement modules, which is the core unit for multi-scale feature extraction, corresponding to the output of four multi-scale feature maps:

[0034] It is the output feature of the first layer local enhancement module group (denoted as f1), where H and W are the height and width of the original image, and C is the initial number of feature channels.

[0035] It is the output feature of the second-layer local enhancement module group (denoted as f2), with 2C channels and a size reduced to 1 / 8 of the original.

[0036] It is the output feature of the third-layer local enhancement module group (denoted as f3), with 4C channels and a size reduced to 1 / 16 of the original.

[0037] It is the output feature of the fourth layer local enhancement module group (denoted as f4), with 8C channels and a size reduced to 1 / 32 of the original.

[0038] Specifically, the patch merging module is a feature downsampling module used to reduce the size of the feature map and increase the number of channels; the cross-scale self-attention module is used to guide the optimization of features; the adaptive detail enhancement module is used to fuse multi-scale semantic enhancement features; the pyramid pooling module is used for global context enhancement processing; and the adaptive global-local fusion gating module is used to strengthen skip connections and fuse features.

[0039] Furthermore, Figure 2 The connections clearly illustrate the network's feature processing flow: the original image is connected to patch embedding; the output of patch embedding is connected to the first-layer local enhancement module x2, whose output f1 is connected to the first cross-scale self-attention module, and this output is also connected to the first patch merge; the output of the first patch merge is connected to the second-layer local enhancement module x2, whose output f2 is connected to the second cross-scale self-attention module, and this output is also connected to the second patch merge; the output of the second patch merge is connected to the third-layer local enhancement module x2, whose output f3 is connected to the third cross-scale self-attention module, and this output is also connected to the third patch merge; the output of the third patch merge is connected to the fourth-layer local enhancement module x2, whose output f4 is connected to the fourth cross-scale self-attention module; the outputs of the four cross-scale self-attention modules are collectively connected to the adaptive detail enhancement module x4; the output of the adaptive detail enhancement module x4 is connected to the pyramid pooling module; the output of the pyramid pooling module is connected to the corresponding... An adaptive global-local fusion gating module, the output of which is sequentially connected to the corresponding... , , The adaptive global-local fusion gating module ultimately outputs the region of interest image.

[0040] Reference manual attached Figure 1 The diagram illustrates a flowchart of an image region of interest extraction method based on the Mamba architecture provided by an embodiment of the present invention.

[0041] This invention provides a method for extracting regions of interest from images based on the Mamba architecture, which may include the following steps:

[0042] S1: Obtain the image to be extracted.

[0043] S2: The local enhancement module (VSSDG, Visual State Space with Style-Driven Globalaugmentation) is used to extract multi-scale features from the image to be extracted, resulting in a multi-scale feature map.

[0044] Among them, the local enhancement module is a multi-scale feature extraction core unit built on the VMamba architecture. It consists of multiple layers of local enhancement modules stacked to form a downsampling structure. Each group contains two local enhancement modules, and downsampling is achieved between adjacent groups through patch merging.

[0045] The VMamba architecture represents a key improvement over the Mamba architecture for visual tasks, while its overall architecture remains rooted in the Mamba architecture. The VMamba architecture resolves a fundamental contradiction in the original Mamba architecture when processing two-dimensional images: flattening a two-dimensional spatial structure into a one-dimensional sequence destroys its inherent spatial locality and directionality.

[0046] Furthermore, each local enhancement module embeds a Local Sequence-wisestyle transformation Augmentation (LSA) module to randomly reparameterize the feature mean and variance to introduce local randomness. At the same time, it combines a state-space model to capture long-distance dependencies and convolution operations to extract local details. Finally, it outputs a multi-scale feature map (f1-f4) covering shallow detail features and deep semantic features, providing a foundation for subsequent cross-scale optimization.

[0047] The multi-scale feature map includes shallow detail features and deep semantic features.

[0048] Among them, shallow detail features are rich in location and boundary information but have weak semantic expression, while deep semantic features have rich global semantics but low spatial resolution, making it difficult to accurately locate edges.

[0049] Specifically, based on the VMamba encoder architecture, this invention introduces and replaces the SS2D module in VMamba with local sequence style transformation enhancement, constructs a local enhancement module, and builds a downsampling hierarchy structure by stacking local enhancement modules.

[0050] In one possible implementation, S2 specifically includes sub-steps S201 and S202:

[0051] S201: Divide the image to be extracted into patches and embed features.

[0052] It should be noted that the processing of input features by each encoding layer involves two key dimensions: capturing long-distance dependencies and global context with the help of state-space model units, and extracting local detailed features and achieving inter-layer propagation by relying on convolutional operations.

[0053] S202: Through a multi-layer local enhancement module, multi-level feature extraction and downsampling are performed on the image to be extracted after embedding features, and multi-scale feature maps (including shallow detail features and deep semantic features) are extracted step by step.

[0054] It should be noted that, as Figure 2 As shown, the "multi-layer" in the multi-layer local enhancement module refers to the fact that multiple local enhancement modules are stacked multiple times in the entire architecture. Each local enhancement module at each layer extracts feature maps at different scales to form multi-scale feature maps.

[0055] In one possible implementation, the local enhancement module includes a local sequence enhancement module.

[0056] It should be noted that the local sequence enhancement module, taking into account the selective scanning mechanism in Mamba, improves the VSS block in the Mamba module to a VSSDG block. By replacing the original SS2D operation with the local sequence enhancement module, it explicitly suppresses domain-specific features in the Mamba input correlation matrix. This module aims to further improve the model's generalization ability through sequence-level local style enhancement.

[0057] In one possible implementation, the local sequence enhancement module specifically includes sub-steps S2021 to S20210:

[0058] S2021: Obtain two-dimensional feature maps from intermediate layers.

[0059] The initial dimension of the obtained feature map is 1. (B=Batch size, C=Number of channels, W=Width, H=Height).

[0060] S2022: By scanning and expanding, the two-dimensional feature map is reshaped into a one-dimensional feature sequence.

[0061] Specifically, the feature map is reshaped into a 1D sequence through "scan expansion". (D = sequence feature dimension, derived from the number of channels C, L = sequence length, calculated from W × H), making the features adaptable to Mamba's sequence processing mode.

[0062] S2023: By using a selective state-space model, the input dependency matrix and selective scanning mechanism are used to capture long-range semantic dependencies within the sequence, perform dependency modeling on the one-dimensional feature sequence, and obtain preliminary sequence features.

[0063] Specifically, The input is a two-dimensional selective scanning module, which utilizes Mamba's input dependency matrix and selective scanning mechanism to capture long-range semantic dependencies within the sequence and outputs sequence features modeled by dependency modeling. (Dimension Preservation) ).

[0064] S2024: Calculate the mean and standard deviation of preliminary sequence characteristics:

[0065]

[0066]

[0067] Where, x i This represents the i-th input sample. This represents the preliminary sequence features of the i-th sample. Let represent the mean of the preliminary sequence features of the i-th sample. Let L represent the l-th element in the preliminary sequence features of the i-th sample, and L represent the total length of the preliminary sequence features of the i-th sample. Let represent the standard deviation of the preliminary sequence characteristics of the i-th sample, and d represent a small constant introduced to avoid the problem of the denominator being zero due to the standard deviation possibly being zero during the calculation process.

[0068] S2025: Estimate the uncertainty of the characteristic mean and standard deviation based on the statistics of all samples within a batch.

[0069] The uncertainty of the characteristic mean and standard deviation is calculated as follows:

[0070]

[0071]

[0072] Where B represents the total number of samples in the batch, This represents the mathematical expectation of the mean of the preliminary sequence characteristics of all samples within a batch. This represents the variance of the mean of the preliminary sequence characteristics of all samples within a batch. The standard deviation represents the initial sequence characteristics of the i-th sample. The variance represents the standard deviation of the preliminary sequence characteristics of all samples within a batch.

[0073] S2026: Based on the Gaussian distribution, new statistics with random perturbations are generated to simulate style differences in different fields.

[0074]

[0075]

[0076] in, Let represent the perturbation mean of the preliminary sequence features of the i-th sample. The perturbation standard deviation represents the initial sequence characteristics of the i-th sample. This represents random noise whose perturbation to the mean follows a standard normal distribution N(0,1). The perturbation of the standard deviation is random noise that follows a standard normal distribution N(0,1).

[0077] S2027: Based on the new statistics, style enhancement is performed on the preliminary sequence features to determine the style-enhanced sequence features:

[0078]

[0079] in, This represents the style enhancement sequence feature of the i-th sample.

[0080] S2028: Generate a random mask for each sample.

[0081] The specific method for calculating the random mask is as follows:

[0082] ,

[0083] in, A binary mask indicating whether the j-th sequence unit in the i-th sample sequence is included in the local style perturbation range. An exponential function indicating whether a sequence position belongs to a perturbation interval, where j represents the sequence position. start This indicates a random starting position, and P represents the mask ratio, with a default value of 0.75.

[0084] It should be noted that the local sequence augmentation module in VSSDG perturbs and mixes local feature regions by randomly generating masks. While preserving the overall feature distribution, it introduces local randomness, which enhances the generalization ability of features and the model's resistance to interference. This design enables features of different scales to learn more robust cross-scale association patterns during information interaction and fusion.

[0085] S029: Based on the random mask, the preliminary sequence features and style-enhanced sequence features are mixed by element-wise multiplication to obtain the local enhanced sequence features:

[0086]

[0087] in, This represents the local enhanced sequence features of the i-th sample. This represents the random mask for the i-th sample. This indicates an element-wise multiplication operation.

[0088] It should be noted that the mask is expanded to (Matching with feature dimensions), through element-wise multiplication mixing and f h (x i This yields the final local enhancement features.

[0089] S20210: By scanning and merging, the local enhanced sequence features are reshaped into a two-dimensional local enhanced feature map.

[0090] It should be noted that "scan merging" will... From 1D sequence Reconstructing the 2D spatial feature map .

[0091] Furthermore, to maintain label dependencies, continuous sequence perturbation is used instead of random pixel selection to ensure the integrity of the feature structure and the stability of semantic relationships. This module generates diverse feature representations by modeling the feature statistics of local random continuous subsequences using a multivariate Gaussian distribution, thereby enhancing the model's robustness to different domain shifts.

[0092] Reference manual attached Figure 3 The diagram shows a structural schematic of a local enhancement module provided in an embodiment of the present invention.

[0093] Specifically, Figure 3 In (a), the local enhancement module includes linear layers, layer normalization, local sequence enhancement modules, and depthwise convolutions. For addition, For element-wise product, This is the Sigmoid function. Figure 3 In (b) of the diagram, the local sequence enhancement module includes scan merging, two-dimensional selective scan module, style enhancement, sequence blending, and scan merging.

[0094] Furthermore, Figure 3 The inclusion relationship is as follows: the local enhancement module includes linear layers, layer normalization, local sequence enhancement, and depthwise convolution; the local sequence enhancement submodule includes scan merging, two-dimensional selective scan module, style enhancement, and sequence blending. Figure 3 In (a), the input is normalized, passed through a linear layer, and then through a depthwise convolutional layer before being connected to local sequence enhancement. The output of the local sequence enhancement is then passed through a sigmoid function and combined with the output of another linear layer. After layer normalization and linearization, it is then compared with the input. Output; Figure 3 In (b) of the image, the original image is scanned, merged and split, and then sequentially connected to the two-dimensional selective scanning module, style enhancement, and sequence mixing, and then scanned and merged again for output.

[0095] In this embodiment of the invention, the advantage of introducing a local sequence enhancement module is that it can explicitly enhance the diversity and generalization ability of the backbone network to extract features. By simulating style differences in different domains, it makes the model more robust to changes in the input image, thereby improving the stability and performance of the model in complex clinical environments.

[0096] S3: Through the Cross-scale Aware Self-Attention (CASA) module, the shallow detail features are optimized layer by layer using deep semantic features to generate multi-scale semantic enhancement features, and then dimensionality reduction is performed through the feature transformation module.

[0097] In one possible implementation, S3 specifically includes sub-steps S301 to S305:

[0098] S301: Based on the multi-scale features extracted by the encoder, and guided by the deeper semantic features of a higher level, the shallow detail features of the current layer are fused to obtain preliminary fused features.

[0099] The specific calculation method for obtaining the preliminary fusion features is as follows:

[0100]

[0101] Among them, z r 'z' represents the initial fusion features, 'c' represents the current layer features, and 'c' represents the context features of the previous layer.

[0102] Specifically, during the feature extraction process, the encoder first generates multi-scale features f1, f2, f3, and f4 from high-resolution shallow layers to low-resolution deep layers through a local enhancement module (where f1 is rich in detailed information and f4 contains global semantics). Then, at the end of the encoding stage, a cross-scale self-attention module is introduced to implement top-down feature optimization: starting with the deepest feature f4, it guides the optimization of f3 and f2 in sequence, and finally applies to f1.

[0103] It should be noted that the cross-scale self-attention module adopts a "layer-by-layer feedback" strategy, introducing deeper features as auxiliary information at each layer. After aligning the channels with the spatial resolution, these features are fused, avoiding semantic confusion and redundancy, and achieving structured, progressive cross-scale semantic enhancement.

[0104] S302: Apply channel attention to the preliminary fusion features to obtain the channel weight map.

[0105] The specific method for calculating the channel weight map is as follows:

[0106]

[0107] Where, m c This represents the channel weight map, Sigmoid represents the Sigmoid activation function, and W... c This represents the channel attention vector.

[0108] S303: Apply spatial attention to the preliminary fusion features to obtain a spatial weight map.

[0109] The specific method for calculating the spatial weight map is as follows:

[0110]

[0111] Where, m s Represents a spatial weighted graph, W s Represents the spatial attention vector. This represents element-wise product.

[0112] S304: Utilizing both channel weight maps and spatial weight maps, the weighted fusion features are double-weighted to output multi-scale semantically enhanced features.

[0113]

[0114] in, This represents multi-scale semantic enhancement features.

[0115] S305: The multi-scale semantic enhancement features are dimensionality reduced by the feature transformation module to obtain the dimensionality-reduced semantic enhancement features.

[0116] It should be noted that the cross-scale self-attention module integrates channel and spatial attention mechanisms: channel attention dynamically adjusts the importance of semantic channels, and spatial attention focuses on key areas such as image edges, so that even the deepest layers can refine features through self-attention.

[0117] Reference manual attached Figure 4 The diagram shows a structural schematic of a cross-scale self-attention module provided in an embodiment of the present invention.

[0118] It should be noted that, Figure 4 Mid-to-low layer features X l With high-level characteristics X h For input, ⊕ represents addition. The multiplication is represented by the cross-scale self-attention module, which includes channel attention and spatial attention sub-modules. The former includes average pooling, max pooling, 1×1 convolution, ReLU activation and sigmoid activation components, while the latter includes average pooling, max pooling, concatenated convolution and sigmoid activation components.

[0119] Furthermore, Figure 4 The connection process is as follows: X l With X h After summing, the input channel attention is applied, and the output is X. e Multiply the sum by the summation result and then input spatial attention to obtain X. s Finally, the result of the multiplication is multiplied and output; in channel attention, the two pooling branches are convolved and activated, then added and passed through the Sigmoid function; in spatial attention, the two pooling results are concatenated and passed through the Sigmoid function.

[0120] In this embodiment of the invention, the cross-scale perceptual self-attention module aims to guide the learning of shallow features layer by layer through deep semantic features, enabling features at each level to incorporate high-level semantic context while preserving details, thereby improving the semantic consistency and edge accuracy of the segmentation results. By combining cross-scale feature injection with the attention mechanism, shallow features continuously absorb deep semantic context while preserving local edge information. Furthermore, the local perturbations introduced by the local sequence enhancement module further promote the consistency and discriminativeness of feature representations at different scales, providing a multi-dimensionally optimized feature foundation for the subsequent decoding process.

[0121] S4: Using the Adaptive Detail Enhancing Module (ADEM), semantic enhancement features at a certain size are used as anchors to fuse semantic enhancement features at other sizes to generate local detail features.

[0122] It should be noted that before S4, the feature transformation module is used to reduce dimensionality to reduce computation, and finally the "dimensionality-reduced and optimized" features are sent to the adaptive detail enhancement module for fusion.

[0123] Specifically, the core idea of ​​this module is to use a certain scale as an anchor point, integrate structural information from other scales, and construct a semantically complete detailed representation.

[0124] It should be noted that using semantic enhancement features at a certain size as anchor points here means arbitrarily selecting semantic enhancement features at a certain size as a benchmark, and then fusing semantic enhancement features at other sizes. Those skilled in the art can adaptively select semantic enhancement features at a certain size as anchor points based on the characteristics of the detection task. This invention does not limit the specific scale used as anchor points.

[0125] In one possible implementation, S4 specifically includes sub-steps S401 to S404:

[0126] S401: Perform convolutional preprocessing on multi-scale semantic enhancement features.

[0127] Specifically, the adaptive detail enhancement module first preprocesses features at each scale through parallel 3×3 convolutions.

[0128] S402: Using semantic enhancement features at a certain size as anchor points, align non-anchored scale features to anchored scale features through upsampling or downsampling.

[0129] S403: Based on the aligned features, detail masks for each non-anchored scale are generated through convolution and the Sigmoid activation function.

[0130]

[0131] Among them, S i W represents the detail mask for the i-th non-anchored scale, Sigmoid represents the Sigmoid activation function, and W... i Let i represent the convolution operation for the i-th non-anchored scale. This represents the i-th non-anchored scale feature.

[0132] It should be noted that this step is helpful in characterizing the salience of the scale's position under the anchored scale.

[0133] S404: Fuses anchored scale features with detail masks of all non-anchored scales to output local detail features.

[0134] The specific method for calculating local detail features is as follows:

[0135]

[0136]

[0137] Where Y represents local detail features, y a W represents the output of the anchored scale features after convolution. a X represents the convolution operation with anchored scale. a This indicates the anchoring scale characteristics.

[0138] Specifically, the anchored features are convolved with themselves and then multiplied element-wise with the masks of all non-anchored scales to finally output the fused features.

[0139] It's important to note that, unlike traditional feature concatenation or fixed weighting methods, the adaptive detail enhancement module employs an "anchor-driven, multi-source injection" strategy: it receives a multi-scale feature list, designates a specific scale as the anchor scale, and generates detail masks by aligning features at other scales (upsampling / downsampling) and performing convolutional processing. Cross-scale fusion is achieved through the dynamic interaction between the mask and the anchor features. The adaptive detail enhancement module possesses bidirectional information flow capabilities: when anchoring at a shallow, high-resolution level, deep features provide semantic guidance, reinforcing semantically consistent regions; when anchoring at a deep level, shallow features inject edge cues to compensate for insufficient localization. Experiments show that the adaptive detail enhancement module significantly improves the model's edge preservation and fine-grained recovery capabilities for complex structures and occluded scenes, enhancing the boundary accuracy and structural consistency of the segmentation results.

[0140] Reference manual attached Figure 5 The diagram shows a structural schematic of an adaptive detail enhancement module provided in an embodiment of the present invention.

[0141] Specifically, Figure 5The middle anchor block is the baseline feature of the adaptive detail enhancement module, while the other blocks are features to be fused. A 3×3 convolution is used for feature transformation.

[0142] Furthermore, Figure 5 The inclusion relationships are as follows: the adaptive detail enhancement module includes anchor blocks, 3×3 convolutions, and multiple scale adaptive detail modules; the SADM submodule includes feature height determination units, average pooling, upsampling, 3×3 convolutions, and the Sigmoid activation function.

[0143] In this embodiment of the invention, the module learns dynamic detail masks to avoid the limitations of fixed structures, achieves "on-demand enhancement" to highlight target boundaries and fine structures, suppresses irrelevant redundancy, and thus significantly improves the model's ability to preserve edges and restore structures in complex image shapes and occluded scenes.

[0144] S5: Through the pyramid pooling module (PPM), global context enhancement processing is performed on local detailed features to generate context-enhanced features.

[0145] The pyramid pooling module in the decoder aims to enhance the global semantic awareness of features through multi-scale context modeling.

[0146] It should be noted that the decoder receives multi-scale features (f1, f2, f3, f4) optimized by the cross-scale self-attention module and reduced in dimensionality by the feature transformation module. The adaptive detail enhancement module then integrates the cross-scale information using an "anchored fusion" strategy to generate four fused features (f11, f21, f31, f41) at different resolutions. The deepest fused feature, f41, first passes through a pyramid pooling module, where multi-scale pooling operations capture the global context and enhance semantic consistency.

[0147] In one possible implementation, S5 specifically includes sub-steps S501 to S504:

[0148] S501: Performs adaptive average pooling operation on local detail features at different scales.

[0149] It should be noted that this module contains multiple parallel branches, each corresponding to an adaptive average pooling operation at a different scale (e.g., bins=(1,2,3,6)), which pools the input feature map to different resolutions to capture multi-level contextual information from global to local.

[0150] S502: The pooling results at each scale are compressed to the channel dimension by 1×1 convolution, batch normalized and ReLU activated, and then upsampled to the original size of the local detail features by bilinear interpolation.

[0151] S503: The upsampled features are concatenated with local detail features to obtain multi-scale concatenated features.

[0152] S504: By using a 1×1 convolutional layer, the number of channels of the multi-scale spliced ​​features is adjusted to the input dimension to obtain context-enhanced features.

[0153] It should be noted that this design effectively compensates for the insufficient edge localization caused by the reduced spatial resolution of deep features. By integrating global semantic cues and local detail features, it significantly improves the model's segmentation accuracy and structural integrity of target regions in complex scenes.

[0154] S6: Upsample the feature map corresponding to the context-enhanced features to gradually restore the feature map resolution.

[0155] Specifically, the decoder employs a four-level decoder module for progressive upsampling: starting with f41 processed by the pyramid pooling module, it sequentially fuses shallow detail features with f31, f21, and f11 through an adaptive global-local fusion gating mechanism. Each module achieves a 2x resolution improvement through transposed convolutions and refines features using convolutional layers and attention mechanisms. Finally, the segmentation result is output through `final_conv`, while a deep supervision mechanism is introduced to supervise and weightedly fuse intermediate features, further improving segmentation accuracy.

[0156] In this embodiment of the invention, this step addresses the problem of "low feature resolution and difficulty in accurately locating image boundaries" in the decoding stage by using "gradual upsampling" instead of a one-time upsampling, thus avoiding the loss of details. The recovered high-resolution features can accurately correspond to the image positions in the original image, providing a "pixel-level" positioning basis for the gated fusion of the AGLI module and the final image extraction, further improving the positioning accuracy of the blurred boundary image.

[0157] S7: By strengthening skip connections through the Adaptive Global-Local Integration Gate (AGLI) module, the context enhancement features after upsampling are fused to obtain the decoded fused features.

[0158] It should be noted that this module design draws on the gating mechanism of AttentionU-Net and optimizes its structure, aiming to achieve dynamic fusion of semantics and details in skip connections through the attention mechanism.

[0159] In one possible implementation, S7 specifically includes sub-steps S701 to S706:

[0160] S701: Obtain the local detail features and the upsampled global semantic features.

[0161] S702: Perform 1×1 convolution and batch normalization on the global semantic features to obtain dimensionality-reduced global features.

[0162] The specific method for calculating the dimensionality reduction global features is as follows:

[0163]

[0164] in, Denotes dimensionality reduction of global features, and BN represents batch normalization operation. This represents a 1×1 global convolution operation, where X... global Represents global semantic features.

[0165] S703: Perform 1×1 convolution and batch normalization on local detail features to obtain dimensionality-reduced local features.

[0166] The specific method for calculating dimensionality reduction local features is as follows:

[0167]

[0168] in, Represents local features in dimensionality reduction. This represents a 1×1 local convolution operation, X local It represents local detailed features.

[0169] S704: Add the dimensionality-reduced global features to the dimensionality-reduced local features, and then activate them using the ReLU activation function to obtain intermediate features.

[0170] The intermediate feature calculation method is as follows:

[0171]

[0172] Where h represents the intermediate feature and ReLU represents the ReLU activation function.

[0173] S705: Perform a 1×1 convolution on the intermediate features and activate them with the Sigmoid function to generate an attention weight map.

[0174] The attention weight map is calculated as follows:

[0175]

[0176] in, This represents the attention weight map, and Sigmoid represents the Sigmoid activation function. This represents a 1×1 attention-weighted convolution operation.

[0177] S706: Multiply the local detail features element-wise with the attention weight map, and then add them to the global semantic features to obtain the decoded fusion features:

[0178]

[0179] Among them, y c This indicates the decoding and fusion features.

[0180] It should be noted that this invention selectively retains local details consistent with global semantics through a weighted graph, suppressing redundant information in conflicting regions. The adaptive global-local fusion gating mechanism can adaptively adjust the fusion ratio of local and global features. When semantics are consistent, detail preservation is enhanced; conversely, interference is suppressed, improving the accuracy of target edge reconstruction and the consistency of semantic structure. Its lightweight structure achieves fine-grained semantic guidance, bridging the semantic-detail gap in feature fusion, helping the decoder focus on semantically reliable and clearly defined regions, and improving target segmentation accuracy and edge perception capabilities.

[0181] Reference manual attached Figure 6 The diagram shows a structural schematic of an adaptive global-local fusion gating module provided in an embodiment of the present invention.

[0182] Specifically, Figure 6 Local features are inputs of local detailed features, while global features are inputs of global context features; 1×1 convolutions are used for channel adjustment, ReLU activation layers introduce non-linearity, transposed convolutions adapt to the global feature scale, Sigmoid generates weights, and ⊕ represents addition. The output is the final fusion result, which is an element-wise multiplication. This module contains local feature branches and global feature branches. The local branches contain two 1×1 convolutions, ReLU, a 1×1 convolution, and a Sigmoid activation function, while the global branches contain a transposed convolution.

[0183] Furthermore, Figure 6 The connection relationship is as follows: local features are split into two paths, convolved by 1×1, and then added together. Then, they are sequentially connected to ReLU, a 1×1 convolution, and a Sigmoid activation function, before being combined with the original local features. The global features, after being transposed and convolved, are then combined with the... The results are added together to obtain the output.

[0184] It should be noted that, Figure 6 By weighted fusion of local and global features, the detailed information and contextual relationships of the features are taken into account, which effectively improves the comprehensive representation ability of the features and provides more comprehensive feature support for subsequent image processing.

[0185] S8: Image extraction is performed based on decoding and fusion features.

[0186] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0187] In this embodiment of the invention, a feature extraction system constructed using multi-layered local enhancement modules overcomes the bottleneck of CNNs in long-range dependency modeling and fully leverages the linear computational complexity advantage of the Mamba architecture in sequence modeling, fundamentally laying the foundation for high accuracy and efficiency. Based on this, the synergistic effect of a cross-scale attention mechanism and an adaptive detail enhancement module achieves accurate perception of multi-scale semantics and detailed reconstruction of blurred boundaries. Supplemented by an adaptive global-local fusion gating module, the feature representation capability for images with blurred boundaries and varying scales is enhanced, thus achieving the optimal balance between segmentation accuracy and processing efficiency. Simultaneously, based on the efficient sequence modeling capabilities of the Mamba architecture, the fusion advantages of combined local perturbation feature enhancement, multi-scale attention mechanisms, and multi-level, multi-scale feature extraction enable accurate localization of image regions.

[0188] The image region of interest (ROI) extraction method based on the Mamba architecture provided in this application can be executed by an image ROI extraction device based on the Mamba architecture. This application uses an image ROI extraction device based on the Mamba architecture executing the image ROI extraction method as an example to illustrate the image ROI extraction device based on the Mamba architecture provided in this application.

[0189] Reference manual attached Figure 7 The diagram shows a schematic representation of an image region of interest extraction system based on the Mamba architecture provided in an embodiment of the present invention.

[0190] This invention provides an image region of interest extraction system 20 based on the Mamba architecture, including: a processor 201 and a memory 202;

[0191] The memory 202 stores programs or instructions that can run on the processor 201. When the program or instructions are executed by the processor 201, they implement the steps of the above-described image region of interest extraction method based on the Mamba architecture and achieve the same technical effect. To avoid repetition, the present invention will not elaborate further.

[0192] It should be understood that the processor 201 in this embodiment of the invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0193] It should also be understood that the memory 202 in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DR RAM).

[0194] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0195] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0196] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0197] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0198] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0199] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0200] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0201] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0202] This invention provides a readable storage medium comprising: storing a program or instructions on the readable storage medium, wherein when the program or instructions are executed by a processor, the program or instructions implement the steps of the above-described image region of interest extraction method based on the Mamba architecture, and can achieve the same technical effect. To avoid repetition, this invention will not elaborate further.

[0203] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A method for image region of interest extraction based on Mamba architecture, characterized in that, The method comprises the following steps: S1: obtaining an image to be extracted; S2: performing multi-scale feature extraction on the image to be extracted through a local enhancement module to obtain a multi-scale feature map; wherein the local enhancement module comprises a local sequence enhancement module, and the multi-scale feature map comprises shallow detail features and deep semantic features; S3: using the deep semantic features to guide the optimization of the shallow detail features layer by layer through a cross-scale self-attention module to generate multi-scale semantic enhancement features, and performing dimension reduction processing through a feature conversion module; S4: taking the semantic enhancement features at a certain size as an anchor point, fusing the semantic enhancement features at other sizes through an adaptive detail enhancement module to generate local detail features; wherein S4 specifically comprises: S401: performing convolution preprocessing on the multi-scale semantic enhancement features; S402: taking the semantic enhancement features at a certain size as an anchor point, aligning the non-anchor scale features to the anchor scale features through upsampling or downsampling; S403: based on the aligned features, generating detail masks of each non-anchor scale through convolution and Sigmoid activation function; S404: fusing the anchor scale features with the detail masks of all non-anchor scales to output the local detail features; S5: performing global context enhancement processing on the local detail features through a pyramid pooling module to generate context enhancement features; S6: performing upsampling processing on the feature map corresponding to the context enhancement features to gradually restore the feature map resolution; S7: strengthening the skip connection through an adaptive global-local fusion gating module to fuse the context enhancement features after upsampling processing to obtain decoding fusion features; wherein S7 specifically comprises: S701: obtaining the local detail features and the global semantic features after upsampling processing; S702: performing 1x1 convolution and batch normalization processing on the global semantic features to obtain reduced dimension global features; S703: performing 1x1 convolution and batch normalization processing on the local detail features to obtain reduced dimension local features; S704: adding the reduced dimension global features and the reduced dimension local features, and activating through the ReLU activation function to obtain intermediate features; S705: performing 1x1 convolution on the intermediate features and activating through the Sigmoid function to generate an attention weight map; S706: multiplying the local detail features and the attention weight map element by element, and then adding the global semantic features to obtain the decoding fusion features; S8: performing image extraction based on the decoding fusion features.

2. The method of claim 1, wherein the Mamba architecture-based image region of interest extraction method is characterized by, S2 specifically comprises: S201: dividing the image to be extracted into patches and embedding features; S202: performing multi-level feature extraction and downsampling on the embedded feature image through a multi-layer local enhancement module to extract the multi-scale feature map step by step.

3. The method of claim 1, wherein the Mamba architecture-based image region of interest extraction method is characterized by, The local sequence enhancement module is specifically used for: S2021: obtaining a two-dimensional feature map from an intermediate layer; S2022: remodeling the two-dimensional feature map into a one-dimensional feature sequence through scanning expansion; S2023: Through a selective state space model, long-range semantic dependency within a sequence is captured by using an input dependency matrix and a selective scanning mechanism, dependency modeling is performed on the one-dimensional feature sequence to obtain preliminary sequence features; S2024: The mean and standard deviation of the preliminary sequence features are calculated; S2025: The uncertainty of the feature mean and standard deviation is estimated based on the statistics of all samples in the batch; S2026: Based on a Gaussian distribution, new statistics with random disturbances are generated to simulate style differences in different fields; S2027: According to the new statistics, the preliminary sequence features are style enhanced to determine style-enhanced sequence features; S2028: Random masks are generated for each sample; S2029: According to the random masks, element-level multiplication mixing is performed on the preliminary sequence features and the style-enhanced sequence features to obtain locally enhanced sequence features; S20210: The locally enhanced sequence features are reshaped into two-dimensional locally enhanced feature maps through scanning merging.

4. The method of claim 1, wherein the Mamba architecture-based image region of interest extraction method is characterized by, The S3 specifically includes: S301: Based on the multi-scale features extracted by the encoder, the shallow detail features of the current layer are fused with higher-level deep semantic features as a guide to obtain preliminary fusion features; S302: Channel attention is applied to the preliminary fusion features to obtain a channel weight map; S303: Spatial attention is applied to the preliminary fusion features to obtain a spatial weight map; S304: The weighted fusion features are double-weighted using the channel weight map and the spatial weight map, and the multi-scale semantic enhancement features are output; S305: The multi-scale semantic enhancement features are processed by a feature conversion module to obtain dimension-reduced semantic enhancement features.

5. The Mamba architecture based image region of interest extraction method according to claim 1, wherein, The S5 specifically includes: S501: Adaptive average pooling operation is performed on the local detail features of different scales; S502: After 1x1 convolution compression channel dimension, batch normalization and ReLU activation, the pooling results of each scale are upsampled to the original size of the local detail features by bilinear interpolation; S503: The upsampled features are concatenated with the local detail features to obtain multi-scale concatenated features; S504: The channel number of the multi-scale concatenated features is adjusted to the input dimension by a 1x1 convolution layer to obtain the context enhancement features.

6. A system for image region of interest extraction based on Mamba architecture, characterized in that, It includes: a processor and a memory; The memory stores programs or instructions that can be run on the processor, and the programs or instructions are executed by the processor to implement the steps of the image interest region extraction method based on the Mamba architecture as claimed in any one of claims 1 to 5.

7. A readable storage medium, characterized by, The program or instruction is stored on the readable storage medium, and the program or instruction is executed by the processor to implement the steps of the image interest region extraction method based on the Mamba architecture as claimed in any one of claims 1 to 5.

Images