Image segmentation method and system based on functional bionic dual-channel fusion

Abstract

The application discloses an image segmentation method and system based on a functional bionic dual-channel fusion, and the method comprises the following steps: acquiring and preprocessing an image to be segmented; inputting the image to be segmented into an encoder to extract multi-scale features, and obtaining shallow feature maps and deep feature maps; inputting the shallow feature maps into an edge topology refining module to enhance edge structure representation through dynamic topology propagation and geometric reconstruction, and obtaining edge structure enhanced feature maps; inputting the deep feature maps into a multi-scale mask context modulation module to inhibit noise and capture discriminative semantics through mask-guided multi-scale feature aggregation, and obtaining semantic enhanced feature maps; inputting the edge structure enhanced feature maps and the semantic enhanced feature maps into a feature similarity complementarity topology fusion module to perform feature fusion through similarity bidirectional topology propagation and complementarity bidirectional topology propagation, and obtaining fusion feature maps; and generating a segmentation result map based on the fusion feature maps.

Description

Technical Field

[0001] This invention belongs to the field of image processing and deep learning technology, specifically relating to an image segmentation method and system based on functional biomimetic dual-path fusion. Background Technology

[0002] Image segmentation refers to the technique of separating target objects or regions from an image. Its applications span multiple fields, including medical imaging, autonomous driving, remote sensing monitoring, and industrial quality inspection. The accuracy of image segmentation directly affects the reliability and effectiveness of downstream analysis tasks.

[0003] Existing image segmentation methods are mainly divided into traditional methods and deep learning methods. Traditional image segmentation methods mainly rely on threshold-based, edge-based, or region-based segmentation techniques. These methods typically depend on human priors or hand-designed features, making them difficult to adapt to complex and diverse image data, and exhibiting significant shortcomings in robustness and segmentation accuracy. In recent years, deep learning algorithms have been widely applied in the field of image segmentation due to their superior capabilities in feature extraction and modeling of complex data. Fully convolutional neural networks (FCNs) were the first to apply deep learning to semantic segmentation tasks; U-Net, with its encoding / decoding structure and skip connection design, has become a widely used benchmark model in the field of image segmentation. With the introduction of the Transformer architecture, VisionTransformer (ViT) brought the Transformer to the field of computer vision, and hybrid architectures such as TransUnet and TransFuse further improved segmentation performance. In addition, PraNet enhanced its ability to model diverse target boundaries through parallel partial decoders and inverse attention mechanisms; I 2 U-Net improves segmentation performance through dual-path information interaction.

[0004] However, existing image segmentation methods still have the following shortcomings: (1) insufficient fine-grained feature recovery during decoding, resulting in rough edges and inaccurate boundary localization in the segmentation results; (2) limited multi-scale semantic perception and context modeling capabilities, making it difficult to effectively handle complex image data with significant background noise, blurred target boundaries, and diverse scene changes; (3) information conflicts and redundancies exist in cross-scale feature fusion, and the resolution differences between features at different levels lead to low fusion efficiency. Therefore, there is an urgent need for an image segmentation method that can simultaneously enhance edge detail modeling, improve multi-scale semantic perception, and optimize cross-scale feature fusion. Summary of the Invention

[0005] To address the shortcomings of existing image segmentation methods in fine-grained feature recovery, multi-scale semantic perception, and cross-scale feature fusion, this paper proposes an image segmentation method and system based on functional biomimetic dual-pathway fusion, which can simultaneously enhance edge structure modeling capabilities, improve multi-scale semantic discrimination capabilities, and optimize cross-scale feature fusion efficiency.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] Firstly, the image segmentation method based on functional biomimetic dual-pathway fusion includes the following steps:

[0008] S1. Acquire and preprocess the image to be segmented;

[0009] S2. Input the image to be segmented into the encoder for multi-scale feature extraction to obtain shallow feature maps and deep feature maps;

[0010] S3. Input the shallow feature map into the constructed edge topology refining module, and enhance the edge structure representation through dynamic topology propagation and geometric reconstruction to obtain an edge structure enhanced feature map;

[0011] S4. Input the deep feature map into the constructed multi-scale mask context modulation module, suppress noise and capture discriminative semantics through mask-guided multi-scale feature aggregation, and obtain semantically enhanced feature map;

[0012] S5. Input the edge structure enhancement feature map and semantic enhancement feature map into the constructed feature similarity and complementarity topology fusion module, and perform feature fusion through similarity bidirectional topology propagation and complementarity bidirectional topology propagation to obtain the fused feature map;

[0013] S6. Generate segmentation result map based on fused feature map.

[0014] Preferably, S2 includes:

[0015] The encoder adopts a pyramid visual Transformer architecture, which consists of four stages. It divides the image to be segmented into image blocks with overlapping regions through overlapping block embedding, and generates multi-scale feature maps while downsampling layer by layer through a hierarchical pyramid structure. The output of the first stage of the encoder is used as the shallow feature map, and the outputs of the second, third and fourth stages of the encoder are used together as the deep feature map.

[0016] Preferably, S3 includes:

[0017] S31. Extract a binary edge map of the same size from the image to be segmented; generate an edge attention mask based on the binary edge map;

[0018] S32. The shallow feature map is processed by convolution to obtain shallow convolution features, and then projected onto the edge attention mask to obtain projected features; at the same time, the shallow convolution features are normalized to obtain shallow normalized projection vectors.

[0019] S33. Adaptive average pooling is used to extract key response points from the projection features, and cross-scale anchor features are established in the feature map; L2 normalization encoding is performed on the anchor features to obtain normalized anchor features.

[0020] S34. Calculate the directional similarity between the shallow normalized projection vector and the normalized anchor point feature to obtain the similarity matrix; aggregate the shallow normalized projection vector to the anchor point feature through the similarity matrix to obtain the anchor point aggregated feature.

[0021] S35. Perform L2 normalization on the anchor point aggregation features to obtain normalized anchor point aggregation features; calculate the similarity matrix between the normalized anchor point aggregation feature nodes and construct an adaptive adjacency graph; use the adjacency graph for feature propagation, calculate the structural propagation features of each node, and calculate the difference with the anchor point aggregation features; finally, perform convolution in the node dimension.

[0022] S36. The anchor points after the adjacency graph features are propagated are mapped back to the location space by projection of the similarity matrix to obtain the location mapping features. After the location mapping features are normalized by convolution, they are residually connected with the shallow feature map to obtain the edge structure enhancement features.

[0023] Preferably, S4 includes:

[0024] S41. Upsample and align the outputs of the second, third, and fourth stages of the encoder and then concatenate them along the channel dimension to obtain cross-stage aligned concatenation features.

[0025] S42. Input the cross-stage aligned splicing features into three parallel sub-modules and one identity direct connection branch respectively; configure different receptive fields for the three sub-modules and generate mask context modulation features for each sub-module respectively;

[0026] S43. The mask context modulation features of each sub-module are concatenated with the cross-stage aligned concatenation features output by the identity direct connection branch, and then processed by convolution, batch normalization and ReLU activation function in sequence to obtain semantic enhancement features.

[0027] Preferably, in S42, the following operations are performed within each submodule:

[0028] The cross-stage aligned splicing features are channel normalized to obtain normalized splicing features. The normalized splicing features are then linearly mapped to obtain query vectors and context vectors. The global semantic vector is obtained based on the context vectors.

[0029] Multi-level depthwise separable convolution is applied to the context vector, and the context after depthwise separable convolution is combined with the mask tensor. The results of combining all levels are aggregated with the global semantic vector to generate context modulation weights. Through linear transformation guided by the query vector, the query vector is weighted and integrated using the context modulation weights to generate mask context modulation features.

[0030] As a preferred embodiment, in S42, the mask tensor is generated by randomly selecting a reserved region within a preset range in the sample, and setting the remaining regions to large negative constants.

[0031] As a preferred embodiment, S5 includes:

[0032] S51. After aligning the edge structure enhancement features and semantic enhancement features by size, extract features separately to obtain the first feature and the second feature. Then, use adaptive average pooling to construct graph nodes to generate the first graph node and the second graph node.

[0033] S52. Input the nodes of the first graph and the nodes of the second graph into the similarity topology fusion module to generate similarity topology propagation fusion features;

[0034] S53, Move the nodes in the first diagram , and the nodes in the second diagram Input the complementary topology fusion module to generate complementary topology propagation fusion features;

[0035] S54. After fusing the similarity topology propagation fusion features and the complementarity topology propagation fusion features according to a dynamic ratio, perform convolution and batch normalization processing. Add the processing result to the semantic enhancement features after convolution and batch normalization to obtain the fused feature map.

[0036] Preferably, S52 includes:

[0037] L2 normalization is performed on the nodes of the first graph and the nodes of the second graph respectively, and the bidirectional similarity matrix is ​​calculated. The bidirectional propagation features are calculated based on the bidirectional similarity matrix. The bidirectional propagation features are weighted and fused to obtain two fused propagation features. The two fused propagation features are concatenated and centered, and then the similarity topology propagation fused features are obtained through node convolution operation.

[0038] As a preferred embodiment, S53 includes:

[0039] Based on the nodes of the first and second graphs, a bidirectional difference graph is calculated by element-wise subtraction, and enhanced difference features are generated by convolutional layers and ReLU activation functions. The contribution of enhanced difference features is dynamically adjusted by learnable parameters, and the obtained bidirectional complementary information is fused to generate complementary fusion features. Node convolution is performed on the complementary fusion features to obtain complementary topology propagation fusion features.

[0040] Secondly, an image segmentation system based on functional biomimetic dual-pathway fusion includes:

[0041] The data acquisition module is used to acquire and preprocess the image to be segmented;

[0042] The image segmentation network model includes a dual-path decoding structure, an edge topology refinement module, a multi-scale mask context modulation module, and a feature similarity complementary topology fusion module.

[0043] The process involves inputting the image to be segmented into a dual-path decoding structure for multi-scale feature extraction, resulting in shallow and deep feature maps. The shallow feature map is then input into an edge topology refinement module, where dynamic topology propagation and geometric reconstruction enhance edge structure representation, yielding an edge structure-enhanced feature map. The deep feature map is input into a multi-scale mask context modulation module, where mask-guided multi-scale feature aggregation suppresses noise and captures discriminative semantics, resulting in a semantically enhanced feature map. Finally, the edge structure-enhanced feature map and the semantically enhanced feature map are input into a feature similarity and complementarity topology fusion module, where similarity-based bidirectional topology propagation and complementarity-based bidirectional topology propagation are used for feature fusion, resulting in a fused feature map.

[0044] The generation module is used to generate segmentation result maps based on the fused feature maps;

[0045] The image segmentation system based on functional biomimetic dual-path fusion is used to implement the image segmentation method and steps based on functional biomimetic dual-path fusion as described in the first aspect.

[0046] Compared with the prior art, the beneficial effects of the present invention are reflected in:

[0047] 1. Unlike existing technologies where edge enhancement, context modeling, and high-low layer fusion are independent and lack coordination, this invention combines an edge topology refinement module, a multi-scale mask context modulation module, and a feature similarity complementary topology fusion module to form a collaborative segmentation mechanism of "edge topology enhancement - multi-scale semantic modulation - dual-path topology fusion". This enables shallow edge details and deep semantic context to be enhanced and complementaryly transferred under topological constraints, thereby simultaneously improving boundary segmentation accuracy, robustness to complex backgrounds, and consistency of multi-target structures, and enhancing the model's generalization ability in different image modalities and task scenarios.

[0048] 2. Unlike traditional image segmentation methods that only employ shallow edge extraction, boundary attention enhancement, or simple feature overlay, this invention uses an edge topology refinement module. Through anchor point generation based on edge prior constraints, adaptive topology propagation between anchor point nodes, and spatial back projection reconstruction, the local edge response is enhanced into a topological representation with global correlation. This enables the model to enhance boundary continuity and detail localization capabilities in scenarios with noise interference, weak boundaries, and edge breaks, thereby improving the accuracy of characterizing target contours and boundary structures.

[0049] 3. Unlike traditional techniques that rely solely on single-scale convolution, ordinary feature pyramids, or direct multi-scale stitching for semantic modeling, this invention employs a multi-scale mask context modulation module. This module performs hierarchical context modeling of mid-to-high-level features through three parallel branches: ultra-wide field, wide field, and micro receptive field. Combined with mask-guided context modulation and global semantic compensation, the model can suppress irrelevant noise and highlight effective semantic regions in images with complex backgrounds, low contrast, and significant scale variations. This enhances the semantic perception capability and regional consistency of targets at different scales.

[0050] 4. Unlike traditional techniques that use direct addition, direct splicing, or single attention fusion of high and low-level features, this invention employs a feature similarity and complementary topological fusion module. This module maps edge structure enhancement features and semantic enhancement features to a unified node space. Bidirectional topological modeling is performed through similarity propagation pathways and complementary interaction pathways, respectively, and adaptive fusion is achieved using dynamic weights. This allows the model to simultaneously retain high-level semantic localization capabilities and low-level edge structure expression capabilities, reducing semantic misalignment, loss of details, and redundant information propagation, thereby improving the segmentation accuracy of complex boundaries, irregular structures, and small targets. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the method framework of Embodiment 1 of the present invention;

[0052] Figure 2 This is a schematic diagram of the internal structure of the Edge Topology Refining Module (ETR) in Embodiment 1 of the present invention;

[0053] Figure 3 This is a schematic diagram of the internal structure of the MCM submodule in the multi-scale mask context modulation module (MMCI) of Embodiment 1 of the present invention;

[0054] Figure 4 This is a schematic diagram of the internal structure of the Feature Similarity Complementarity Topology Fusion Module (SCTF) in Embodiment 1 of the present invention;

[0055] Figure 5 This is a visual comparison of the segmentation results of the present invention and existing methods on multiple image datasets;

[0056] Figure 6 This is a comparison of characteristic thermograms of the ablation experiments of each module of the present invention;

[0057] Figure 7 This is a bar chart showing the ablation experiment results inside the MMCI module of this invention. Detailed Implementation

[0058] To make the technical means, inventive features, objectives, and effects of the invention readily understandable, the invention is further described below with reference to specific illustrations. However, the invention is not limited to the embodiments described below.

[0059] It should be noted that the structures, proportions, sizes, etc., illustrated in the accompanying drawings of this specification are only used to complement the content disclosed in the specification for those skilled in the art to understand and read, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0060] Example 1:

[0061] like Figure 1 The image segmentation method shown includes the following steps:

[0062] S1. Acquire and preprocess the image to be segmented;

[0063] Specifically, the images to be segmented are acquired, including colonoscopy images of polyps, breast ultrasound images, or multi-organ CT images. The images are then adjusted to a preset size (e.g., 352×352, 256×256, or 224×224) and subjected to normalization preprocessing.

[0064] This embodiment verifies the effectiveness of the invention on the following dataset:

[0065] Polyp Datasets: Following the PraNet experimental setup, five colonoscopy polyp datasets were used. The training set contained 1,450 images, including 900 images from Kvasir-SEG and 550 images from CVC-ClinicDB; the test set contained 798 images, including 100 images from Kvasir-SEG, 62 images from CVC-ClinicDB, 380 images from CVC-ColonDB, 196 images from ETIS-LaribPolypDB, and 60 images from CVC-300.

[0066] Multi-organ CT dataset: The Synapse dataset was used, covering 30 abdominal CT scan samples, totaling 3,779 clinical abdominal CT images. Each CT scan consisted of 85 to 198 tomographic slices with a resolution of 512×512 pixels. Following the TransUNet experimental setup, the training set contained 18 cases (2,212 axial slices), and the validation set contained 12 cases.

[0067] Breast ultrasound dataset: The BUSI dataset is used, containing ultrasound images and segmentation maps of normal, benign, and malignant breast cancer cases. This example only uses benign and malignant images, totaling 647 images, uniformly adjusted to 256×256 resolution, and randomly divided into training and test sets in an 8:2 ratio.

[0068] S2. Input the image to be segmented into the encoder for multi-scale feature extraction to obtain shallow feature maps and deep feature maps;

[0069] The encoder is used to extract multi-scale features from the input image. The encoder consists of four stages. It divides the input image into image patches with overlapping regions through overlap patch embedding to obtain smooth feature representations while maintaining local continuity. The encoder generates multi-scale feature maps by downsampling layer by layer through a hierarchical pyramid structure. The shallow features are used to capture edge and texture information, while the deep features are used to model semantic representations.

[0070] The encoder's output is fed into two complementary processing pathways: the first pathway, the edge topology refinement pathway, processes spatial structure and edge detail information; the second pathway, the multi-scale mask context modulation pathway, processes semantic and contextual information. The outputs of the two pathways are integrated through a feature similarity complementary topology fusion module.

[0071] Specifically, a Pyramid Vision Transformer architecture is used as the encoder. The input image is divided into image patches with overlapping regions through overlapping block embedding, and multi-scale feature maps are generated while downsampling layer by layer through a hierarchical pyramid structure. The output of the first stage of the encoder is used as the shallow feature map, and the outputs of the second, third, and fourth stages of the encoder are used as deep feature maps.

[0072] S3. Input the shallow feature map into the Edge Topology Refinement (ETR) module, and enhance the edge structure representation through dynamic topology propagation and geometric reconstruction to obtain an edge structure enhanced feature map;

[0073] See Figure 2The input of the edge topology refinement module is connected to the first-stage output of the encoder, and is used to enhance the detailed structural representation through dynamic topology propagation and geometric reconstruction.

[0074] The innovation of this invention in the ETR module lies in proposing an anchor point topology propagation and spatial back-projection reconstruction mechanism based on edge prior constraints. This mechanism transforms edge structure responses from pixel-level attention information into topological relationships in the anchor point node space, and achieves information propagation between boundary nodes through adaptive adjacency graph construction. The propagated topological features are then back-projected into the original spatial features. Therefore, it enhances the representation ability of weak boundaries, broken edges, and fine structures in the image, and improves the continuity and localization accuracy of segmentation boundaries. Specifically, the ETR module includes the following processing steps:

[0075] S31. Edge map extraction and edge attention mask generation

[0076] Dynamic Gaussian filtering is applied to each channel of the image to be segmented to remove noise, and the Sobel operator is used to extract a binary edge map of the same size as the input image. ; Binary edge map The edge attention mask is generated after processing by the Softmax function.

[0077] S32, Feature Projection

[0078] The first stage output of the encoder Shallow convolution features are obtained through convolution processing. and shallow convolution features Projecting onto the edge attention mask yields the projected features. ; for shallow convolution features Normalization is performed to obtain the shallow normalized projection vector. .

[0079] S33, Anchor Point Feature Extraction

[0080] For projection features Adaptive average pooling is used to extract key response points, and cross-scale anchor features are established in the feature map. :

[0081]

[0082] Anchor point features L2 normalized encoding is performed to obtain normalized anchor features. .

[0083] S34. Calculate the similarity matrix and perform feature aggregation.

[0084] shallow normalized projection vectors are calculated using cosine metrics. With normalized anchor point features The directional similarity between them introduces a learnable temperature parameter. Temperature adjustment of similarity distribution:

[0085]

[0086] in, This is a similarity matrix.

[0087] Shallow normalized projection vector Through the similarity matrix Aggregate to anchor point features to obtain anchor point aggregated features. :

[0088]

[0089] S35, Topological Space Propagation

[0090] Anchor point aggregation features L2 normalization is performed to obtain normalized anchor point aggregation features. Calculate the similarity matrix between nodes and construct an adaptive adjacency graph:

[0091]

[0092] in, The temperature parameter is learnable; feature propagation is performed using an adjacency graph, calculating the structural propagation features of each node, and then subtracting them from the aggregated features of the original anchor points. Finally, convolution is performed along the node dimension.

[0093]

[0094] S36, Feature Reconstruction and Residual Connection

[0095] The anchor points are mapped back to the location space by projection of the similarity matrix to obtain the location mapping features. :

[0096]

[0097] Map location features After convolution normalization, it is compared with the first stage output of the encoder. Perform residual connections to obtain edge structure enhancement features. .

[0098] S4. Input the deep feature map into the multi-scale mask context modulation (MMCI) module, suppress noise and capture discriminative semantics through mask-guided multi-scale feature aggregation, and obtain a semantically enhanced feature map;

[0099] See Figure 1 and Figure 3 The input of the multi-scale mask context modulation module is connected to the outputs of the second, third, and fourth stages of the encoder, and is used to suppress noise and capture semantic representations through mask-guided multi-scale feature aggregation.

[0100] The innovation of this invention in the MMCI module lies in proposing a parallel heterogeneous receptive field mask context modulation mechanism. After aligning and fusing high-level multi-scale semantic features from the encoder, these features are input into three mask context modulation branches: ultra-wide field, wide field, and micro receptive field. Each branch generates context modulation weights for the corresponding scale through decoupling the query vector from the context vector, layer-level depthwise separable convolution, random spatial mask suppression, and global semantic compensation. These weights are then fused with the identity direct connection branch. This results in: while preserving the original semantic information, suppressing background noise, local spurious responses, and redundant texture interference in the image; enhancing the network's ability to model multi-scale semantics for targets of different sizes and low-contrast regions; and improving the robustness and regional consistency of the segmentation results.

[0101] Specifically, the MMCI module includes the following processing steps:

[0102] S41. Feature Alignment and Stitching

[0103] The outputs of the second, third, and fourth stages of the encoder , , By using a convolution transformation with a preset channel number (e.g., 512), the output of the third stage... The output of the fourth stage Upsampling aligned to the output of the second stage For items of the same size, stitching them together along the channel dimension yields cross-stage aligned stitching features. .

[0104] S42, Multi-branch parallel processing

[0105] The MMCI module comprises three parallel sub-modules: a Masked Context Modulation (MCM) module and an identity direct-connect branch. These three sub-modules are configured with different receptive fields: ultra-wide, wide, and micro receptive fields, with corresponding parameter settings as follows:

[0106] First submodule: Initial window Hierarchical depth ;

[0107] Second submodule: Initial window Hierarchical depth ;

[0108] Third submodule: Initial window Hierarchical depth .

[0109] For each submodule, perform the following operations:

[0110] Cross-stage alignment splicing features The input module performs channel normalization to obtain normalized concatenated features. These normalized concatenated features are then linearly mapped to obtain the query vector. With context vector ;

[0111] For context vectors application Level-deep separable convolution:

[0112]

[0113] in, As the initial window, The kernel size is Depth-separable convolution;

[0114] Combine the mask with the context:

[0115]

[0116] Among them, the mask tensor The generation method is as follows: randomly select a retention region from the sample, control the retention ratio within a preset range (e.g., 0 to 0.5), and set the remaining regions to a large negative constant; combine all hierarchical context vectors with the global semantic vector. polymerization:

[0117]

[0118] By using a linear transformation guided by the query vector and then weighting and integrating the query vector with context modulation weights, the i-th branch mask context modulation feature is generated. :

[0119] S43, Multi-branch feature fusion

[0120] Output all submodules Directly connected branch characteristics with identity The data is then concatenated and processed sequentially using convolution, batch normalization, and the ReLU activation function.

[0121]

[0122] Obtain semantic enhancement features .

[0123] S5. Input the edge structure enhancement feature map and the semantic enhancement feature map into the Feature Similarity Complementarity Topology Fusion (SCTF) module, and perform feature fusion through similarity bidirectional topology propagation and complementarity bidirectional topology propagation to obtain the fused feature map;

[0124] See Figure 4 The first input of the SCTF module is connected to the output of the ETR module, which is the edge structure enhancement feature, and the second input is connected to the output of the MMCI module, which is the semantic enhancement feature. It is used to perform feature fusion through two pathways: similarity propagation and complementary interaction.

[0125] The innovation of this invention in the SCTF module lies in proposing a dual-path topology fusion mechanism that combines similarity propagation and complementary interaction. This mechanism maps the edge structure enhancement features output by the ETR module and the semantic enhancement feature map output by the MMCI module to a unified graph node space. Topological interaction between high- and low-level features is achieved through bidirectional similarity propagation and bidirectional difference complementarity modeling, respectively. Then, dynamic fusion weights adaptively integrate the propagation results of the two paths. This results in a technical effect that preserves high-level semantic localization capabilities while compensating for low-level edge details, reducing semantic misalignment and redundant noise propagation, and improving the segmentation accuracy of blurred boundaries and fine structures. Specifically, the SCTF module includes the following processing steps:

[0126] S51, Feature Preprocessing

[0127] semantic enhancement features Upsampling to edge structure enhancement features Same size; enhance edge structure features With semantic enhancement features The first feature is obtained by sequentially processing the data through convolution, batch normalization, and ReLU activation. Second feature Adaptive average pooling is used to process the first feature. and the second feature Construct as a node in the first graph , and the nodes in the second diagram .

[0128] S52, Similarity Topology Propagation

[0129] For the nodes in the first graph , and the nodes in the second diagram Perform L2 normalization on each side and calculate the bidirectional similarity matrix:

[0130]

[0131] in, The temperature parameter is learnable;

[0132] Calculate the bidirectional propagation features based on similarity:

[0133]

[0134] The similarity-based bidirectional propagation features are fused to obtain the fused propagation features. and :

[0135]

[0136] in, These are learnable weight parameters;

[0137] Characteristics of converged propagation and After splicing and centering, the similarity topology propagation fusion features are obtained through node convolution operations:

[0138]

[0139] S53, Complementary Topology Propagation

[0140] The bidirectional difference map is calculated by element-wise subtraction, and the difference features are enhanced by convolutional layers and the ReLU activation function.

[0141]

[0142] Learnable parameters Dynamic regulation of complementary features contributes:

[0143]

[0144] Integrate complementary information from both sides:

[0145]

[0146] complementary and fusion characteristics By performing information transfer between nodes, complementary topology propagation fusion characteristics are obtained:

[0147]

[0148] S54, Dual-path dynamic fusion

[0149] The similarity topology propagation fusion feature and the complementarity topology propagation fusion feature are dynamically fused together, and then combined with the semantic enhancement feature. After convolution and batch normalization, the features are summed to obtain the final bidirectional propagation features based on similarity and complementarity topology:

[0150]

[0151] in, The weights are learnable parameters, and BN stands for Batch Normalization.

[0152] S6. Upsample and convolution the fused feature map to output a segmentation result map of the same size as the input image.

[0153] Specifically, the fused feature map output by the SCTF module is sequentially passed through convolutional layers and upsampling layers to restore it to the same spatial resolution as the input image, which is then output as the final segmentation result map.

[0154] The comparative experimental results of this invention and the prior art are as follows:

[0155] Table 1: Comparison of Model Metrics with Polyp Dataset

[0156]

[0157] This embodiment compares and evaluates the present invention with various existing methods on five colonoscopy polyp datasets. Experimental results show that the present invention achieves an average Dice coefficient of 84.61 and an average IoU of 77.31, reaching the best level among all compared methods. On the CVC-ClinicDB dataset, the present invention achieves a Dice coefficient of 94.22, an improvement of 1.24 compared to the PolypPVT method. On complex background datasets such as CVC-ColonDB and ETIS-LaribPolypDB, the present invention significantly outperforms existing methods in both Dice and IoU. Specific comparison results are shown in Table 1.

[0158] Table 2: Comparison of Model Metrics for the Synapse Dataset

[0159]

[0160] On the Synapse multi-organ CT dataset, the average Dice coefficient of this invention reached 82.71, and the Hausdorff distance (HD) reached 16.12, both of which are superior to all comparable methods. Compared with TransUNet, the average Dice coefficient is improved by 5.23, and the HD score is reduced by 15.57. The segmentation of organs such as the left kidney, right kidney, liver, and stomach achieved optimal or near-optimal results. Specific segmentation results for each organ are shown in Table 2.

[0161] Table 3: Comparison of Model Indicators for BUSI Dataset

[0162]

[0163] On the BUSI breast ultrasound dataset, the Dice coefficient of this invention reached 81.82, IoU reached 72.89, MAE was 0.0343, and HD was 22.57, achieving the best performance among all comparison methods. Specific comparison results are shown in Table 3.

[0164] Ablation experiment:

[0165] Table 4: Ablation Test Table

[0166]

[0167] To verify the effectiveness of the modules proposed in this invention, this embodiment conducted ablation studies on the CVC-ClinicDB and ETIS-LaribPolypDB datasets. The ETR module was removed from the overall network architecture and labeled as a variant model (w / o ETR). The experimental results are shown in Table 4. On the CVC-ClinicDB dataset, the mDice metric of the model decreased by 1.70 after removing the ETR module. Figure 6 The feature heatmap analysis shown indicates that the complete model with the ETR module significantly outperforms the model lacking the ETR module in understanding edge structures. The first column displays the input image and its corresponding ground truth mask, alternating rows. Columns 2 through 4 show the feature map comparisons of the ETR, MMCI, and SCTF modules, respectively; odd numbers represent the complete DVF model, and even numbers represent the corresponding ablation model (without the module). The MMCI module is removed from the overall network and labeled as a variant model (w / o MMCI). Experimental results show that on the ETIS-LaribPolypDB dataset, removing the MMCI module reduces the model's mDice and mIoU metrics by 3.13 and 3.37, respectively. Feature heatmap analysis (see [link to feature heatmap analysis]). Figure 6 The results show that introducing the MMCI module allows the feature heatmap to focus more on the target region and pay more attention to the overall semantic information of the image. The lack of the MMCI module leads to weakened multi-scale context information modulation capability and decreased noise resistance, thus reducing the model's generalization performance. The SCTF module was removed from the overall network and replaced with element-wise addition, labeled as a variant model (w / o SCTF). Experimental results show that on the ETIS-LaribPolypDB dataset, removing the SCTF module reduced the model's mDice and mIoU metrics by 1.72 and 2.02, respectively. Feature heatmap analysis (see...) Figure 6The results show that after adding the SCTF module, the feature heatmap exhibits a highly concentrated response in the target region, showing a significant difference from non-target regions. Without this module, the model's attention to the target region and its discrimination against surrounding regions are low, introducing irrelevant background noise. This indicates that the SCTF module can effectively enhance the discriminative power of features, reduce redundant information and feature conflicts, and thus improve segmentation accuracy. Comparing the base model with all three modules removed with the complete model containing all modules, on the CVC-ClinicDB and ETIS-LaribPolypDB datasets, the combination of the three modules significantly increases the Dice coefficient by 4.07 and 5.55, respectively. This result fully validates that the proposed module combination can effectively extract deep semantic information and significantly improve image segmentation performance.

[0168] Table 5: Ablation Experiment Configuration Table for MMCI Module

[0169]

[0170] To systematically explore the importance of multi-scale design in the MMCI module, this embodiment designed and evaluated various module configurations (as shown in Table 5), and the results are as follows: Figure 7 As shown. Branch number comparison: The model with two parallel branches performs better in segmentation than the model with a single branch, indicating that a single branch is insufficient to effectively capture multi-scale features. The absence of any branch leads to performance degradation, demonstrating the irreplaceable role of each branch in capturing specific scale information. Parallel convolution number comparison: When each branch contains only a single convolution, model performance significantly decreases; introducing parallel convolution structures improves segmentation performance. However, more parallel convolutions are not necessarily better, as excessive parallel convolutions may introduce parameter redundancy and overfitting risks. Hierarchical parallel design: The final configuration of this invention adopts a hierarchical parallel design—large-scale branches are allocated more parallel convolution kernels (L=7) to ensure effective capture of global contextual information, while small-scale branches are allocated fewer parallel convolution kernels (L=1) to focus on extracting local detailed features. This hierarchical parallel design concept ensures effective fusion of multi-scale features. The reverse parallel allocation strategy causes window aliasing between different sub-modules, leading to model performance degradation, which conversely confirms the effectiveness of the design strategy of this invention.

[0171] The mask mechanism was removed from the MMCI module, resulting in a variant model (w / o mask). Experimental results show that removing the mask mechanism reduced the model's Dice and IoU by 0.68 and 0.49, respectively, demonstrating the importance of the mask mechanism in the MMCI module.

[0172] Visualization results analysis:

[0173] This embodiment performs a visual comparative analysis of segmentation results on multiple image datasets, and the results are as follows: Figure 5 As shown.

[0174] In low-contrast scenarios, taking colonoscopy images as an example, when there is a lack of significant difference between the target and the surrounding background, other contrast models show varying degrees of segmentation deviation and severe false positives. However, this invention, with its powerful detection sensitivity, can still achieve accurate differentiation between the target and the background, demonstrating a significant advantage under low-contrast conditions.

[0175] Even when the image is blurry and the target is poorly distinguishable from the surrounding area, this invention can still accurately locate the target position, while other models exhibit undersegmentation or oversegmentation.

[0176] In target segmentation tasks with blurred edges and complex shapes, taking breast ultrasound images as an example, the target region is not only difficult to distinguish from the surrounding background, but also has a complex and varied shape. The segmentation results of this invention on such images are highly close to the real annotation in terms of edge localization accuracy, and can accurately capture the complete shape of the target. Other models, on the other hand, have relatively coarse edge processing when dealing with such targets, resulting in incomplete segmentation or over-segmentation.

[0177] In multi-organ segmentation tasks, taking abdominal CT images as an example, this invention accurately identifies and segments difficult-to-segment organs such as the gallbladder and pancreas, while other models generally suffer from missed detections or missegmentation. Simultaneously, this invention maintains high-precision segmentation of the left and right kidneys, achieving more complete preservation of the target contours.

[0178] Characteristic heatmap analysis (e.g.) Figure 6 As shown, this further confirms that the ETR module enhances the understanding of edge structures, the MMCI module makes the feature heatmap more focused on the target region, and the SCTF module makes the feature heatmap show a highly concentrated response in the target region. The three modules work together to achieve segmentation performance that is significantly better than the base model.

[0179] In summary, the visualization results fully demonstrate that this invention, through functional simulation of dorsal and ventral flow and its interactive fusion mechanism, effectively suppresses background noise interference, enhances fine-grained feature expression, and exhibits stronger robustness and detailed structure restoration capabilities in complex image analysis.

[0180] Data sets and evaluation metrics:

[0181] The evaluation indicators are as follows:

[0182] Polyp dataset: Dice coefficients, Intersection over Union (IoU)

[0183] Multi-organ CT dataset: Mean Dice coefficient, Mean Hausdorff distance (HD)

[0184] Breast ultrasound dataset: Dice coefficient, IoU, mean absolute error (MAE), Hausdorff distance (HD)

[0185] Example 2:

[0186] This embodiment uses the PyTorch 2.4.0 framework to train the model on a single NVIDIA GeForce RTX 4090 GPU (24GB VRAM).

[0187] Optimizer and learning rate: The AdamW optimizer was used, with the β parameter set to (0.9, 0.999), weight decay to 1e-4, and initial learning rate to 1e-4.

[0188] Polyp segmentation task:

[0189] Training rounds: 150 rounds, with the learning rate decayed at rounds 50, 90, and 120.

[0190] Data augmentation: random rotation, random horizontal flip, random vertical flip, color jitter

[0191] Input image size: 352×352

[0192] Batch size: 16

[0193] Loss function: Deep supervised training strategy, the main output and two shallow auxiliary outputs all use a composite loss function of weighted binary cross-entropy (wBCE) and weighted intersection-over-union (wIoU).

[0194] Multi-scale training: Employing a scaling strategy of {0.75, 1.0, 1.25}.

[0195] Breast ultrasound segmentation task:

[0196] Training rounds: 150 rounds, with the learning rate decayed at rounds 50, 90, and 120.

[0197] Data augmentation: random horizontal flip, random vertical flip

[0198] Input image size: 256×256

[0199] Batch size: 16

[0200] Loss function: Similar to the polyp segmentation task, a composite loss function of wBCE and wIoU and a deep supervision strategy are adopted.

[0201] Multi-scale training: Employing a scaling strategy of {0.75, 1.0, 1.25}.

[0202] Multi-organ segmentation task:

[0203] Training rounds: 150 rounds, with the learning rate decayed at rounds 50 and 100.

[0204] Data augmentation: random rotation, random horizontal flip, random vertical flip, random scaling

[0205] Input image size: 224×224

[0206] Batch size: 16

[0207] Loss function: In the deep supervised training strategy, the loss function of the three output layers is a weighted sum, and the loss of each layer consists of cross-entropy loss and Dice loss.

[0208] Example 3:

[0209] Image segmentation systems based on functional biomimetic dual-path fusion include:

[0210] The data acquisition module is used to acquire and preprocess the image to be segmented;

[0211] The image segmentation network model includes a dual-path decoding structure, an edge topology refinement module, a multi-scale mask context modulation module, and a feature similarity complementary topology fusion module.

[0212] The process involves inputting the image to be segmented into a dual-path decoding structure for multi-scale feature extraction, resulting in shallow and deep feature maps. The shallow feature map is then input into an edge topology refinement module, where dynamic topology propagation and geometric reconstruction enhance edge structure representation, yielding an edge structure-enhanced feature map. The deep feature map is input into a multi-scale mask context modulation module, where mask-guided multi-scale feature aggregation suppresses noise and captures discriminative semantics, resulting in a semantically enhanced feature map. Finally, the edge structure-enhanced feature map and the semantically enhanced feature map are input into a feature similarity and complementarity topology fusion module, where similarity-based bidirectional topology propagation and complementarity-based bidirectional topology propagation are used for feature fusion, resulting in a fused feature map.

[0213] The generation module is used to generate segmentation result maps based on the fused feature maps.

Claims

1. An image segmentation method based on functional biomimetic dual-path fusion, characterized in that, Includes the following steps: S1. Acquire and preprocess the image to be segmented; S2. Input the image to be segmented into the encoder for multi-scale feature extraction to obtain shallow feature maps and deep feature maps; S3. Input the shallow feature map into the constructed edge topology refining module, and enhance the edge structure representation through dynamic topology propagation and geometric reconstruction to obtain an enhanced edge structure feature map; including: S31. Extract a binary edge map of the same size from the image to be segmented; generate an edge attention mask based on the binary edge map; S32. The shallow feature map is processed by convolution to obtain shallow convolution features, and then projected onto the edge attention mask to obtain projected features; at the same time, the shallow convolution features are normalized to obtain shallow normalized projection vectors. S33. Adaptive average pooling is used to extract key response points from the projection features, and cross-scale anchor features are established in the feature map; L2 normalization encoding is performed on the anchor features to obtain normalized anchor features. S34. Calculate the directional similarity between the shallow normalized projection vector and the normalized anchor point feature to obtain the similarity matrix; aggregate the shallow normalized projection vector to the anchor point feature through the similarity matrix to obtain the anchor point aggregated feature. S35. Perform L2 normalization on the anchor point aggregation features to obtain normalized anchor point aggregation features; calculate the similarity matrix between the normalized anchor point aggregation feature nodes and construct an adaptive adjacency graph; use the adjacency graph for feature propagation, calculate the structural propagation features of each node, and calculate the difference with the anchor point aggregation features; finally, perform convolution in the node dimension. S36. The anchor points after the adjacency graph features are propagated are mapped back to the location space by projection of the similarity matrix to obtain the location mapping features. After the location mapping features are processed by convolution normalization, they are residually connected with the shallow feature map to obtain the edge structure enhancement features. S4. Input the deep feature map into the constructed multi-scale mask context modulation module, suppress noise and capture discriminative semantics through mask-guided multi-scale feature aggregation, and obtain semantically enhanced feature map; S5. Input the edge structure enhancement feature map and semantic enhancement feature map into the constructed feature similarity and complementarity topology fusion module, and perform feature fusion through similarity bidirectional topology propagation and complementarity bidirectional topology propagation to obtain the fused feature map; S6. Generate segmentation result map based on fused feature map.

2. The image segmentation method based on functional biomimetic dual-path fusion according to claim 1, characterized in that, S2 include: The encoder adopts a pyramid visual Transformer architecture, which consists of four stages. It divides the image to be segmented into image blocks with overlapping regions through overlapping block embedding, and generates multi-scale feature maps while downsampling layer by layer through a hierarchical pyramid structure. The output of the first stage of the encoder is used as the shallow feature map, and the outputs of the second, third and fourth stages of the encoder are used together as the deep feature map.

3. The image segmentation method based on functional biomimetic dual-path fusion according to claim 1, characterized in that, S4 include: S41. Upsample and align the outputs of the second, third, and fourth stages of the encoder and then concatenate them along the channel dimension to obtain cross-stage aligned concatenation features. S42. Input the cross-stage aligned splicing features into three parallel sub-modules and one identity direct connection branch respectively; configure different receptive fields for the three sub-modules and generate mask context modulation features for each sub-module respectively; S43. The mask context modulation features of each sub-module are concatenated with the cross-stage aligned concatenation features output by the identity direct connection branch, and then processed by convolution, batch normalization and ReLU activation function in sequence to obtain semantic enhancement features.

4. The image segmentation method based on functional biomimetic dual-path fusion according to claim 3, characterized in that, In S42, the following operations are performed within each submodule: The cross-stage aligned splicing features are channel normalized to obtain normalized splicing features. The normalized splicing features are then linearly mapped to obtain query vectors and context vectors. The global semantic vector is obtained based on the context vectors. Multi-level depthwise separable convolution is applied to the context vector, and the context after depthwise separable convolution is combined with the mask tensor; the results of combining all levels are aggregated with the global semantic vector to generate context modulation weights. By using a linear transformation guided by the query vector, and then using context modulation weights to weight and integrate the query vector, a masked context modulation feature is generated.

5. The image segmentation method based on functional biomimetic dual-path fusion according to claim 4, characterized in that, In S42, the mask tensor is generated by randomly selecting a reserved region within a preset range in the sample, and setting the remaining regions to large negative constants.

6. The image segmentation method based on functional biomimetic dual-path fusion according to claim 1, characterized in that, S5 include: S51. After aligning the edge structure enhancement features and semantic enhancement features by size, extract features separately to obtain the first feature and the second feature. Then, use adaptive average pooling to construct graph nodes to generate the first graph node and the second graph node. S52. Input the nodes of the first graph and the nodes of the second graph into the similarity topology fusion module to generate similarity topology propagation fusion features; S53, Move the nodes in the first diagram , and the nodes in the second diagram Input the complementary topology fusion module to generate complementary topology propagation fusion features; S54. After fusing the similarity topology propagation fusion features and the complementarity topology propagation fusion features according to a dynamic ratio, perform convolution and batch normalization processing. Add the processing result to the semantic enhancement features after convolution and batch normalization to obtain the fused feature map.

7. The image segmentation method based on functional biomimetic dual-path fusion according to claim 6, characterized in that, S52 includes: L2 normalization is performed on the nodes of the first graph and the nodes of the second graph respectively, and the bidirectional similarity matrix is ​​calculated. The bidirectional propagation features are calculated based on the bidirectional similarity matrix. The bidirectional propagation features are weighted and fused to obtain two fused propagation features. The two fused propagation features are concatenated and centered, and then the similarity topology propagation fused features are obtained through node convolution operation.

8. The image segmentation method based on functional biomimetic dual-path fusion according to claim 6, characterized in that, S53 includes: Based on the nodes of the first and second graphs, a bidirectional difference graph is calculated by element-wise subtraction, and enhanced difference features are generated by convolutional layers and ReLU activation functions. The contribution of enhanced difference features is dynamically adjusted by learnable parameters, and the obtained bidirectional complementary information is fused to generate complementary fusion features. Node convolution is performed on the complementary fusion features to obtain complementary topology propagation fusion features.

9. An image segmentation system based on functional biomimetic dual-path fusion, characterized in that, include: The data acquisition module is used to acquire and preprocess the image to be segmented; The image segmentation network model includes a dual-path decoding structure, an edge topology refinement module, a multi-scale mask context modulation module, and a feature similarity complementary topology fusion module. The process involves inputting the image to be segmented into a dual-path decoding structure for multi-scale feature extraction, resulting in shallow and deep feature maps. The shallow feature map is then input into an edge topology refinement module, where dynamic topology propagation and geometric reconstruction enhance edge structure representation, yielding an edge structure-enhanced feature map. The deep feature map is input into a multi-scale mask context modulation module, where mask-guided multi-scale feature aggregation suppresses noise and captures discriminative semantics, resulting in a semantically enhanced feature map. Finally, the edge structure-enhanced feature map and the semantically enhanced feature map are input into a feature similarity and complementarity topology fusion module, where similarity-based bidirectional topology propagation and complementarity-based bidirectional topology propagation are used for feature fusion, resulting in a fused feature map. The generation module is used to generate segmentation result maps based on the fused feature maps; The image segmentation system based on functional biomimetic dual-path fusion is used to implement the image segmentation method and steps based on functional biomimetic dual-path fusion as described in claim 1.

Images