A method for kidney tumor segmentation based on boundary guidance and hybrid network

Abstract

This invention discloses a method for renal tumor segmentation based on boundary guidance and hybrid networks. The method includes: acquiring endoscopic images of the renal tumor; extracting multi-scale global contextual features from the endoscopic images using PVTv2 as the backbone network of the Transformer branch, while simultaneously capturing local texture and edge detail features using HardNet-MSEG as the backbone network of the CNN branch; refining and fusing the features of the Transformer and CNN branches using the AFRF module; performing top-down semantic calibration and alignment aggregation of the fused features at different scales based on the CAMF module; adaptively and dynamically allocating the prediction map weights of the fusion branch and the CNN branch through a boundary guidance weighting mechanism, and constructing a boundary guidance hybrid loss function optimization model; and weightedly fusing the prediction results of the fusion branch and the CNN branch to obtain the final renal tumor segmentation prediction map.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and computer vision technology, and more specifically to a method for segmenting kidney tumors based on boundary guidance and hybrid networks. Background Technology

[0002] Kidney tumors are among the most common malignant tumors of the urinary system, with renal cell carcinoma (RCC) accounting for over 90% of all kidney malignancies. Statistics show that more than 400,000 new cases of kidney cancer are diagnosed globally each year, and the incidence rate is increasing annually. Accurate segmentation of kidney tumors in endoscopic images is of great value for clinical diagnosis, surgical planning, and postoperative evaluation. However, factors such as the complex distribution of fat around kidney tumors, intraoperative bleeding, and obstruction by surgical instruments often result in blurred tumor boundaries and low contrast in images, posing a significant challenge to automated segmentation.

[0003] Traditional image segmentation methods rely on manual feature design, which struggles to handle the morphological diversity and complex background interference of kidney tumors. In recent years, deep learning technology has made significant progress in medical image segmentation. Convolutional Neural Networks (CNNs), with their powerful local feature extraction capabilities, have become the mainstream method for medical image segmentation. In particular, U-Net and its variants, through their encoder-decoder structure and skip connection design, have achieved excellent performance in various medical image segmentation tasks. However, the inherent local receptive field of CNNs limits their ability to model global contextual information, easily leading to incomplete segmentation or blurred boundaries when dealing with targets like kidney tumors with complex backgrounds and irregular shapes.

[0004] Meanwhile, Transformer networks based on self-attention mechanisms exhibit superior global context awareness by modeling long-range dependencies. Vision Transformer and its variants, such as PVTv2 and Swin-Transformer, have achieved good results in image segmentation tasks through hierarchical design. However, Transformer networks still fall short in capturing local details and edge texture information, and require a large amount of training data to fully realize their performance advantages.

[0005] To combine the local feature extraction capabilities of CNNs with the global context modeling advantages of Transformers, researchers have proposed several hybrid CNN-Transformer architectures. TransUNet is the first to integrate Transformers into the U-Net framework, while TransFuse employs a dual-branch parallel structure to simultaneously extract complementary features from both CNNs and Transformers. However, existing hybrid methods typically use simple feature concatenation or element-wise weighted summation strategies to fuse features from different branches, failing to fully consider the heterogeneity of CNN and Transformer features in statistical distribution and semantic hierarchy, leading to feature redundancy or mutual inhibition. Furthermore, existing methods lack the ability to explicitly model tumor boundary information, leaving significant room for improvement in segmentation accuracy and boundary localization capabilities in renal tumor segmentation tasks with blurred boundaries and complex backgrounds.

[0006] To address the issue of blurred boundaries in medical image segmentation, existing methods mainly fall into two categories: adaptive boundary extraction methods and explicit boundary prediction methods. Adaptive methods, through the design of specific attention mechanisms or feature enhancement modules, enable the model to learn boundary-related features without additional boundary annotations. However, they struggle to achieve stable results when dealing with targets facing strong background noise and irregular boundary heights. Explicit boundary prediction methods introduce boundary segmentation as an auxiliary task into a multi-task learning framework. However, the existing interaction mechanisms between the boundary prediction task and the main segmentation task are relatively simple, failing to fully utilize boundary information to dynamically guide the fusion process of multi-branch features.

[0007] Therefore, how to design a more effective boundary guidance mechanism under a hybrid network framework, so that boundary cues can play a substantial guiding role in the segmentation process from feature extraction and feature fusion to the final decision, is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0008] In view of this, the present invention provides a kidney tumor segmentation method based on boundary guidance and hybrid networks, which can effectively integrate global context and local detailed features, enhance the perception of kidney tumor boundaries, and solve the problem of blurred boundaries in kidney tumor segmentation under complex backgrounds.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: A kidney tumor segmentation method based on boundary guidance and hybrid networks includes: Step 1: Obtain endoscopic images of the kidney tumor; Step 2: PVTv2 is used as the backbone network of the Transformer branch to extract multi-scale global contextual features in the endoscopic images, while HardNet-MSEG network is used as the backbone network of the CNN branch to capture local texture and edge detail features. Step 3: The features of the Transformer branch and the CNN branch are refined and adaptively fused using the Adaptive Feature Refinement and Fusion (AFRF) module to obtain fused features at different scales; Step 4: Based on the multi-scale feature calibration and aggregation module CAMF, perform top-down semantic calibration and alignment aggregation on the fused features at different scales; Step 5: Adaptively and dynamically allocate the prediction graph weights of the fusion branch and the CNN branch through the boundary-guided weighting mechanism, and construct the boundary-guided hybrid loss function optimization model; Step 6: Perform weighted fusion of the prediction results from the fusion branch and the CNN branch to obtain the final kidney tumor segmentation prediction map.

[0010] Preferably, the PVTv2 backbone network in step 2 includes a four-stage encoder for extracting pyramid features at different scales layer by layer; the HardNet-MSEG network preserves and restores the high-frequency edges and texture details of the endoscopic image through step-by-step downsampling and feature upsampling operations.

[0011] Preferably, the adaptive feature refinement and fusion module AFRF in step 3 adopts a parallel processing strategy of independent channel refinement and dynamic gating weighting, including: The features input from the CNN branch and the Transformer branch are convolved and cross-fused, and then refined by the efficient channel attention module ECA to obtain the refined features. An adaptive gating branch is introduced, which concatenates the features of the original CNN branch and the Transformer branch through channels, extracts joint features through convolution, and forms dynamic gating weights through feature activation. The probability maps of pixels of the same size after Sigmoid activation are multiplied by their respective refined features, and the fused features of different scales are obtained by adaptive weighted summation.

[0012] Preferably, step 4 specifically includes: The fused features at different scales are flattened by the GBR (Granular Sphere Refinement) module, the attention weights from each spatial location to the center of the anisotropic ellipsoid are calculated, and the features are aggregated according to the weights to complete the reconstruction of the semantic core. Then, the features are flattened back to restore the refined feature map. Convolution and upsampling are performed on the fusion features at different scales to align and fuse high-resolution features with low-resolution features step by step. The highest-level features and multi-level fused features are concatenated through channels, and the aggregated features are output after convolution and activation.

[0013] Preferably, the boundary-guided weighting mechanism in step 5 dynamically adjusts the fusion weights of the fusion branch and the CNN branch by introducing a boundary prediction auxiliary task. The boundary-guided weighting formula is as follows: ; in, This represents the fusion weight of the nth branch. This indicates the weights of the introduced boundary prediction auxiliary task; Based on the above formulas, the calculation formulas for the fusion branch weights and CNN branch weights are as follows: ; ; ; ; in, Indicates the weight of the merged branch. Indicates the weights of CNN branches. This represents the loss value of the main splitting task in the nth branch. , and These represent the loss values ​​for the fusion branch, the CNN branch, and the boundary prediction auxiliary task, respectively. This indicates the weights of the fusion branch boundary prediction auxiliary task. This represents the weights of the CNN branch boundary prediction auxiliary task, where m represents the number of training epochs. Predict the weighting coefficients for the boundaries.

[0014] Preferably, in step 5, an inner boundary-guided hybrid loss function optimization model is constructed using the output results of the main segmentation task and the boundary prediction auxiliary segmentation task for kidney tumors as the optimization objective. This model includes a main segmentation loss composed of a weighted intersection-over-union loss function and a weighted binary cross-entropy loss function, and a boundary prediction auxiliary loss composed of a Dice loss function, as shown in the following formula: ; in, The boundary-guided hybrid loss function BGH,L p L represents the loss function for the main segmentation task. b This represents the loss function for the boundary prediction auxiliary task. and These represent the real masks for the main segmentation task and the real masks for the auxiliary task, respectively. and These represent the prediction graphs for the primary task and the auxiliary task, respectively. This represents the weighting coefficient of the auxiliary task.

[0015] As can be seen from the above technical solutions, compared with the prior art, this invention provides a kidney tumor segmentation method based on boundary guidance and hybrid networks. It uses PVTv2 as a Transformer branch to extract multi-scale global contextual information, resulting in stronger global semantic perception capabilities. HardNet-MSEG is used as a CNN branch to capture local texture and edge details, preserving rich spatial detail information. The adaptive feature refinement and fusion module AFRF performs cross-fusion refinement and dynamic gating weighting of features from the two heterogeneous branches, solving the problem of feature redundancy or mutual inhibition caused by the heterogeneity of CNN features and Transformer features at the distribution space and semantic level. The multi-scale feature calibration and aggregation module CAMF introduces particle-sphere computation for feature propagation calibration, filtering redundant features and noise, adaptively solving the spatial misalignment problem between heterogeneous multi-scale features. The boundary guidance weighting mechanism dynamically adjusts branch weights using the boundary prediction auxiliary task, and combines the boundary guidance hybrid loss function BGH to jointly optimize the main segmentation task and the boundary prediction auxiliary task, effectively enhancing the network's spatial constraint ability on the kidney tumor boundary region and alleviating the boundary blurring problem caused by complex backgrounds. In summary, this invention can effectively integrate global context and local detailed features, enhance the ability to perceive tumor boundaries, and achieve high-precision kidney tumor segmentation in complex scenarios. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the method flow provided by the present invention.

[0018] Figure 2 This is a schematic diagram of the AFRF module structure provided by the present invention.

[0019] Figure 3 This is a schematic diagram of the CAMF module structure provided by the present invention.

[0020] Figure 4 This is a visual comparison diagram of the segmentation results of different methods provided by the present invention on kidney tumor images.

[0021] Figure 5This is a visual comparison diagram of the attention of each branch provided by the present invention.

[0022] Figure 6 This is a visual comparison diagram of the ablation experiments of each module provided by the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] like Figure 1 As shown, this embodiment of the invention discloses a method for segmenting kidney tumors based on boundary guidance and hybrid networks, including: Step 1: Obtain endoscopic images of the kidney tumor; Step 2: PVTv2 is used as the backbone network of the Transformer branch to extract multi-scale global contextual features in the endoscopic images, while HardNet-MSEG network is used as the backbone network of the CNN branch to capture local texture and edge detail features. Step 3: The features of the Transformer branch and the CNN branch are refined and adaptively fused using the Adaptive Feature Refinement and Fusion (AFRF) module to obtain fused features at different scales; Step 4: Based on the multi-scale feature calibration and aggregation module CAMF, perform top-down semantic calibration and alignment aggregation on the fused features at different scales; Step 5: Adaptively and dynamically allocate the prediction graph weights of the fusion branch and the CNN branch through the boundary-guided weighting mechanism, and construct the boundary-guided hybrid loss function optimization model; Step 6: Perform weighted fusion of the prediction results from the fusion branch and the CNN branch to obtain the final kidney tumor segmentation prediction map.

[0025] Specifically, the PVTv2 backbone network in step 2 includes a four-stage encoder for extracting pyramid features at different scales layer by layer; the HardNet-MSEG network preserves and restores the high-frequency edges and texture details of the endoscopic image through step-by-step downsampling and feature upsampling operations.

[0026] In a specific embodiment of this invention, an asymmetric encoder-decoder architecture is used to construct the hybrid network TransBGNet. The left Transformer branch uses PVTv2 as the backbone network, containing a four-stage encoder that extracts multi-scale global contextual information layer by layer. The right CNN branch uses the HardNet-MSEG network, which preserves and restores high-frequency edges and texture details of the image through progressive downsampling and feature upsampling operations. PVTv2 achieves hierarchical multi-scale feature extraction while reducing the computational overhead of self-attention through a progressively shrinking pyramid structure and overlapping block embedding strategy. HardNet-MSEG employs a hard attention mechanism and a multi-scale feature fusion strategy, demonstrating excellent local detail extraction performance in medical image segmentation with complex backgrounds. The parallel processing of the two branches provides complementary global semantic features and local detail features for the subsequent fusion stage.

[0027] Specifically, the adaptive feature refinement and fusion module AFRF in step 3 adopts a parallel processing strategy of independent channel refinement and dynamic gating weighting, including: The features input from the CNN branch and the Transformer branch are convolved and cross-fused, and then refined by the efficient channel attention module ECA to obtain the refined features. An adaptive gating branch is introduced, which concatenates the features of the original CNN branch and the Transformer branch through channels, extracts joint features through convolution, and forms dynamic gating weights through feature activation; The probability maps of pixels of the same size after Sigmoid activation are multiplied by their respective refined features, and the fused features of different scales are obtained by adaptive weighted summation.

[0028] In a specific embodiment of the present invention, the features of the Transformer branch and the CNN branch are dynamically refined and adaptively fused by the adaptive feature refinement and fusion module AFRF.

[0029] To fully leverage the advantages and compensate for the shortcomings of CNN and Transformer branches, and based on the strengths of CNN in extracting local texture and edge features and the strength of Transformer in capturing long-range global contextual dependencies, an Adaptive Feature Refinement and Fusion module (AFRF) was designed. This module employs a strategy of independent channel refinement and dynamically gated weighted parallel processing to achieve adaptive dynamic feature fusion while minimizing the loss of original feature information. This strategy reduces the differences between these two heterogeneous features in terms of distribution space and semantics, mitigating the problem of feature redundancy or mutual inhibition caused by the network's inability to adaptively adjust the contribution weights of each feature to different target scenes. The overall structure of this module is as follows: Figure 2 As shown.

[0030] First, the features input from the CNN and Transformer branches are convolved and cross-fused, then refined using an Efficient Channel Attention (ECA) module. This design enhances the model's focus on important features and effectively suppresses interference from the complex background of kidney tumors. The above process is illustrated in the following formula: ; in, and This indicates the characteristics after cross-fertilization and refinement. and These represent the features input to the Transformer branch and the CNN branch, respectively.

[0031] Secondly, to dynamically evaluate the importance of different branches in the current scenario, an adaptive gating branch is introduced. This branch concatenates the features of the two original branches, then extracts joint features through convolution, and forms dynamic gating weights through feature activation. This design integrates global contextual information and local features, enhancing feature synergy. The above process is shown in the following formula: ; in, and These represent the weight matrices for the CNN branch and the Transformer branch, respectively.

[0032] Finally, the probability map of pixels of the same size after Sigmoid activation is... and Each feature is multiplied by its respective refined feature to obtain the final adaptive weighted feature map. Through adaptive weighted summation, the contributions of different branches can be dynamically adjusted to better control their contribution, mitigating feature redundancy and mutual inhibition. The above process is illustrated in the following formula: ; Here, ⊙ represents element-wise multiplication. This represents the feature output after fusion and weighting.

[0033] The AFRF module fully leverages the advantages of local features from CNN branches and global contextual information from Transformer branches through cross-fusion refinement and dynamic gating weighting strategies, achieving reasonable dynamic fusion and providing more refined input features for the subsequent aggregation part.

[0034] Specifically, step 4 includes: The fused features at different scales are flattened by the GBR (Granular Sphere Refinement) module, the attention weights from each spatial location to the center of the anisotropic ellipsoid are calculated, and the features are aggregated according to the weights to complete the reconstruction of the semantic core. Then, the features are flattened back to restore the refined feature map. Convolution and upsampling are performed on the fusion features at different scales to align and fuse high-resolution features with low-resolution features step by step. The highest-level features and multi-level fused features are concatenated through channels, and the aggregated features are output after convolution and activation.

[0035] In a specific embodiment of the present invention, the multi-scale feature calibration and aggregation module (CAMF) performs top-down semantic calibration and alignment aggregation on fused features of different scales.

[0036] To fully aggregate fused features from three different scales, a multi-scale feature calibration and aggregation module (CAMF) was designed. This module constructs a top-down semantic calibration and multi-scale aggregation framework and introduces particle sphere computation as a calibrator for feature propagation. This strategy can filter redundant features and noise and adaptively address the spatial misalignment problem between heterogeneous multi-scale features. The overall module structure is as follows: Figure 3 As shown.

[0037] First is the Granular Ball Refinement (GBR) module. After convolution and batch normalization, the input features are flattened, and the attention weights from each spatial location to the center of the anisotropic ellipsoid are calculated. These weighted features are then aggregated to reconstruct the semantic core, and finally, the flattened features are reversed to restore the refined feature map. For kidney tumor segmentation, this process enables the model to more clearly distinguish tumor boundaries, suppress background noise, and better model tumor heterogeneity, thereby improving segmentation accuracy. The above process is illustrated in the following formula: ; Among them, X in X represents the input feature. out The output features are represented by CB, which represents convolution and batch normalization, and BA represents sphere attention computation.

[0038] Secondly, the multi-scale fusion features Fi are convolved, upsampled, and then fused. The highest-level feature F1 is upsampled after convolution, batch normalization, and activation to align its resolution with that of the mid-level feature F2, and then fused. Then, the refined mid-level feature F2 is upsampled after convolution, batch normalization, and activation and fused with the low-level feature F3. Finally, the highest-level feature F1 and the two-level fusion features are concatenated channel by channel. The above process is illustrated in the following formula: ; ; ; in, and Features obtained after cross-scale fusion These are the features after the final channel splicing.

[0039] Finally, the module output is obtained after two convolutions, batch normalization, and activation, as shown in the following formula: .

[0040] The CAMF module extracts the most representative features in the middle layer by introducing a granular refinement module with skip connections, thereby better enabling subsequent feature fusion and channel splicing. This solves the problem of cross-scale semantic misalignment and provides multi-scale and accurate stable features for subsequent boundary prediction and mask prediction.

[0041] Specifically, the boundary-guided weighting mechanism in step 5 dynamically adjusts the fusion weights of the fusion branch and the CNN branch by introducing a boundary prediction auxiliary task. The boundary-guided weighting formula is as follows: ; in, This represents the fusion weight of the nth branch. This indicates the weights of the introduced boundary prediction auxiliary task; Based on the above formulas, the calculation formulas for the fusion branch weights and CNN branch weights are as follows: ; ; ; ; in, Indicates the weight of the merged branch. Indicates the weights of CNN branches. This represents the loss value of the main splitting task in the nth branch. , and These represent the loss values ​​for the fusion branch, the CNN branch, and the boundary prediction auxiliary task, respectively. This indicates the weights of the fusion branch boundary prediction auxiliary task. This represents the weights of the CNN branch boundary prediction auxiliary task, where m represents the number of training epochs. Predict the weighting coefficients for the boundary.

[0042] Specifically, in step 5, an inner boundary-guided hybrid loss function optimization model is constructed using the output results of the main segmentation task and the boundary prediction auxiliary segmentation task for kidney tumors as the optimization objective. This model includes a main segmentation loss composed of a weighted intersection-over-union loss function and a weighted binary cross-entropy loss function, and a boundary prediction auxiliary loss composed of a Dice loss function. The formula is as follows: ; in, The boundary-guided hybrid loss function BGH,L p L represents the loss function for the main segmentation task. b This represents the loss function for the boundary prediction auxiliary task. and These represent the real masks for the main segmentation task and the real masks for the auxiliary task, respectively. and These represent the prediction graphs for the primary task and the auxiliary task, respectively. This represents the weighting coefficient of the auxiliary task.

[0043] In a specific embodiment of the present invention, the prediction graph weights of the fusion branch and the CNN branch are adaptively and dynamically allocated through a boundary-guided weighting mechanism, and a boundary-guided hybrid loss function optimization model is constructed.

[0044] In the boundary-guided weighting mechanism, to effectively integrate the segmentation prediction results of the fusion branch and the CNN branch, a boundary prediction auxiliary task is introduced to dynamically adjust the fusion weights of the two branches, thereby adaptively evaluating the importance of different branches. The specific boundary-guided weighting is shown in the following formula: ; in, This represents the fusion weight of the nth branch. This represents the loss value of the main splitting task in the nth branch. represents the weights of the introduced boundary prediction auxiliary task, and m represents the training rounds.

[0045] Because the design includes a fusion branch and a CNN branch, and the final prediction result is achieved by weighted fusion of the two branches, the weights of the two branches are as follows: ; ; The weights for the boundary prediction auxiliary task in the fusion branch and the CNN branch are as follows: ; ; in, The boundary prediction weight coefficient was set to 0.55 in the experiment.

[0046] By utilizing a boundary-guided weighting mechanism on different branches, weights can be adaptively and dynamically allocated based on the boundary segmentation performance of the fusion branch, thereby making reasonable use of the advantages of different branches and achieving better segmentation performance.

[0047] In the boundary-guided hybrid loss function section, a boundary-guided hybrid loss function (BGH) is designed to address the problem of blurred boundary segmentation caused by the complex background of renal tumor segmentation. Due to the multi-task learning strategy, the proposed TransBGNet includes two loss functions. Specifically, the classic Dice loss function is used for the auxiliary task of boundary prediction. For the main renal tumor segmentation task, a hybrid loss function is employed. This function is composed of the weighted intersection-union loss function. and weighted binary cross-entropy loss function composition: ; Overall loss function Defined as: ; in, and These represent the true masks for the primary segmentation task of renal tumor segmentation and the true masks for the boundary prediction-assisted segmentation task, respectively. and These represent the prediction graphs for the primary task and the auxiliary task, respectively. The weight coefficients represent the auxiliary tasks, used to adjust the contribution of boundary supervision. In the experiment... Set it to 0.15.

[0048] By jointly training the main segmentation task of kidney tumor with the auxiliary segmentation task of boundary prediction, the model's ability to use boundary cues for spatial constraints is improved, thereby solving the problem of blurred boundaries of kidney tumor caused by fat and surgical instruments.

[0049] Furthermore, to evaluate the performance of TransBGNet, this embodiment uses the Re-TMRS dataset of endoscopic images of kidney tumors for training and testing. This dataset contains 2823 images with resolutions ranging from 719×516 to 1920×1080, using real-world kidney tumor resection surgery videos, and is labeled frame-by-frame under the guidance of urologists. In this experiment, the training to testing set ratio was set to 8:2, with 2258 images used for training and 565 images used for testing. Since the Re-TMRS dataset lacks real boundary annotations, this embodiment uses dilation and erosion operations, and generates a dataset boundary mask through subtraction.

[0050] To comprehensively evaluate the performance of TransBGNet on the kidney tumor segmentation task, this embodiment uses five evaluation metrics for analysis: mean Dice coefficient (mDice), mean intersection-over-union ratio (mIoU), weighted F-measure, and weighted F-measure. The metrics used are Hausdorff distance (HD) and mean absolute error (MAE). Specifically, mDice, mIoU, and MAE are used to measure the overall segmentation accuracy of kidney tumors. Precision and recall are weighted differently to capture segmentation details of small tumors more sensitively; HD is used to quantify the maximum geometric deviation of the segmentation boundary, thus effectively evaluating targets with complex backgrounds and sensitive boundaries, such as kidney tumors. Through the synergistic analysis of these metrics, model performance can be evaluated from multiple dimensions, including region overlap, boundary stability, and pixel error. The calculation formulas for these evaluation metrics are as follows: ; ; Where TP represents true positive, FP represents false positive, FN represents false negative, and N represents the number of images.

[0051] ; ; ; Where Precision represents precision and Recall represents recall. This represents the weighting coefficient.

[0052] ; ; ; in, and Let A and B represent the one-way Hausdorff distances from point set A to B and from point set B to A, respectively.

[0053] ; Where N represents the number of images, P represents the predicted image, and G represents the ground truth mask.

[0054] All experiments in this embodiment were conducted using PyTorch 2.7.1 and Python 3.10.5, and trained on an NVIDIA RTX 3090 (24GB). Considering computational resource limitations, the input images were uniformly adjusted to 352×352 pixels, and a multi-scale training strategy {0.75, 1.0, 1.25} was employed. The Adam optimizer was used for optimization, with an initial learning rate of 8e-5, a batch size of 16, and 100 training epochs. To prevent overfitting during training, dropout was set to 0.3.

[0055] To evaluate the segmentation performance of TransBGNet, this embodiment compares it with several representative and state-of-the-art methods on the Re-TMRS dataset, including: U-Net, PraNet, HardNet-MSEG, Polyp-PVT, TransFuse, TransUNet, and Mamba-UNet. For comprehensive analysis of model performance, experiments were conducted using mDice, mIoU, and [other methods not specified in the original text]. The model performance was evaluated using five metrics, including HD, MAE, and MIDR, across multiple dimensions such as region overlap, boundary stability, and pixel error. Quantitative experimental results are shown in Table 1.

[0056] Table 1 Comparison of quantitative experimental results of different methods on the Re-TMRS kidney tumor dataset

[0057] Quantitative experimental results show that TransBGNet exhibits strong overall segmentation performance in the kidney tumor segmentation task. Specifically, the mDice and mIoU metrics are 91.6% and 87.3%, respectively, which are 1.3% and 1.8% higher than TransUNet, demonstrating high accuracy in overall kidney tumor segmentation. The model outperforms Mamba-UNet and TransUNet by 0.6 and 0.8% respectively, indicating that it can effectively identify smaller kidney tumors. While not optimal in terms of HD (high-resolution) metrics, it still maintains a level comparable to Mamba-UNet, demonstrating accurate boundary segmentation capabilities.

[0058] To more intuitively observe the model's segmentation performance, this method was also qualitatively compared with other representative and advanced methods. The visualized experimental results are shown below. Figure 4 As shown in the qualitative visualization experiment results, significant differences exist between different models in various complex scenarios, such as complex fat distribution, surgical instrument occlusion, and low contrast. Specifically, U-Net, PraNet, and TransFuse show a significant decrease in recognition accuracy when facing complex fat distributions, resulting in incomplete segmentation; while HardNet-MSEG and TransUNet exhibit reduced boundary segmentation capabilities and blurred segmentation contours when facing low contrast and surgical instrument occlusion. In contrast, TransBGNet, by introducing an auxiliary task of boundary prediction and collaborating with AFRF and CAMF modules, demonstrates stronger background recognition capabilities and boundary segmentation accuracy.

[0059] TransBGNet combines Transformer and CNN networks through its proposed dual-branch fusion module. To further verify the necessity of the Transformer and CNN branches, this section's experiments independently performed kidney tumor segmentation on the Re-TMRS dataset under three conditions: removing the CNN branch, removing the Transformer branch, and removing the fusion branch. When the CNN branch was removed, the Transformer branch used convolution and upsampling operations for decoding. When the fusion branch was removed, element-wise addition was used instead. The quantitative results of the ablation experiments for each branch are shown in Table 2.

[0060] Table 2 Comparison of ablation experimental results for each branch on the Re-TMRS dataset

[0061] Quantitative results show that removing the CNN branch, Transformer branch, and fusion branch all resulted in varying degrees of performance degradation. For example, the mDice coefficient decreased by 1.4%, 0.7%, and 0.5% respectively after removing the CNN, Transformer, and fusion branches. The Hausdorff distance increased, while the mIoU index also decreased to varying degrees. This indicates that each branch played its corresponding role in recognizing global contextual information and local features.

[0062] To more intuitively analyze the impact of each branch on the segmentation results, gradient-weighted class activation mapping (Grad-CAM) was also used for visualization analysis. The experimental results are as follows: Figure 5 As shown in the visualization activation graph, green, yellow, and red represent attention weights that increase sequentially. As shown in (b), the Transformer branch establishes a global context, thus significantly increasing the area of ​​focus and reducing weights at the kidney tumor boundary. As shown in (c), the CNN branch excels at capturing local features, recognizing boundary and texture features, with higher weights at the tumor boundary and texture regions, and lower weights within the tumor. As shown in (d), the fusion branch, by fusing features from the CNN and Transformer branches, effectively captures texture and boundary information. As shown in (e), through the boundary-guided weighting mechanism, the model can flexibly adjust the weights of key information from each branch, ultimately achieving optimal segmentation performance.

[0063] Overall, the fusion branch combines the global contextual information processing capabilities of Transformer and the local boundary feature processing capabilities of CNN, surpassing the segmentation capabilities of a single branch of CNN or Transformer. Furthermore, the multi-branch approach ultimately achieves optimal feature integration performance from multiple perspectives through a boundary-guided weighted mechanism, thereby improving the model's segmentation accuracy.

[0064] To further analyze the role of each module in TransBGNet in kidney tumor segmentation, this section's experiments will use the complete TransBGNet as a baseline and construct the following versions to analyze the contribution of each module to segmentation performance: w / o AFRF: Remove the adaptive feature refinement fusion module and replace it with an element-wise addition version to verify the importance of dual-branch fusion.

[0065] w / o CAMF: The multi-scale feature calibration aggregation module was removed and replaced with upsampling and element-wise addition to verify the importance of the multi-scale fusion module in extracting features at different scales.

[0066] w / o AFRF&CAMF: Simultaneously remove the adaptive feature refinement fusion module and the multi-scale feature calibration aggregation module.

[0067] The quantitative results of the ablation experiments for each module are shown in Table 3.

[0068] Table 3 Comparison of ablation experimental results for each module on the Re-TMRS dataset

[0069] Quantitative results show that removing or replacing the aforementioned modules significantly degrades the model's segmentation performance. Taking the mDice metric as an example, removing the AFRF or CAMF module resulted in a performance decrease of 1.2% and 0.9%, respectively, with the most significant decrease of 1.7% observed when both modules were removed simultaneously, along with a significant increase in the Hausdorff distance. Experimental results demonstrate that the CAMF module effectively addresses the spatial misalignment problem of features at different scales, and that the AFRF module effectively integrates global contextual information and boundary texture information.

[0070] To further and more intuitively analyze the impact of each module on the segmentation results, feature heatmap results of the complete TransBGNet and its abridged versions on the kidney tumor dataset are shown below. Figure 6 As shown, the model with some modules removed exhibits a significant decrease in segmentation ability when dealing with complex scenes such as surgical instrument occlusion, complex fat distribution, and significant bleeding. The decrease in segmentation accuracy is particularly pronounced when surgical instrument boundaries are present, resulting in segmentation results that appear to be fused together.

[0071] Overall, AFRF and CAMF improve the fusion of global context and local features, as well as the misalignment of spatial information at different scales, enabling the model to achieve accurate segmentation even when dealing with various complex scenes. Therefore, the collaborative optimization of the AFRF and CAMF modules significantly enhances the model's overall segmentation performance.

[0072] In summary, this embodiment constructs an asymmetric dual-branch encoder architecture using PVTv2 and HardNet-MSEG to extract global context and local detail features respectively; it achieves adaptive dynamic fusion of heterogeneous features through the AFRF module, fully utilizing the complementary advantages of the two branches; it introduces particle-sphere computation through the CAMF module for feature calibration, solving the cross-scale semantic misalignment problem; and it enhances the spatial constraint capability of the tumor boundary region through a boundary-guided weighting mechanism and a boundary-guided hybrid loss function. It performs excellently in the kidney tumor segmentation task, effectively handling challenging scenarios such as complex backgrounds and blurred boundaries.

[0073] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0074] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for segmenting renal tumors based on boundary guidance and hybrid networks, characterized in that, include: Step 1: Obtain endoscopic images of the kidney tumor; Step 2: PVTv2 is used as the backbone network of the Transformer branch to extract multi-scale global contextual features in the endoscopic images, while HardNet-MSEG network is used as the backbone network of the CNN branch to capture local texture and edge detail features. Step 3: The features of the Transformer branch and the CNN branch are refined and adaptively fused using the Adaptive Feature Refinement and Fusion (AFRF) module to obtain fused features at different scales; Step 4: Based on the multi-scale feature calibration and aggregation module CAMF, perform top-down semantic calibration and alignment aggregation on the fused features at different scales; Step 5: Adaptively and dynamically allocate the prediction graph weights of the fusion branch and the CNN branch through the boundary-guided weighting mechanism, and construct the boundary-guided hybrid loss function optimization model; Step 6: Perform weighted fusion of the prediction results from the fusion branch and the CNN branch to obtain the final kidney tumor segmentation prediction map.

2. The method for segmenting renal tumors based on boundary guidance and hybrid networks according to claim 1, characterized in that, The PVTv2 backbone network in step 2 includes a four-stage encoder for extracting pyramid features at different scales layer by layer; the HardNet-MSEG network preserves and restores the high-frequency edges and texture details of the endoscopic image through step-by-step downsampling and feature upsampling operations.

3. The method for segmenting renal tumors based on boundary guidance and hybrid networks according to claim 1, characterized in that, The adaptive feature refinement and fusion module AFRF in step 3 adopts a parallel processing strategy of independent channel refinement and dynamic gating weighting, including: The features input from the CNN branch and the Transformer branch are convolved and cross-fused, and then refined by the efficient channel attention module ECA to obtain the refined features. An adaptive gating branch is introduced, which concatenates the features of the original CNN branch and the Transformer branch through channels, extracts joint features through convolution, and forms dynamic gating weights through feature activation. The probability maps of pixels of the same size after Sigmoid activation are multiplied by their respective refined features, and the fused features of different scales are obtained by adaptive weighted summation.

4. The method for segmenting renal tumors based on boundary guidance and hybrid networks according to claim 1, characterized in that, Step 4 specifically includes: The fused features at different scales are flattened by the GBR (Granular Sphere Refinement) module, the attention weights from each spatial location to the center of the anisotropic ellipsoid are calculated, and the features are aggregated according to the weights to complete the reconstruction of the semantic core. Then, the features are flattened back to restore the refined feature map. Convolution and upsampling are performed on the fusion features at different scales to align and fuse high-resolution features with low-resolution features step by step. The highest-level features and multi-level fused features are concatenated through channels, and the aggregated features are output after convolution and activation.

5. The method for segmenting renal tumors based on boundary guidance and hybrid networks according to claim 1, characterized in that, The boundary-guided weighting mechanism in step 5 dynamically adjusts the fusion weights of the fusion branch and the CNN branch by introducing a boundary prediction auxiliary task. The boundary-guided weighting formula is as follows: ； in, This represents the fusion weight of the nth branch. This indicates the weights of the introduced boundary prediction auxiliary task; Based on the above formulas, the calculation formulas for the fusion branch weights and CNN branch weights are as follows: ； ； ； ； in, Indicates the weight of the merged branch. Indicates the weights of CNN branches. This represents the loss value of the main splitting task in the nth branch. , and These represent the loss values ​​for the fusion branch, the CNN branch, and the boundary prediction auxiliary task, respectively. This indicates the weights of the fusion branch boundary prediction auxiliary task. This represents the weights of the CNN branch boundary prediction auxiliary task, where m represents the number of training epochs. Predict the weighting coefficients for the boundary.

6. The method for segmenting renal tumors based on boundary guidance and hybrid networks according to claim 5, characterized in that, The boundary mask for the boundary prediction auxiliary task is generated by subtracting dilation and erosion operations, and boundary prediction is performed in the fusion branch.

7. The method for segmenting renal tumors based on boundary guidance and hybrid networks according to claim 1, characterized in that, In step 5, an optimization model with an inner boundary-guided hybrid loss function is constructed using the output results of the main segmentation task and the boundary prediction auxiliary segmentation task for renal tumors as the optimization objective. This model includes a main segmentation loss composed of a weighted intersection-over-union loss function and a weighted binary cross-entropy loss function, and a boundary prediction auxiliary loss composed of a Dice loss function, as shown in the following formula: ； in, The boundary-guided hybrid loss function BGH,L p L represents the loss function for the main segmentation task. b This represents the loss function for the boundary prediction auxiliary task. and These represent the real masks for the main segmentation task and the real masks for the auxiliary task, respectively. and These represent the prediction graphs for the primary task and the auxiliary task, respectively. This represents the weighting coefficient of the auxiliary task.

Images