A kidney tumor image segmentation method and system based on boundary extraction
By enhancing edge detail information through the PVTv2 backbone network and the inverse attention mechanism, and combining it with an efficient hybrid decoding module, the problem of boundary recognition in kidney tumor image segmentation is solved, and high-precision tumor segmentation results are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-07
- Publication Date
- 2026-05-15
AI Technical Summary
Existing kidney tumor segmentation algorithms struggle to accurately identify tumor boundaries in endoscopic images, and traditional Transformer models are insufficient in extracting local detail features, leading to inaccurate segmentation.
Image features are extracted using a PVTv2 backbone network, and edge detail information is enhanced by combining a reverse attention mechanism and a feature self-attention module. Decoding is performed through an efficient hybrid decoding module to ensure local continuity, and local detail features are extracted using depthwise separable convolution.
It achieves accurate segmentation of kidney tumor images, improves the clarity and segmentation accuracy of boundary recognition, enhances global information extraction capability and robustness, and outputs clear predictive masks.
Smart Images
Figure CN118115738B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of kidney tumor image segmentation technology, and more specifically to a kidney tumor image segmentation method and system based on boundary extraction. Background Technology
[0002] Kidney tumors are currently one of the ten most common cancers, with over 400,000 new cases and 150,000 deaths annually. Kidney cancer is a common acute urinary tract disease, with a lifetime probability of 1.34%. As a precursor to kidney cancer, early detection and treatment of kidney tumors are crucial. Radical nephrectomy (RN) and partial nephrectomy (PN) are the main treatments for kidney tumors. Medical image segmentation can help doctors locate tumors during surgery, achieving accurate resection. Segmenting endoscopic images helps doctors better identify lesions and remove them; therefore, automatic and accurate segmentation of kidney tumors in endoscopic images is of great clinical significance.
[0003] However, due to the vertical movement of the laparoscopy during surgery and the varying physical conditions of each patient, the size and location of kidney tumors are not constant, which poses a challenge to tumor identification. Secondly, existing segmentation algorithms are not accurate and have problems with boundary recognition, resulting in blurred boundaries. Furthermore, most existing kidney tumor segmentation algorithms are based on CT image segmentation, focusing on preoperative diagnosis of kidney tumors and localization of tumor location. Automatic segmentation of endoscopic images of kidney tumors would greatly assist surgeons in their operations.
[0004] Common methods for segmenting endoscopic images fall into two main categories: CNNs and Transformers. CNNs, due to their simple structure and translation invariance, perform exceptionally well in computer vision. With the advent of U-Net, the U-shaped encoder-decoder structure has become increasingly widely used in image segmentation tasks. However, the encoder-decoder structure is too simple, resulting in poor modeling ability for long-range features and a tendency to overfit when segmenting tumor images with varying shapes. Furthermore, when CNNs introduce low-resolution semantic features in the decoder stage, it can easily lead to semantic discrepancies between the encoder and decoder. In addition, CNNs are relatively poor at summarizing global information.
[0005] Since its inception, the Visual Transformer (ViT) has been widely used in computer vision tasks. The Transformer can capture long-range dependencies, enabling the network to extract information from a global perspective. It exhibits powerful feature representation capabilities, extracting global features by perceiving the global context. However, in segmentation tasks, in addition to considering the global context, the local continuity of features must also be considered; the network's ability to recover details is particularly important during the decoding stage. Traditional Transformers cannot extract and model features from the neighborhood, and cannot accurately represent local details.
[0006] Therefore, how to provide a method that can effectively extract local detail information and solve the difficulty of feature extraction caused by the appearance of kidney tumors is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0007] In view of this, the present invention provides a kidney tumor image segmentation method and system based on boundary extraction. It adopts a feature self-attention module to increase edge detail information and transmit encoder-decoder information; it adopts an efficient hybrid decoding module to ensure local continuity while decoding, obtain more high-level semantic features, and output accurate prediction map.
[0008] To achieve the above objectives, the present invention adopts the following technical solution:
[0009] A kidney tumor image segmentation method based on boundary extraction, comprising:
[0010] Tumor images are acquired, and feature information of the tumor images is extracted using a PVTv2 backbone extraction network. The PVTv2 backbone extraction network contains patch embedding layers of different scales, and tumor image feature information of different scales is obtained through the patch embedding layers of different scales.
[0011] Edge detail information of tumor image features at different scales is enhanced by reverse attention mechanism, and the enhanced tumor image features are then fused through parallel convolution to obtain the first output.
[0012] The first output corresponding to the tumor image feature information at each different scale is fused step by step to obtain fused feature information. After each fusion, local detail features are extracted by introducing Transformer and depthwise separable convolution during hybrid decoding to further obtain pyramid feature output. The final prediction mask is obtained by fusion step by step.
[0013] Preferably, the enhancement of edge detail information of tumor image feature information at different scales through the reverse attention mechanism specifically includes:
[0014] The tumor image feature information input is processed by 1x1 convolution to change the dimension, and then the attention range is increased by the Softmax function.
[0015] A reverse attention mechanism is introduced to segment the boundaries of tumor image feature information, increasing the weight of non-tumor regions and decreasing the weight of lesion regions. The formula is as follows:
[0016] R i =1-Sigmoid(S i );
[0017] F i =E i +E i *RA(SoftMax(Conv3(E i )));
[0018] Among them, R i For the output of the reverse attention mechanism, S i The result is the tumor image feature information after the attention range is increased by the Softmax function. RA is the reverse attention operation, Conv3() is a 3×3 convolution, and E... i For tumor image feature information, F i These are intermediate output features of the feature self-attention module, and can also be regarded as intermediate inputs.
[0019] Preferably, obtaining the first output specifically includes: converting F... i The input is fed into two parallel convolutional paths: a 1x1 convolution and a 1x1 transposed convolution. After matrix multiplication, the attention map is expanded using a softmax operation, then fused using matrix multiplication, and finally fused with the input via a 1x1 convolution to obtain the feature self-attention output O. FSA The specific formula is as follows:
[0020]
[0021]
[0022]
[0023] Conv1() represents a 1×1 convolution. This represents a 1×1 transposed convolution. The output of a 1x1 convolution. This is the output of a 1x1 transposed convolution.
[0024] Preferably, the high-efficiency decoding module specifically includes: a fully connected layer, a Transformer module, a high-efficiency Transformer module, and a spatial attention module. The fully connected layer outputs the first query Q1 and the first key K1 of the fused feature information. The Transformer module calculates and obtains the first value V1. The first query Q1 and the first key K1 are input into the high-efficiency Transformer module to obtain the second query Q2, the second key K2, and the second value V2. The spatial attention module obtains the pyramid feature output based on Q1, K1, V1, Q2, K2, and V2.
[0025] Preferably, the high-efficiency Transformer module includes a high-efficiency self-attention module and a high-efficiency forward feedback module. The high-efficiency Transformer module replaces the MSA in the Transformer module with the high-efficiency self-attention module and replaces the FFN in the Transformer module with the high-efficiency forward feedback module.
[0026] Preferably, the high-efficiency self-attention module specifically includes:
[0027]
[0028] Where D represents the dimension of the feature, O ESA For efficient self-attention module output, Q represents the query, K represents the key, and V represents the value.
[0029] Preferably, the high-efficiency forward feedback module specifically includes:
[0030] O EFFN =O ESA +fc2(GELU(Norm(DW(fc1(O ESA )))));
[0031] Among them, fc1() and fc2() are two linear fully connected layers.
[0032] Preferably, the spatial attention module specifically includes: multiplying Q1, K1, Q2, and K2 together and performing a Concat concatenation operation; performing matrix multiplication with V1 and V2 after a 1×1 convolution; performing another 1×1 convolution; and finally fusing with V2 to obtain the pyramid feature output O. SA , represented as:
[0033] S i =Concat((Q1*Q2),(Q1*K1),(K1*K2));
[0034] O SA =V2*Conv1(Concat((V1*Conv1(Ai )),(V2*Conv1(A i )));
[0035] Among them, A i This is the intermediate output of the spatial attention module.
[0036] A kidney tumor image segmentation system based on boundary extraction, comprising:
[0037] The feature information acquisition module acquires tumor images and uses a PVTv2 backbone extraction network to extract feature information from the tumor images; wherein, the PVTv2 backbone extraction network contains patch embedding layers of different scales, and tumor image feature information of different scales is acquired through the patch embedding layers of different scales;
[0038] The output acquisition module enhances the edge detail information of the tumor image feature information at different scales through a reverse attention mechanism, and obtains the first output by parallel convolution fusion operation on the enhanced tumor image feature information.
[0039] The prediction map acquisition module fuses the first output corresponding to the tumor image feature information at each different scale level to obtain fused feature information. After each level of fusion, local detail features are extracted by introducing Transformer and depthwise separable convolution during hybrid decoding to further obtain pyramid feature output. The final prediction mask is obtained by fusing the features level by level.
[0040] As can be seen from the above technical solution, compared with the prior art, this invention discloses a method and system for renal tumor image segmentation based on boundary extraction. It extracts more robust backbone features from endoscopic images using a PVTv2 backbone extraction network, providing more foreground information for the subsequent decoding stage, exhibiting stronger global information extraction capabilities and better robustness against input interference. The Feature Self-Attention (FSA) module, acting as a skip connection between the encoder and decoder, not only passes the encoder output to the efficient hybrid decoding module but also processes the decoder output, refining boundary information and promoting the output of a correct mask. The EHT module improves upon the PVTv2 architecture, ensuring local feature continuity through a spatial attention submodule while focusing on global information, thereby outputting a segmentation mask with clear and accurate boundary contours. In summary, this invention can effectively extract local detail information and solve the problem of feature extraction difficulties caused by the appearance of renal tumors. Attached Figure Description
[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0042] Figure 1 The attached figure is a flowchart of the method steps provided by the present invention.
[0043] Figure 2 The attached figure is a schematic diagram of the network structure provided by the present invention.
[0044] Figure 3 The attached figure is a schematic diagram of the feature self-attention module structure provided by the present invention.
[0045] Figure 4 The attached figure is a schematic diagram of the high-efficiency hybrid decoding module provided by the present invention.
[0046] Figure 5 The attached figure is a schematic diagram of the high-efficiency Transformer module structure provided by the present invention.
[0047] Figure 6 The attached figure is a schematic diagram of the spatial attention module structure provided by the present invention.
[0048] Figure 7 The attached figure is a schematic diagram showing the results of a qualitative comparison between the present invention and current advanced models. Detailed Implementation
[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0050] This invention discloses a kidney tumor image segmentation method based on boundary extraction, such as... Figure 1 As shown, it includes:
[0051] Tumor images are acquired, and feature information of the tumor images is extracted using the PVTv2 backbone extraction network. The PVTv2 backbone extraction network contains patch embedding layers of different scales, and tumor image feature information of different scales is obtained through patch embedding layers of different scales.
[0052] Edge detail information of tumor image features at different scales is enhanced by reverse attention mechanism, and the enhanced tumor image features are then fused through parallel convolution to obtain the first output.
[0053] The first output corresponding to the tumor image feature information at each different scale is fused step by step to obtain fused feature information. After each fusion, local detail features are extracted by introducing Transformer and depthwise separable convolution during hybrid decoding to further obtain pyramid feature output. The final prediction mask is obtained by fusion step by step.
[0054] In one specific embodiment, such as Figure 2 As shown, the input image enters the encoder path, and the input I∈R 3×H×W The outputs are obtained at four different levels, namely Compared to traditional CNN methods that focus more on local information, PVTv2 exhibits stronger global information extraction capabilities and better robustness to input disturbances. The output feature E1 contains rich details such as texture, boundary and color details, while E2, E3 and E4 have rich semantic information.
[0055] In one specific embodiment, because the Transformer's ability to extract local features is not high, the low-level features output from the encoder need to be further extracted by the FSA module to refine the edge information of the features before being passed to the decoder. Furthermore, as a skip connection, the FSA module must ensure that the encoder's output features and input features are successfully fused, thereby outputting the correct prediction mask at the decoder. FSA utilizes an attention mechanism to maintain high resolution during computation, and uses parallel paths to process input features, employing convolution to change the dimensionality of the input features, and softmax to increase attention to the features. Its structure is as follows... Figure 3 As shown.
[0056] The input is processed using a 1×1 convolution to change its dimensionality, and then the attention range is increased using the Softmax function. An inverse attention mechanism is introduced in the FSA module to segment the target boundary. In the image, the non-tumor region is larger, and directly locating the tumor to find its boundary would lose much edge detail. Therefore, by increasing the weight of non-tumor regions and decreasing the weight of lesion regions, a feature map rich in boundary detail is obtained. The inverse attention mechanism enhances the edge detail information of tumor image features at different scales, specifically including:
[0057] The tumor image feature information input is processed by 1x1 convolution to change the dimension, and then the attention range is increased by the Softmax function;
[0058] A reverse attention mechanism is introduced to segment the boundaries of tumor image feature information, increasing the weight of non-tumor regions and decreasing the weight of lesion regions. The formula is as follows:
[0059] R i =1-Sigmoid(S i );
[0060] The complementary regions obtained through the above operations are fused with the input features to gradually delineate the edge details of the tumor image and extract the most realistic boundary information. The formula expression for the above operations is shown below:
[0061] F i =E i +E i *RA(SoftMax(Conv3(E i )));
[0062] Among them, R i For the output of the reverse attention mechanism, S i The result is the tumor image feature information after the attention range is increased by the Softmax function. RA is the reverse attention operation, Conv3() is a 3×3 convolution, and E... i Input feature information for tumor images, F i The intermediate output feature of the feature self-attention module can also be seen as the intermediate input modification completed.
[0063] In one specific embodiment, obtaining the first output specifically includes: setting F i The input is fed into two parallel convolutional paths: a 1x1 convolution and a 1x1 transposed convolution. After matrix multiplication, the attention map is expanded using a softmax operation, then fused using matrix multiplication, and finally fused with the input via a 1x1 convolution to obtain the feature self-attention output O. FSA The specific formula is as follows:
[0064]
[0065]
[0066]
[0067] Conv1() represents a 1×1 convolution. This represents a 1×1 transposed convolution. The output of a 1x1 convolution. This is the output of a 1x1 transposed convolution. In the feature self-attention module, F is obtained through the RA operation. iIt has rich local edge details, which can accurately depict the boundary contour of the lesion area. The final result obtained through a series of convolution fusion operations can accurately use the surrounding local receptive domain to accurately simulate adjacent features, improve the ability to extract local details, and restore the most realistic detail image.
[0068] In one specific embodiment, by Figures 4-6 As can be seen, the efficient decoding module contains two Transformer blocks. One is inherent to PVTv2 and consists of LayerNorm, Self-Attention, and FeedForward modules. The other is an efficient transformer block, which consists of LayerNorm, Efficient Self-Attention, and Efficient FeedForward modules. There is also a Spatial Attention Block, which performs convolution operations on the query Q, key K, and value V obtained from the fully connected layer to model the spatial dependencies of the features. Specifically, the efficient decoding module includes: a fully connected layer, a Transformer module, an efficient Transformer module, and a spatial attention module. The fully connected layer outputs a first query Q1 and a first key K1 with fused feature information. The Transformer module calculates the first value V1. The first query Q1 and the first key K1 are input into the efficient Transformer module to obtain a second query Q2, a second key K2, and a second value V2. The spatial attention module obtains the pyramid feature output based on Q1, K1, V1, Q2, K2, and V2.
[0069] In one specific embodiment, the high-efficiency Transformer module includes a high-efficiency self-attention module and a high-efficiency feedback-forward module. The high-efficiency Transformer module replaces the MSA in the Transformer module with the high-efficiency self-attention module and replaces the FFN in the Transformer module with the high-efficiency feedback-forward module.
[0070] The efficient self-attention module can obtain high-resolution feature maps for the input feature map. The affinity matrix is calculated using Q and K, and then multiplied by V to obtain the output O. ESA , making It can be represented as:
[0071]
[0072] Where D represents the dimension of the feature, O ESA For efficient self-attention module output, Q represents the query, K represents the key, and V represents the value.
[0073] The computational complexity of the above formula is O(N). 2 This allows for effective self-attention by reducing spatial resolution. That is:
[0074]
[0075]
[0076] W is a linear projection operation. It reshapes K and V as... Then, W is used to restore the channel depth, transforming it into C. This reduces the computational complexity to O(W). This is the process of achieving high-resolution applications.
[0077] In one specific embodiment, the efficient forward feedback module specifically includes:
[0078] O EFFN =O ESA +fc2(GELU(Norm(DW(fc1(O ESA )))));
[0079] Here, fc1() and fc2() are two linear fully connected layers with a scaling factor of r, which is 4 according to PVT.
[0080] In one specific embodiment, the spatial attention module specifically includes: multiplying Q1, K1, Q2, and K2 together and performing a Concat concatenation operation; after a 1×1 convolution, performing matrix multiplication with V1 and V2; performing another 1×1 convolution; and finally fusing with V2 to obtain the pyramid feature output O. SA During this process, the reshaping operation is used to dimensionally calibrate all Q, K, and V values, that is... Become This ensures that the final output is correctly passed to the next EHT module, as shown below:
[0081] S i =Concat((Q1*Q2),(Q1*K1),(K1*K2));
[0082] O SA =V2*Conv1(Concat((V1*Conv1(A i )),(V2*Conv1(A i )));
[0083] Among them, A i This is the intermediate output of the spatial attention module.
[0084] A kidney tumor image segmentation system based on boundary extraction, comprising:
[0085] The feature information acquisition module acquires tumor images and uses the PVTv2 backbone extraction network to extract the feature information of the tumor images. The PVTv2 backbone extraction network contains patch embedding layers of different scales, and tumor image feature information of different scales is obtained through patch embedding layers of different scales.
[0086] The output acquisition module enhances the edge detail information of tumor image feature information at different scales through the reverse attention mechanism, and obtains the first output by parallel convolution fusion operation on the enhanced tumor image feature information.
[0087] The prediction map acquisition module fuses the first output corresponding to the tumor image feature information at each different scale level to obtain fused feature information. After each level of fusion, local detail features are extracted by introducing Transformer and depthwise separable convolution during hybrid decoding to further obtain pyramid feature output. The final prediction mask is obtained by fusing the features level by level.
[0088] The EHT module addresses the problem of traditional Transformers' poor ability to extract local details by combining two Transformers and introducing them into a spatial attention module. The entire decoder can establish long-range dependencies and extract local detail features, ensuring local continuity. Furthermore, the EHT module has overlapping patches when embedding patches, which can increase the number of channels in the feature map and reduce its size, ultimately outputting an accurate prediction mask.
[0089] This invention uses weighted binary cross-entropy (mBCE) loss and weighted cross-linking loss (mIoU) to supervise the quality of the output predicted map. In the prediction, each pixel is assigned a corresponding weight to highlight its importance. The loss function is as follows:
[0090]
[0091] L i =L IoU (P i ,G)+L BCE (P i ,G);
[0092] In the formula P i Let G be the prediction result of the i-th decoder, and let G be the ground truth of the input training image.
[0093] This invention is based on PyTorch and uses an NVIDIA RTX3090 for GPU acceleration. Images are standardized to 352×352 pixels. A multi-scale training strategy of {0.75, 1.0, 1.25} is used instead of data augmentation. The AdamW optimizer is employed for optimization, with an initial learning rate of 0.0001, a maximum epoch of 100, and a batch size of 16.
[0094] The network of this invention performs segmentation on the Re-TMRS dataset and compares the segmentation results with those of other state-of-the-art algorithms on the Re-TMRS dataset. Other state-of-the-art methods include: U-Net, U-Net++, CaraNet, DCRNet, HarDMSEG, MSRAformer, PolypPVT, and PraNet.
[0095] The evaluation results using the evaluation metrics are shown in Table 1. It can be seen that the model of this invention improved the results on each evaluation metric, with mDice and mIoU reaching 91.2% and 91.8%, respectively. Specifically, the mDice of this invention is +1.5% higher than PraNet and 1.9% higher than MSRAformer, while mIoU exceeds PraNet by 0.9% and is 1.6% higher than MSRAformer. Compared to pure CNN algorithms, this invention has better segmentation performance, and compared to Polyp-PVT, which combines CNN and Transformer, the method of this invention also improves on all evaluation metrics, with mDice being 5.8% higher and mIoU even 8.6% higher than Polyp-PVT.
[0096] The MAE of this invention reaches the lowest value among all comparison algorithms, fully demonstrating that the result of this invention has small error and accurate segmentation. MAP also achieved the best results among all algorithms, proving the effectiveness of the model in segmentation.
[0097] Table 1 Segmentation Comparison Results
[0098]
[0099] To more intuitively observe the segmentation performance of the proposed algorithm, Figure 7The visualization results of the proposed model on Re-TMRS are shown, and a comparison with six advanced algorithms is also presented. As can be seen from the figures, the mask obtained by this invention is closer to the ground truth, while the masks obtained by other models exhibit inaccurate segmentation. The U-Net model has poor learning performance, resulting in a completely black mask. U-Net++, DCRNet, and MSRAformer perform poorly in identifying small lesion regions, even exhibiting severe oversegmentation, mistaking background regions without lesions for target regions. Besides these issues, the images segmented by the compared algorithms generally have blurred contours, poor performance in extracting boundary detail features, and cannot identify all pixels belonging to lesion regions. Even the segmentation results of CaraNet and Poly-PVT do not encompass all target regions, showing a significant difference from the ground truth image. In summary, this method can... Figure 7 It is intuitive to see that the segmentation performance of BEFormer of this invention is superior to other comparative algorithms.
[0100] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.
[0101] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A kidney tumor image segmentation method based on boundary extraction, characterized in that, include: Tumor images are acquired, and feature information of the tumor images is extracted using a PVTv2 backbone extraction network. The PVTv2 backbone extraction network contains patch embedding layers of different scales, and tumor image feature information of different scales is obtained through the patch embedding layers of different scales. Edge detail information of tumor image feature information at different scales is enhanced by reverse attention mechanism, and the enhanced tumor image feature information is then used to obtain the first output through parallel convolution fusion operation; The enhancement of edge detail information of tumor image feature information at different scales through the reverse attention mechanism specifically includes: The tumor image feature information input is processed by 1x1 convolution to change the dimension, and then the attention range is increased by the Softmax function; A reverse attention mechanism is introduced to segment the boundaries of tumor image feature information, increasing the weight of non-tumor regions and decreasing the weight of lesion regions. The formula is as follows: ； ； in, For the output of the reverse attention mechanism, The result is the tumor image feature information after applying the Softmax function to increase the attention range. For reverse attention operation, ()for convolution, For tumor image feature information, These are the intermediate output features of the feature self-attention module; The specific steps of obtaining the first output include: The input is fed into two parallel convolutional paths: a 1x1 convolution and a 1x1 transposed convolution. After matrix multiplication, the attention map is expanded using a softmax operation, then fused using matrix multiplication, and finally fused with the input using a 1x1 convolution to obtain the feature self-attention output. The specific formula is as follows: in, ()express convolution, ()express Transposed convolution, The output of a 1x1 convolution. This is the output of a 1x1 transposed convolution; The first output corresponding to the tumor image feature information at each different scale is fused step by step to obtain fused feature information. After each fusion, local detail features are extracted by introducing Transformer and depthwise separable convolution during hybrid decoding to further obtain pyramid feature output. The final prediction mask is obtained by fusion step by step. The hybrid decoding includes a fully connected layer, a Transformer module, an efficient Transformer module, and a spatial attention module.
2. The kidney tumor image segmentation method based on boundary extraction according to claim 1, characterized in that, The hybrid decoding specifically includes: a fully connected layer, a Transformer module, an efficient Transformer module, and a spatial attention module. The fully connected layer outputs a query of the fused feature information. AND key The first query is obtained through the Transformer module. First key With the first value , the query AND key Input the efficient Transformer module to obtain the second query Second key With the second value The spatial attention module is based on , , , , , Obtain the pyramid feature output.
3. The kidney tumor image segmentation method based on boundary extraction according to claim 2, characterized in that, The high-efficiency Transformer module includes a high-efficiency self-attention module and a high-efficiency forward feedback module. The high-efficiency Transformer module replaces the MSA in the Transformer module with the high-efficiency self-attention module and replaces the FFN in the Transformer module with the high-efficiency forward feedback module.
4. The kidney tumor image segmentation method based on boundary extraction according to claim 3, characterized in that, The highly efficient self-attention module specifically includes: ； Where D represents the dimension of the feature. For efficient self-attention module output, Q represents the query, K represents the key, and V represents the value.
5. The kidney tumor image segmentation method based on boundary extraction according to claim 4, characterized in that, The high-efficiency forward feedback module specifically includes: ； in, ()and () are two linear fully connected layers.
6. The kidney tumor image segmentation method based on boundary extraction according to claim 2, characterized in that, The spatial attention module specifically includes: Let , , , Combinatorial multiplication and passing through Connection operation, in After convolution and , Perform matrix multiplication, and then... Convolution, and finally with The pyramid feature output is obtained by fusion. , represented as: ； ； in, This is the intermediate output of the spatial attention module.
7. A kidney tumor image segmentation system based on boundary extraction, characterized in that, include: The feature information acquisition module acquires tumor images and uses a PVTv2 backbone extraction network to extract feature information from the tumor images; wherein, the PVTv2 backbone extraction network contains patch embedding layers of different scales, and tumor image feature information of different scales is acquired through the patch embedding layers of different scales; The output acquisition module enhances the edge detail information of the tumor image feature information at different scales through a reverse attention mechanism, and obtains the first output by parallel convolution fusion operation on the enhanced tumor image feature information. The prediction map acquisition module fuses the first output corresponding to the tumor image feature information at each different scale level to obtain fused feature information. After each level of fusion, local detail features are extracted by introducing Transformer and depthwise separable convolution during hybrid decoding to further obtain pyramid feature output. The final prediction mask is obtained by fusing the features level by level.