Ureter segmentation method and system based on boundary supervision mechanism and multiple scales

Abstract

The invention discloses a ureter segmentation method and system based on a boundary supervision mechanism and multiple scales. Relates to the technical field of medical image processing. Relates to the technical field of medical image processing. The method comprises the steps that ureter image data to be segmented are acquired and preprocessed; constructing a double-branch encoder-decoder network architecture based on a pyramid visual converter; an edge attention preserving module and an expansion separable convolution module are introduced into a double-branch encoder-decoder network architecture and are used for extracting feature information of ureter image data to be preprocessed; establishing an association relationship between different levels of feature information by using a multi-level pairwise regression module, and realizing feature fusion through pairwise feature matching and self-adaptive weight generation; and training the double-branch encoder-decoder network architecture based on the weighted loss function, and outputting a ureter segmentation result and an edge prediction result. According to the method, the accuracy and robustness of ureter segmentation can be remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and more specifically to a method and system for ureter segmentation based on boundary supervision mechanism and multi-scale. Background Technology

[0002] The ureter is a vital component of the urinary system, responsible for transporting urine produced by the kidneys to the bladder. It is one of the body's essential tissues. Due to its proximity to the kidneys and bladder, the ureter is easily damaged during surgeries for kidney tumors, bladder tumors, and similar procedures. Therefore, deep learning-based technologies are needed to automate the segmentation of the ureteral structure, assisting surgeons and preventing accidental damage during these procedures. Automated segmentation of the ureteral structure aids in surgical planning. In endoscopic surgery, precise identification of the ureter's location helps surgeons plan better, reducing surgical risks and increasing success rates. Real-time feedback is also possible: During endoscopic surgery, AI algorithms can provide real-time image analysis, helping surgeons quickly identify the ureter and its surrounding structures, and respond promptly to any unexpected situations that may arise during the procedure.

[0003] In recent years, deep learning-based medical image segmentation technology has made significant progress. However, existing medical image segmentation methods, especially those related to ureter segmentation, mainly suffer from the following technical problems: Insufficient edge segmentation accuracy: Deep neural networks are prone to edge information degradation during feature extraction, resulting in inaccurate ureteral boundary segmentation; Insufficient multi-scale feature fusion: Existing methods struggle to effectively integrate local detail features and global semantic information, affecting segmentation performance; Difficulty in handling regions with blurred boundaries: Traditional methods have poor segmentation results for areas with unclear boundaries or low contrast with surrounding tissues; Therefore, there is an urgent need for a technical solution that can improve the accuracy of medical image segmentation, especially the accuracy of edge segmentation. Summary of the Invention

[0004] In view of this, the present invention provides a ureter segmentation method and system based on boundary supervision mechanism and multi-scale, which solves the problems of insufficient edge segmentation accuracy and insufficient multi-scale feature fusion in the prior art by means of innovative edge attention preservation technology, multi-scale context awareness mechanism and multi-level pairwise regression module.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a ureteral segmentation method based on boundary supervision mechanism and multi-scale, comprising: Step 1: Acquire the image data of the ureter to be segmented and perform preprocessing; Step 2: Construct a dual-branch encoder-decoder network architecture based on the pyramid visual transformer; Step 3: Introduce an edge attention preservation module and a dilated separable convolution module into the dual-branch encoder-decoder network architecture to extract feature information from the ureteral image data to be preprocessed; Step 4: Establish the correlation between feature information at different levels using the multi-level pairwise regression module, and achieve feature fusion through pairwise feature matching and adaptive weight generation; Step 5: Train the dual-branch encoder-decoder network architecture based on the weighted loss function, and output the ureter segmentation result and edge prediction result.

[0006] Preferably, the dual-branch encoder-decoder network architecture in step 2 uses the Pyramid Visual Transformer PVT-v2 as the backbone network, including an encoder, a decoder, and a skip connection layer; the encoder is used to extract multi-scale feature representations; the decoder is used to progressively restore spatial resolution; and the skip connection layer is used to preserve detailed information.

[0007] Preferably, the edge attention preservation module adopts a dual-branch architecture, including: a main feature processing branch unit, an edge feature branch unit, and a feature fusion unit; The main feature processing branch unit uses lightweight deep convolution to extract semantic information; The edge feature branch unit uses the Sobel operator to calculate the gradient and generate an edge attention map; The feature fusion unit achieves adaptive enhancement of edge information through element-wise multiplication.

[0008] Preferably, the dilated separable convolution module captures feature information from different receptive fields through a parallel multi-branch structure, including a parallel multi-branch processing unit, a dilation rate configuration unit, and a feature splicing and channel fusion unit. The parallel multi-branch processing unit includes multiple branches, batch normalization, and ReLU activation; The expansion rate configuration unit is used to set different expansion rates for each branch; The feature splicing and channel fusion unit is used to integrate the output features while keeping the size of the output feature map unchanged.

[0009] Preferably, the multi-level pairwise regression module includes: a pairwise feature matching unit, an adaptive weight generation unit, and a weighted feature fusion unit; The pairwise feature matching unit calculates the similarity matrix between features at different levels to obtain the matching results; The adaptive weight generation unit generates feature fusion weights based on the matching results; The weighted feature fusion unit optimizes and integrates multi-level features based on feature fusion weights.

[0010] Preferably, the weighted loss function includes: main segmentation loss, edge supervision loss, and auxiliary loss; The main segmentation loss uses a combination of weighted binary cross-entropy loss and weighted IoU loss; Edge-supervised loss is used for supervised training of edge prediction results; The auxiliary loss employs multi-scale supervised loss to balance different training objectives.

[0011] Preferably, a ureteral segmentation system based on boundary supervision mechanism and multi-scale includes: Image acquisition module: used to acquire and preprocess the ureter image data to be segmented; Network building block: Used to build a dual-branch encoder-decoder network architecture based on the pyramid visual transformer; Feature extraction module: used to introduce an edge attention preservation module and a dilated separable convolution module in the dual-branch encoder-decoder network architecture to extract feature information from the ureteral image data to be preprocessed; Feature fusion module: It uses a multi-level pairwise regression module to establish the correlation between feature information at different levels, and achieves feature fusion through pairwise feature matching and adaptive weight generation; Training module: Trains the dual-branch encoder-decoder network architecture based on a weighted loss function; Output module: Used to output ureter segmentation results and edge prediction results.

[0012] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a method and system for ureteral segmentation based on boundary supervision mechanism and multi-scale, which has the following beneficial effects: Significantly improve edge segmentation accuracy: By using the Edge Attention Preservation Module (EAP), the problem of edge information degradation in deep networks is effectively solved, thereby improving edge segmentation accuracy; Enhanced multi-scale feature fusion capability: The dilatant separable convolutional module (DSC) achieves effective integration of local details and global semantic information through a parallel multi-branch structure; Improved segmentation of regions with blurred boundaries: The multi-level pairwise regression module (MPR) establishes the correlation between features at different levels, thereby improving the segmentation accuracy of regions with blurred boundaries. Reduce computational complexity: By employing depthwise separable convolutions and a lightweight network design, computational complexity is significantly reduced while maintaining high accuracy. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0014] Figure 1 The method flowchart provided by the present invention.

[0015] Figure 2 This is an overall architecture diagram provided for the present invention.

[0016] Figure 3 The diagram shows the structure of the EAP module provided by this invention.

[0017] Figure 4 The structure diagram of the DSC module provided by this invention.

[0018] Figure 5 The structure diagram of the MPR module provided by this invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] like Figures 1-2 As shown, this embodiment of the invention discloses a ureteral segmentation method based on boundary supervision mechanism and multi-scale, including: Step 1: Acquire the image data of the ureter to be segmented and perform preprocessing; Step 2: Construct a dual-branch encoder-decoder network architecture based on the pyramid visual transformer; Step 3: Introduce an edge attention preservation module and a dilated separable convolution module into the dual-branch encoder-decoder network architecture to extract feature information from the ureteral image data to be preprocessed; Step 4: Establish the correlation between feature information at different levels using the multi-level pairwise regression module, and achieve feature fusion through pairwise feature matching and adaptive weight generation; Step 5: Train the dual-branch encoder-decoder network architecture based on the weighted loss function, and output the ureter segmentation result and edge prediction result.

[0021] Specifically, the dual-branch encoder-decoder network architecture in step 2 uses the Pyramid Visual Transformer PVT-v2 as the backbone network, including an encoder, a decoder, and a skip connection layer; the encoder is used to extract multi-scale feature representations; the decoder is used to progressively restore spatial resolution; and the skip connection layer is used to preserve detailed information.

[0022] In one specific embodiment of the present invention, the Pyramid Visual Transformer (PVT-v2) is used as the backbone network, supporting multiple versions from b0 to b5.

[0023] Specifically, the Edge Attention Preservation Module (EAP Module) adopts a dual-branch architecture, including: a main feature processing branch unit, an edge feature branch unit, and a feature fusion unit; The main feature processing branch unit uses lightweight deep convolution to extract semantic information; The edge feature branch unit uses the Sobel operator to calculate the gradient and generate an edge attention map; The feature fusion unit achieves adaptive enhancement of edge information through element-wise multiplication.

[0024] In a specific embodiment of the present invention, such as Figure 3 As shown, the EAP module adopts a dual-branch architecture: Main feature processing branch: The input features are compressed through 1×1 convolution (C→C / 4); where C is the number of channels. Spatial features are extracted using 3×3 depthwise convolution; Channel weights are learned through the SE attention mechanism; Use a 1×1 convolution to restore the number of channels (C / 4→C).

[0025] Edge feature branches: Calculate image gradients using the Sobel operator; Edge features are enhanced using 3×3 convolution; Generate an edge attention map.

[0026] Feature fusion: Multiply the main branch features element-wise with the edge attention map; Achieve adaptive enhancement of edge information.

[0027] Furthermore, the specific implementation details of the EAP module are as follows: Input: Feature map F∈R^(C×H×W) Processing procedure: Main branch processing: F1=Conv1x1(F) # Channel compression: C→C / 4 F2=DWConv3x3(F1) #Depthwise convolution F3=SE(F2)#SE attention =Conv1x1(F3)# Channel recovery: C / 4 → C Edge branch handling: = (F)# Horizontal gradient = (F)# Vertical gradient G= #Gradient magnitude E=Conv3x3(G)#Edge feature enhancement =Sigmoid(E)#Attention map Feature fusion: # Element-by-element multiplication Output: Enhanced feature map ∈R^(C×H×W) Specifically, the dilatable separable convolutional module (DSC module) captures feature information from different receptive fields through a parallel multi-branch structure, including a parallel multi-branch processing unit, a dilation rate configuration unit, a feature concatenation and channel fusion unit; The parallel multi-branch processing unit includes multiple branches, batch normalization, and ReLU activation; The expansion rate configuration unit is used to set different expansion rates for each branch; The feature splicing and channel fusion unit is used to integrate the output features while keeping the size of the output feature map unchanged.

[0028] In a specific embodiment of the present invention, such as Figure 4 As shown, the DSC module includes: Parallel multi-branch structure: Branch 1: Expansion rate = 1, receptive field 3×3; Branch 2: Expansion rate = 3, receptive field 7×7; Branch 3: Expansion rate = 5, receptive field 11×11; Branch 4: Expansion rate = 7, receptive field 15×15.

[0029] Feature processing: Each branch contains batch normalization and ReLU activation; Keep the size of the feature map unchanged.

[0030] Feature fusion: The output features of the four branches are concatenated; The dimensionality is reduced to the original number of channels using 1×1 convolution.

[0031] Further details regarding the specific parameter configuration of the DSC module: Expansion rate setting: Branch 1: dilation=1, padding=1 Branch 2: dilation=3, padding=3 Branch 3: dilation=5, padding=5 Branch 4: dilation=7, padding=7 Kernel size: 3×3 for all Output channels: Each branch outputs a C / 4 channel, which is then concatenated to form a C channel.

[0032] Specifically, the multi-level pairwise regression module (MPR module) includes: a pairwise feature matching unit, an adaptive weight generation unit, and a weighted feature fusion unit; The pairwise feature matching unit calculates the similarity matrix between features at different levels to obtain the matching results; The adaptive weight generation unit generates feature fusion weights based on the matching results; The weighted feature fusion unit optimizes and integrates multi-level features based on feature fusion weights.

[0033] In a specific embodiment of the present invention, the edge attention preservation module (EAP) and the dilated separable convolution module (DSC) are located in the decoder part. The different level features refer to four feature processing branches of different scales, and each branch adopts a receptive field of different size.

[0034] In a specific embodiment of the present invention, such as Figure 5 As shown, the processing flow of the MPR module is as follows: Pairwise feature matching: Calculate the similarity matrix between features at different levels; a total of 6 pairs of matching are performed (C(4,2)=6).

[0035] Adaptive weight generation: The fusion weights are generated based on the matching results; the weight parameters include α, β, and γ. Feature fusion: Feature fusion is performed using a weighted summation method; feature refinement is achieved through 3×3 convolution.

[0036] The algorithm flow of the MPR module: Input: Multi-level features {C1, C2, C3, C4} Pair matching: for i=1 to 4: for j = i + 1 to 4: =Similarity( , ) =Softmax( ) The pairing process refers to the weights generated by comparing the four different levels {C1, C2, C3, C4} above, S ij The similarity results between them, M ij The result is then processed by an activation function.

[0037] Weight generation: =FC( ) Feature fusion: .

[0038] in, It refers to the local features of the match. This refers to global features.

[0039] Specifically, the weighted loss function includes: main segmentation loss, edge supervision loss, and auxiliary loss; The main segmentation loss uses a combination of weighted binary cross-entropy loss and weighted IoU loss; Edge-supervised loss is used for supervised training of edge prediction results; The auxiliary loss employs multi-scale supervised loss to balance different training objectives.

[0040] In a specific embodiment of the present invention, the loss function includes: Principal Segmentation Loss: Weighted binary cross-entropy loss: ; Weighted IoU loss: ; Among them, y i It is the real label for image i, w i That is the corresponding weight, p i X represents the model's predicted probability for the image; X and Y represent the image's true label and the model's predicted label, respectively.

[0041] Edge supervision loss: Specifically designed for supervising edge prediction results; employs weighted binary cross-entropy loss.

[0042] Auxiliary loss: Multi-scale supervised loss, balancing different training objectives.

[0043] Furthermore, the network output includes: Main segmentation result: Binary mask of the ureteral region; Edge prediction results: Binary map of the ureteral edge; Confidence map: the segment confidence of each pixel.

[0044] Specifically, a ureteral segmentation system based on boundary supervision mechanisms and multi-scale methods includes: Image acquisition module: used to acquire and preprocess the ureter image data to be segmented; Network building block: Used to build a dual-branch encoder-decoder network architecture based on the pyramid visual transformer; Feature extraction module: used to introduce an edge attention preservation module and a dilated separable convolution module in the dual-branch encoder-decoder network architecture to extract feature information from the ureteral image data to be preprocessed; Feature fusion module: It uses a multi-level pairwise regression module to establish the correlation between feature information at different levels, and achieves feature fusion through pairwise feature matching and adaptive weight generation; Training module: Trains the dual-branch encoder-decoder network architecture based on a weighted loss function; Output module: Used to output ureter segmentation results and edge prediction results.

[0045] Furthermore, this invention can be widely applied to: medical device manufacturers: integration into endoscopic devices; medical imaging software companies: development of professional medical image analysis software; hospitals and clinics: assisting doctors in performing surgery; telemedicine platforms: providing automated medical image analysis services.

[0046] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0047] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A ureteral segmentation method based on boundary supervision mechanism and multi-scale, characterized in that, include: Step 1: Acquire the image data of the ureter to be segmented and perform preprocessing; Step 2: Construct a dual-branch encoder-decoder network architecture based on the pyramid visual transformer; Step 3: Introduce an edge attention preservation module and a dilated separable convolution module into the dual-branch encoder-decoder network architecture to extract feature information from the ureteral image data to be preprocessed; Step 4: Establish the correlation between feature information at different levels using the multi-level pairwise regression module, and achieve feature fusion through pairwise feature matching and adaptive weight generation; Step 5: Train the dual-branch encoder-decoder network architecture based on the weighted loss function, and output the ureter segmentation result and edge prediction result.

2. The ureteral segmentation method based on boundary supervision mechanism and multi-scale as described in claim 1, characterized in that, The dual-branch encoder-decoder network architecture in step 2 uses the Pyramid Visual Transformer PVT-v2 as the backbone network, including an encoder, a decoder, and a skip connection layer; the encoder is used to extract multi-scale feature representations. The decoder is used to gradually restore spatial resolution; the skip connection layer is used to preserve detailed information.

3. The ureteral segmentation method based on boundary supervision mechanism and multi-scale as described in claim 1, characterized in that, The edge attention preservation module adopts a dual-branch architecture, including: a main feature processing branch unit, an edge feature branch unit, and a feature fusion unit; The main feature processing branch unit uses lightweight deep convolution to extract semantic information; The edge feature branch unit uses the Sobel operator to calculate the gradient and generate an edge attention map; The feature fusion unit achieves adaptive enhancement of edge information through element-wise multiplication.

4. The ureteral segmentation method based on boundary supervision mechanism and multi-scale according to claim 1, characterized in that, The dilated separable convolution module captures feature information from different receptive fields through a parallel multi-branch structure, including a parallel multi-branch processing unit, a dilation rate configuration unit, a feature concatenation and channel fusion unit; The parallel multi-branch processing unit includes multiple branches, batch normalization, and ReLU activation; The expansion rate configuration unit is used to set different expansion rates for each branch; The feature splicing and channel fusion unit is used to integrate the output features while keeping the size of the output feature map unchanged.

5. The ureteral segmentation method based on boundary supervision mechanism and multi-scale as described in claim 1, characterized in that, The multi-level pairwise regression module includes: a pairwise feature matching unit, an adaptive weight generation unit, and a weighted feature fusion unit; The pairwise feature matching unit calculates the similarity matrix between features at different levels to obtain the matching results; The adaptive weight generation unit generates feature fusion weights based on the matching results; The weighted feature fusion unit optimizes and integrates multi-level features based on feature fusion weights.

6. The ureteral segmentation method based on boundary supervision mechanism and multi-scale according to claim 1, characterized in that, The weighted loss function includes: main segmentation loss, edge supervision loss, and auxiliary loss; The main segmentation loss uses a combination of weighted binary cross-entropy loss and weighted IoU loss; Edge-supervised loss is used for supervised training of edge prediction results; The auxiliary loss employs multi-scale supervised loss to balance different training objectives.

7. A ureteral segmentation system based on boundary supervision mechanism and multi-scale, characterized in that, include: Image acquisition module: used to acquire and preprocess the ureter image data to be segmented; Network building block: Used to build a dual-branch encoder-decoder network architecture based on the pyramid visual transformer; Feature extraction module: used to introduce an edge attention preservation module and a dilated separable convolution module in the dual-branch encoder-decoder network architecture to extract feature information from the ureteral image data to be preprocessed; Feature fusion module: It uses a multi-level pairwise regression module to establish the correlation between feature information at different levels, and achieves feature fusion through pairwise feature matching and adaptive weight generation; Training module: Trains the dual-branch encoder-decoder network architecture based on a weighted loss function; Output module: Used to output ureter segmentation results and edge prediction results.

Images