Medical image segmentation method, system and electronic device

Abstract

The present invention discloses a medical image segmentation method, belonging to the field of medical image processing. The method includes steps such as dataset preprocessing, establishing a U-shaped baseline segmentation network, constructing a tubular feature extraction module, constructing a multi-scale information fusion module, constructing a boundary guidance module and spatial coordinate information features, constructing a self-attention distillation module, establishing a tubular organ segmentation network, and automatically segmenting lesions through the segmentation network. The method improves the network's segmentation performance for fine terminal branches of tubular organs, learns contextual information at both the global and local scales of the image, and fully utilizes boundary information and spatial position information of the trachea and pulmonary vascular regions to guide the network to focus more on the boundary regions of the target to be segmented, thereby improving the network's segmentation accuracy. The present invention also relates to a system and electronic device that implement the above-mentioned medical image segmentation method.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and in particular to a medical image segmentation method, system and electronic device. Background Art

[0002] Medical image segmentation is the process of dividing a medical image into several non-intersecting, "connected" regions based on certain similarity features. It is a key step in computer-aided diagnosis, such as the segmentation of the trachea and pulmonary vessels in chest CT scans. Tracheal and pulmonary vessel segmentation helps observe morphological changes and diagnose diseases such as bronchial stenosis, chronic obstructive pulmonary disease, pulmonary embolism, and arteriovenous malformations. It also plays a vital role in bronchoscopy and surgical navigation.

[0003] Currently, mainstream tracheal and pulmonary vascular segmentation techniques mostly use deep learning methods based on convolutional neural networks. Although these methods have achieved excellent performance on multiple public datasets, the following challenges remain: 1. Voxel intensity distributions differ between large and small tubular structures. The main branches of the trachea and pulmonary vessels have a clear intensity contrast with the background, but as the diameter decreases and the number of bifurcations increases, the intensity difference between the tubular structures and the background decreases, and the boundaries become increasingly blurred. 2. The spatial distribution of the complete tracheal and pulmonary vascular tree is wide, while the distal ends of the tracheal and pulmonary vascular tubular structures are small. Therefore, tracheal and pulmonary vascular segmentation requires the network to consider contextual information at both global and local scales. 3. Convolutional neural networks primarily extract features based on voxel intensity in the image, but they do not fully utilize spatial structural information (such as boundary information and spatial coordinate position information). To address these challenges, it is of great significance to develop an accurate and widely applicable medical image segmentation algorithm for tracheal and pulmonary vascular segmentation. Summary of the Invention

[0004] In order to overcome the deficiencies of the prior art, one of the objectives of the present invention is to provide a medical image segmentation method that is accurate and widely applicable for segmenting trachea and pulmonary blood vessels.

[0005] In order to overcome the deficiencies of the prior art, a second object of the present invention is to provide a medical image segmentation system with accurate segmentation and wide application for segmenting trachea and pulmonary blood vessels.

[0006] In order to overcome the deficiencies of the prior art, a third object of the present invention is to provide an electronic device with accurate segmentation and wide application, so as to facilitate the segmentation of trachea and pulmonary blood vessels.

[0007] One of the purposes of the present invention is achieved by the following technical solution:

[0008] A medical image segmentation method comprises the following steps:

[0009] Dataset preprocessing: Collect 3D medical images to form a data set and perform data enhancement processing;

[0010] Establish a U-shaped baseline segmentation network: Use 3D U-Net as the baseline network for segmentation. The 3D U-Net network includes an encoding stage and a decoding stage. The encoding stage consists of a three-dimensional convolutional layer, a normalization layer, a nonlinear activation function ReLU, and a maximum pooling layer, and adopts a four-layer resolution structure.

[0011] Constructing a tubular feature extraction module: By stacking graph convolutional layers, nodes in the graph can learn information outside their immediate neighborhood, allowing the network to learn from image regions that are far away from the current voxel but share feature information;

[0012] Construct a multi-scale information fusion module: Use three different-scale dilated convolutions to expand the receptive field and obtain multi-scale feature maps. Use a feature recalibration layer to automatically obtain the weight of each channel. Then, use the weights to enhance useful features and suppress features that are less useful for the current task.

[0013] Constructing a boundary guidance module and spatial coordinate information features: By constructing a boundary guidance module and spatial coordinate information features, the boundary information and spatial location information of the tubular organ region are fully utilized to guide the network to pay more attention to the boundary area of the target to be segmented, thereby improving the network's segmentation accuracy;

[0014] Constructing a self-attention distillation module: By constructing a self-attention distillation module, we can fully capture valuable contextual information. When the supervision signal is insufficient, we explore the use of self-attention distillation itself as an auxiliary learning task, providing additional supervision signals to assist training.

[0015] Establish a tubular organ segmentation network: Insert the tubular feature extraction module, multi-scale information fusion module, and self-attention distillation module into the established baseline network;

[0016] Training the tubular organ segmentation network: The pre-processed original image and the segmentation gold standard are fed into the entire network for supervised learning;

[0017] The segmentation network automatically segments out the lesions: After the network is trained, the network weights are saved. When testing the segmentation network, you only need to input the test image, and the segmentation network will automatically segment out the lesions based on the test image.

[0018] Furthermore, in the step of constructing the tubular feature extraction module, the graph convolution operation of each graph convolution layer is defined as:

[0019]

[0020] Among them, σ() is the rectified linear unit activation function (ReLU), Xl ∈R M×N1 It is a node feature matrix containing M nodes and N1 features input to the lth GNN layer. The weight of the learnable GNN convolution is Z is a binary adjacency matrix, and Y is a degree matrix derived from Z. The adjacency matrix is largely sparse. By processing the adjacency matrix as a sparse tensor, the above graph convolution operation can be completed quickly.

[0021] Furthermore, in the step of constructing the tubular feature extraction module, the initial feature map of the tubular feature extraction module is defined as:

[0022] X 0 =f(H) (2)

[0023] where H∈R M×F is the F-dimensional node feature matrix derived from the 3D U-Net, f() has a linear rectifier unit with F input units and N1 output units and a normalization layer.

[0024] Furthermore, in the multi-scale information fusion module, a feature recalibration layer is used to automatically obtain the weight of each channel. Specifically, the feature recalibration layer first performs global average pooling on the input feature map to achieve feature compression in the spatial dimension, generates a compressed feature map, then activates the feature map, and generates a learnable weight parameter for the feature map of each channel. The weight parameter is used to represent the importance of different feature channels.

[0025] Furthermore, useful features are promoted based on the weights and features that are less useful for the current task are suppressed. Specifically, the activated feature map is broadcasted to restore it to its original size, the corresponding elements are added and then multiplied element-by-element with the original input feature map to obtain a recalibrated feature map, the extracted multi-scale feature map is connected to the feature recalibration layer, and different weights are assigned to information from different channels according to their importance, highlighting the channel information that is helpful for the segmentation task and suppressing the channel information that is not helpful for the segmentation task.

[0026] Furthermore, the network structure of the step of constructing the boundary guidance module and the spatial coordinate information feature is as follows: local boundary information is detected from the extracted shallow feature map to obtain a boundary feature map, the boundary feature map is restored to the original image size after two convolutional layers, and the loss is calculated with the boundary map obtained from the gold standard, the error is obtained and the feature map is updated by backpropagation, the boundary feature map is supervised, the network is guided to strengthen its attention to the boundary area, and the network is helped to better learn the boundary features, the coordinates of the tubular organ voxels are extracted from the original image, and the coordinate information is converted into a multi-channel spatial coordinate information feature map consistent with the network size.

[0027] Furthermore, the loss function used in the process of calculating the boundary feature map is:

[0028] L Edge =-G Edge logP Edge -(1-G Edge )log(1-P Edge ) (3)

[0029] Among them, P Edge represents the boundary feature map, G Edge represents the gold standard of the boundary map, L Edge represents the cross entropy loss between the boundary feature map and the boundary map ground standard.

[0030] Furthermore, the network structure in the step of constructing the self-attention distillation module is as follows: the self-attention distillation module generates an activation-based attention map in the current layer, and uses the later layers as the "teacher layer" and the earlier layers as the "student layer". The attention map obtained by the "teacher layer" is used as the distillation target of the "student layer". Valuable contextual information can be used as a form of auxiliary supervision to further learn by performing attention distillation within the network.

[0031] Furthermore, the steps for constructing the self-attention distillation module are as follows: self-attention distillation is implemented between two consecutive feature layers, where the activation-based attention map is:

[0032]

[0033] Among them, F m is the attention map generation function, A m Represents the activation output of the network layer m, A mi represents the i-th channel of the activation output of the m-th layer. P>1 can focus more on highly activated areas.

[0034] Then perform softmax operation and trilinear interpolation operation on the obtained attention map to obtain the processed attention map Then by minimizing the distillation losses Closer to the next layer of attention map The distillation loss is defined as:

[0035]

[0036] By optimizing distillation losses; Will gradually approach This allows attention to be transferred from the deepest layer to the shallowest layer.

[0037] Furthermore, the step of establishing the tubular organ segmentation network is specifically as follows: the tubular feature extraction module is placed in the last two stages of the encoding end and the first two stages of the decoding end. The tubular feature extraction module obtains the graph structure from the feature graph output by the previous level encoding block / decoding block and uses it as its input. After four graph convolution operations, it outputs a graph with the same nodes as the input graph and the learned feature vector corresponding to each node. The output of the tubular feature extraction module serves as the input of the next level encoding block / decoding block. Then, the multi-scale information fusion module is inserted at the connection between the encoding end and the decoding end, which can capture multi-scale contextual information and reduce the loss of image details. The multi-scale information fusion module introduces a feature recalibration layer, which can adaptively obtain the weight of each channel, and then use this weight to enhance useful features and suppress features that are less useful for the current task.

[0038] Furthermore, in the step of training the tubular organ segmentation network, the total network loss consists of three parts: dice loss is used to reflect the similarity between the network segmentation result and the gold standard, focal loss is used to improve the segmentation accuracy of difficult-to-distinguish samples in the trachea and pulmonary vessels, and radial loss is introduced to describe the spatial topological structure of the trachea and blood vessels. The sum of these three parts of the loss is taken as the total loss, and backpropagation is used to update the parameters to guide the optimization of the entire network.

[0039] Furthermore, the loss function is defined as:

[0040]

[0041] Among them, L dice represents dice loss, i represents each training sample, N2 is the batch size, P i Represents the output probability map of the network, G i Indicates the gold standard of the sample, L focal Represents Focal loss, when G i When p is 1, t (i) = P i , otherwise p t (i) = 1-P i , L radial represents radial loss, p i,k represents the prediction result of the i-th voxel, k∈[0,1], d i,k represents the radial distance map of the i-th voxel, W is the inverse of the volume ratio of different categories, and the total loss is defined as:

[0042] L total =L dice +L focal +λL radial (9)

[0043] Among them, λ is the weight parameter.

[0044] The second object of the present invention is achieved by adopting the following technical solution:

[0045] A medical image segmentation system is provided, wherein the medical image segmentation system is used to implement the above-mentioned medical image segmentation method.

[0046] The third object of the present invention is achieved by adopting the following technical solution:

[0047] An electronic device comprising

[0048] processor;

[0049] a memory, the memory being communicatively connected to the processor;

[0050] The memory stores instructions that can be executed by the processor, and the instructions are executed by the processor to implement the above-mentioned medical image segmentation method.

[0051] Compared with the existing technology, the medical image segmentation method of the present invention has the following advantages:

[0052] 1. This application addresses the problem of unclear intensity contrast between fine tubular structures and the background, as well as blurred boundaries that make segmentation difficult. A tubular feature extraction module is introduced. Through multi-layer graph convolution, the network is able to learn out-of-neighborhood information that is far away from the current node but shares certain features, thereby improving the network's segmentation performance for the fine branches of the trachea and pulmonary blood vessels.

[0053] 2. This application addresses the problem that the trachea and pulmonary blood vessels have a wide spatial distribution range, the internal scale differences of the tubular structure are large, and the network needs to take into account both global and local scale context information. Therefore, a multi-scale information fusion module is introduced, which can extract global scale context information while reducing the loss of image details, and through the feature recalibration layer, it enhances features that are useful for segmentation tasks and suppresses features that are less useful.

[0054] 3. To address the problem of insufficient utilization of spatial structure information by CNN, this application introduces a boundary guidance module and spatial coordinate information features, thereby guiding the network to pay more attention to the boundary area of the tubular structure and improving the network's segmentation performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 is a flow chart of the medical image segmentation method of the present invention;

[0056] Figure 2 The network architecture of the medical image segmentation method of the present invention;

[0057] Figure 3 It is a multi-scale information fusion module of the medical image segmentation method of the present invention. DETAILED DESCRIPTION

[0058] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0059] It should be noted that when a component is referred to as being "fixed to" another component, it may be directly on the other component or there may be another intermediate component through which it is fixed. When a component is considered to be "connected to" another component, it may be directly connected to the other component or there may be another intermediate component at the same time. When a component is considered to be "set on" another component, it may be directly set on the other component or there may be another intermediate component at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention pertains. The terms used in this specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0061] Figure 1 Flowchart of the medical image segmentation method of the present invention, which includes the following steps:

[0062] Dataset preprocessing: Collect 3D medical images to form a data set and perform data enhancement processing;

[0063] Establish a U-shaped baseline segmentation network: Use 3D U-Net as the baseline network for segmentation. The 3D U-Net network includes an encoding stage and a decoding stage. The encoding stage consists of a three-dimensional convolutional layer, a batch normalization layer, a nonlinear activation function ReLU, and a maximum pooling layer, and adopts a four-layer resolution structure.

[0064] Constructing a tubular feature extraction module: By stacking graph convolutional layers, nodes in the graph can learn information outside their immediate neighborhood, allowing the network to learn from image regions that are far away from the current voxel but share feature information;

[0065] Construct a multi-scale information fusion module: Use three different-scale dilated convolutions to expand the receptive field and obtain multi-scale feature maps. Use a feature recalibration layer to automatically obtain the weight of each channel. Then, use the weights to enhance useful features and suppress features that are less useful for the current task.

[0066] Constructing a boundary guidance module and spatial coordinate information features: By constructing a boundary guidance module and spatial coordinate information features, the boundary information and spatial location information of the tubular organ region are fully utilized to guide the network to pay more attention to the boundary area of the target to be segmented, thereby improving the network's segmentation accuracy;

[0067] Constructing a self-attention distillation module: By constructing a self-attention distillation module, we can fully capture valuable contextual information. When the supervision signal is insufficient, we explore the use of self-attention distillation itself as an auxiliary learning task, providing additional supervision signals to assist training.

[0068] Establish a tubular organ segmentation network: Insert the tubular feature extraction module, multi-scale information fusion module, and self-attention distillation module into the established baseline network;

[0069] Training the tubular organ segmentation network: The pre-processed original image and the segmentation gold standard are fed into the entire network for supervised learning;

[0070] The segmentation network automatically segments out the lesions: After the network is trained, the network weights are saved. When testing the segmentation network, you only need to input the test image, and the segmentation network will automatically segment out the lesions based on the test image.

[0071] Please continue reading Figure 2 , the data set preprocessing steps are as follows:

[0072] The data used in this application is a 3D medical image dataset, using CT images as the imaging modality. The CT images have a resolution of 512×512. The CT values of all CT images are truncated to [-1000, 400] and normalized to [0, 1]. Morphological methods are then used to extract lung parenchyma. Due to GPU memory limitations, the CT images are cropped before being fed into the network. Data augmentation uses online data augmentation methods such as random horizontal flips, random -90 to 90 degree rotations, random shifts, and Gaussian smoothing.

[0073] The specific steps to establish a U-shaped baseline segmentation network are:

[0074] In this application, the segmentation network model uses 3D U-Net as the baseline network for segmentation. The encoding phase of the original 3D U-Net network consists of a series of 3D convolutional layers, normalization layers, nonlinear activation functions (ReLU), and maximum pooling layers, while the decoding phase consists of multiple 3D deconvolutional layers and 3D convolutional layers. In this application, a four-layer resolution structure is used to expand the network's receptive field and enhance the network's ability to perceive the overall spatial structure of the trachea and pulmonary vasculature.

[0075] The specific steps for constructing the tubular feature extraction module are:

[0076] The difficulty of segmentation is caused by the unclear contrast between the intensity of fine tubular structures and the background and the blurred boundaries. The present invention is based on a graph neural network (GNN) to construct a tubular feature extraction module to improve the network's ability to segment fine tubular structures. Traditional CNN relies on continuous convolution and pooling to obtain local features and long-range information, but this will lead to a serious reduction in resolution and a large amount of loss of image details. The tubular feature extraction module stacks graph convolution layers to enable the nodes in the graph to learn information outside their directly adjacent neighborhood, allowing the network to learn from image areas that are far away from the current voxel but share certain feature information (such as shape, bifurcation direction, etc.). This information can help the network better predict the subtle branches of the trachea and pulmonary blood vessels, and improve the accuracy of network segmentation.

[0077] Specifically, the tubular feature extraction module consists of multiple graph convolutional layers. By stacking multiple graph convolutional layers together, each node in the graph convolutional layer updates its features using information from high-order neighborhoods, which can improve the segmentation results. The graph convolution operation of each graph convolutional layer is defined as:

[0078]

[0079] Among them, σ() is the rectified linear unit activation function (ReLU), X l ∈R M×N1 is the node feature matrix containing M nodes and N1 features input to the lth GNN layer. The weight of the learnable GNN convolution is W1 l ∈R N×N Z is a binary adjacency matrix and Y is a degree matrix derived from Z. The adjacency matrix is largely sparse, and by processing the adjacency matrix as a sparse tensor, the above graph convolution operation can be completed quickly.

[0080] The initial feature map of the tubular feature extraction module is defined as:

[0081] X 0 =f(H) (2)

[0082] where H∈R M×F The F-dimensional node feature matrix f() is derived from a 3D U-Net. It has F input units and N1 output units, a linear rectifier unit, and a normalization layer. The tubular feature extraction module helps the network learn and combine information from larger image regions, improving the segmentation performance of fine terminal branches of the trachea and pulmonary vessels. This overcomes the shortcomings of CNNs, which rely on local feature extraction and acquire long-range information through continuous convolution and pooling, resulting in severe resolution degradation. This helps alleviate the problem of blurred boundaries and unclear intensity contrast between fine tubular structures such as peripheral bronchi and capillaries and the background.

[0083] The specific steps for constructing a multi-scale information fusion module are:

[0084] To address the problem that the trachea and pulmonary vascular networks need to take into account both global and local scale context information, this paper constructs a multi-scale information fusion module to fuse feature information of different scales, while extracting features with a larger receptive field and retaining image details as much as possible. In terms of network structure design, Figure 3 As shown in the figure, the multi-scale information fusion module consists of two parts: 1. 1×1 convolution, three dilated convolutions of different scales; and 2. a feature recalibration layer. The first part is used to expand the receptive field and obtain multi-scale feature maps. The second part automatically obtains the weights of each channel through learning. Then, based on the weights, it promotes useful features and suppresses features that are less useful for the current task. Specifically, the feature recalibration layer first performs global average pooling on the input feature map to achieve feature compression in the spatial dimensions (depth, height, and width), generating three compressed feature maps. The feature maps are then activated and learnable weight parameters are generated for each channel's feature map. The weight parameters are used to indicate the importance of different feature channels. Finally, the three activated feature maps are broadcasted to restore them to their original size. The corresponding elements are added and then multiplied element-wise with the original input feature map to obtain the recalibrated feature maps. After the multi-scale feature maps extracted by the first part are connected to the feature recalibration layer in the second part, the interdependencies between feature channels are explicitly modeled. The introduction of the feature recalibration layer can assign different weights to information from different channels according to their importance, highlighting the channel information that is helpful for the segmentation task and suppressing the channel information that is not helpful for the segmentation task.

[0085] The specific steps for constructing the boundary guidance module and spatial coordinate information features are:

[0086] To address the problem of insufficient utilization of spatial structure information, the present invention fully utilizes the boundary information and spatial position information of the trachea and pulmonary vascular regions by constructing a boundary guidance module and spatial coordinate information features, thereby guiding the network to pay more attention to the boundary area of the target to be segmented and improving the network's segmentation accuracy.

[0087] In terms of network structure design, since the low-level features extracted by the network contain rich image details, local boundary information is detected from the extracted shallow feature map to obtain a boundary feature map. After passing through two convolutional layers, the boundary feature map is restored to the original image size and the loss is calculated with the boundary map obtained from the gold standard. The error is then backpropagated to update the feature map. This supervises the boundary feature map, guiding the network to strengthen its focus on boundary areas and helping the network to better learn boundary features. Based on prior anatomical knowledge, spatial coordinate information features assist in the segmentation of trachea and pulmonary vessels. The coordinates of the trachea and pulmonary vessel voxels are extracted from the original image and converted into a multi-channel spatial coordinate information feature map consistent with the network size.

[0088] Next, we explain the definition of the loss function used in the process of calculating the boundary feature map:

[0089] L Edge =-G Edge logP Edge -(1-G Edge )log(1-P Edge ) (3)

[0090] Among them, P Edge represents the boundary feature map, G Edge represents the gold standard of the boundary map, L Edge represents the cross entropy loss between the boundary feature map and the boundary map ground standard.

[0091] This application inserts multiple boundary guidance modules and spatial coordinate information features at the network decoding end, which can fully utilize the boundary features and contextual information of the trachea and pulmonary blood vessels at multiple stages, thereby improving the segmentation accuracy of the network.

[0092] The specific steps to build the self-attention distillation module are:

[0093] The present invention fully obtains valuable contextual information by constructing a self-attention distillation module, and explores the use of self-attention distillation itself as an auxiliary learning task when the supervision signal is insufficient, providing additional supervision signals to assist training. In terms of network structure design, the self-attention distillation module generates an activation-based attention map at the current layer, and uses the later layers as the "teacher layer" and the earlier layers as the "student layer". The attention map obtained by the "teacher layer" serves as the target of distillation of the "student layer". The valuable contextual information can be used as a form of auxiliary supervision to further learn by performing attention distillation within the network. Specifically, self-attention distillation is implemented between two consecutive feature layers, where the activation-based attention map is defined as:

[0094]

[0095] Among them, F m is the attention map generation function, A m Represents the activation output of the network layer m, A mi represents the i-th channel of the activation output of the m-th layer. P>1 can pay more attention to highly activated areas (such as lung boundaries, bronchi, blood vessels, etc.). Then, the obtained attention map is subjected to softmax operation and trilinear interpolation operation to obtain the processed attention map. Then by minimizing the distillation losses Closer to the next layer of attention map The distillation loss is defined as:

[0096]

[0097] By optimizing distillation losses; Will gradually approach This allows attention to be transferred from the deepest layer to the shallowest layer. This improves the network's attention to the details of the segmentation target without the need for additional annotation. The self-attention distillation module designed in this invention is placed at the decoding end of the network. This is because the high-level features in the decoder are high-resolution and highly relevant to the segmentation target. Self-attention distillation at the decoding end is used to transfer details that were previously missing in the low-resolution attention map.

[0098] The specific steps to establish a tubular organ segmentation network are:

[0099] The tubular feature extraction module proposed in this paper is inserted into the established baseline network. This module is placed in the last two stages of the encoding and the first two stages of the decoding. It takes the graph structure derived from the feature graph output by the previous encoding / decoding block as its input. After four graph convolution operations, it outputs a graph with the same nodes as the input graph and the learned feature vectors corresponding to each node. The output of this module serves as the input to the next encoding / decoding block.

[0100] The proposed multi-scale information fusion module is then inserted at the junction of the encoder and decoder. This module captures multi-scale contextual information and reduces the loss of image detail. Furthermore, the module incorporates an advanced feature recalibration layer that adaptively weights each channel, using this weighting to enhance useful features and suppress less useful features for the task at hand. This module updates parameters through self-learning, guiding the optimization of the entire segmentation network.

[0101] In order to fully utilize the ability of self-attention distillation to extract pulmonary vascular details, the present invention places a self-attention distillation module on each decoding block at the decoding end. The reason for placing it on the decoding end is that the high-level features in the decoder have high resolution and are highly correlated with the segmentation target, while the encoder end contains a large number of low-level features, which are more local and universal, and the effect of placing the self-attention distillation module is not good. The feature map extracted by the boundary guidance module is inserted into each decoding block at the decoding end, so that the decoding end can learn the boundary feature information from the shallow layer of the network. The spatial coordinate information feature map is cascaded with the output of the third decoding block at the decoding end.

[0102] The specific steps for training the tubular organ segmentation network are:

[0103] To train the entire trachea and pulmonary vascular segmentation network, the preprocessed original image and the segmentation gold standard must be fed into the entire network for supervised learning. The network's total loss consists of three parts: dice loss, which reflects the similarity between the network segmentation result and the gold standard; focal loss, which improves the segmentation accuracy of difficult-to-separate samples in the trachea and pulmonary vessels; and radial loss, which describes the spatial topology of the trachea and blood vessels. The sum of these three losses is used as the total loss, and backpropagation is used to update the parameters, guiding the optimization of the entire network. The loss function is defined as:

[0104]

[0105] Among them, L dice represents dice loss, i represents each training sample, N2 is the batch size, P i Represents the output probability map of the network, G i Indicates the gold standard for this sample. focal Represents Focal loss, when G i When p is 1, t (i) = P i , otherwise p t (i) = 1-P i . L radial represents radial loss, p i,k represents the prediction result of the i-th voxel, k∈[0,1], d i,k represents the radial distance map of the i-th voxel, and W is the inverse of the volume ratio of different categories. The total loss is defined as:

[0106] L total =L dice +L focal +λL radial (9)

[0107] Among them, λ is the weight parameter.

[0108] The specific steps for the segmentation network to automatically segment the lesion are:

[0109] After the network is trained, the network weights are saved. When testing the segmentation network, only the image to be tested is required, without the need for a gold standard. The segmentation network will automatically segment the trachea and pulmonary vessels based on the test image.

[0110] To address the problem of unclear intensity contrast between fine tubular structures and the background and blurred boundaries that make segmentation difficult, the present invention introduces a tubular feature extraction module. Through multi-layer graph convolution, the network can learn out-of-neighborhood information that is far away from the current node but shares certain features, thereby improving the network's segmentation performance for the fine branches of the trachea and pulmonary blood vessels.

[0111] To address the problems of wide spatial distribution of trachea and pulmonary blood vessels, large internal scale differences in tubular structures, and the need for the network to take into account both global and local scale contextual information, this paper introduces a multi-scale information fusion module, which can extract global scale contextual information while reducing the loss of image details, and uses a feature recalibration layer to enhance features that are useful for segmentation tasks and suppress features that are less useful.

[0112] To address the problem that CNN does not fully utilize spatial structure information, the present invention introduces a boundary guidance module and spatial coordinate information features to guide the network to pay more attention to the boundary area of the tubular structure and improve the network's segmentation performance.

[0113] The present application also relates to a system for implementing a medical image segmentation method, wherein the medical image segmentation system includes a data set preprocessing module, a segmentation network, a tubular feature extraction module, a multi-scale information fusion module, a boundary guidance module, a self-attention distillation module, and a training module. The data set preprocessing module is used to process image data, the segmentation network is used to segment images of tubular organs, the tubular feature extraction module enables the network to learn information outside the neighborhood that is far away from the current node but shares certain features, thereby improving the network's segmentation performance for fine branches at the end of the trachea and pulmonary blood vessels, the multi-scale information fusion module can reduce the loss of image details while extracting global scale context information, and enhance features that are useful for segmentation tasks and suppress features that are less useful through a feature recalibration layer. The boundary guidance module guides the network to pay more attention to the boundary area of the tubular structure, thereby improving the segmentation performance of the network.

[0114] The present application also relates to an electronic device for implementing a medical image segmentation method. The electronic device includes a processor and a memory, the memory being communicatively connected to the processor and storing instructions executable by the processor, the instructions being used by the processor to implement the medical image segmentation method.

[0115] The above embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patented invention. It should be noted that those skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention. These variations and improvements are equivalent modifications and improvements to the above embodiments based on the essential technology of the present invention and fall within the scope of protection of the present invention.

Claims

1. A medical image segmentation method, characterized in that: The following steps are involved: Dataset preprocessing: Collect 3D medical images to form a data set and perform data enhancement processing; Establish a U-shaped baseline segmentation network: Use 3D U-Net as the baseline network for segmentation. The 3D U-Net network includes an encoding stage and a decoding stage. The encoding stage consists of a three-dimensional convolutional layer, a normalization layer, a nonlinear activation function ReLU, and a maximum pooling layer, and adopts a four-layer resolution structure. Constructing a tubular feature extraction module: By stacking graph convolution layers, nodes in the graph can learn information outside their immediate neighborhood, allowing the network to learn from image regions that are far away from the current voxel but share feature information. The graph convolution operation of each graph convolution layer is defined as: Where σ is the rectified linear unit activation function, X l ∈R M×N1 It is a node feature matrix containing M nodes and N1 features input to the lth GNN layer. The weight of the learnable GNN convolution is Z is a binary adjacency matrix, and Y is a degree matrix derived from Z. The adjacency matrix is largely sparse. By processing the adjacency matrix into a sparse tensor, the above graph convolution operation can be completed quickly. Construct a multi-scale information fusion module: Use three different scales of dilated convolution to expand the receptive field and obtain multi-scale feature maps. Use a feature recalibration layer to automatically obtain the weight of each channel. Then, based on the weight, use it to enhance useful features and suppress features that are less useful for the current task. Use a feature recalibration layer to automatically obtain the weight of each channel. Specifically, the feature recalibration layer first performs global average pooling on the input feature map to achieve feature compression in the spatial dimension, generating a compressed feature map. Then, it activates the feature map and generates a learnable weight parameter for the feature map of each channel. The weight parameter is used to represent the importance of different feature channels. Constructing a boundary guidance module and spatial coordinate information features: By constructing a boundary guidance module and spatial coordinate information features, the boundary information and spatial position information of the tubular organ region are fully utilized to guide the network to pay more attention to the boundary area of the target to be segmented, thereby improving the segmentation accuracy of the network. The network structure of the step of constructing the boundary guidance module and spatial coordinate information features is as follows: local boundary information is detected from the extracted shallow feature map to obtain a boundary feature map. The boundary feature map is restored to the original image size after two convolutional layers, and the loss is calculated with the boundary map obtained from the gold standard to obtain the error and back-propagate to update the feature map. The boundary feature map is supervised to guide the network to strengthen its attention to the boundary area, helping the network to better learn the boundary features, extract the coordinates of the tubular organ voxels from the original image, and convert the coordinate information into a multi-channel spatial coordinate information feature map consistent with the network size; Constructing a self-attention distillation module: By constructing a self-attention distillation module, we can fully obtain valuable contextual information and explore the use of self-attention distillation itself as an auxiliary learning task when the supervision signal is insufficient, providing additional supervision signals to assist training. The specific steps of constructing the self-attention distillation module are as follows: Self-attention distillation is implemented between two consecutive feature layers, where the activation-based attention map is: Among them, F m is the attention map generation function, A m Represents the activation output of the network layer m, A mi represents the i-th channel of the activation output of the m-th layer. P>1 can focus more on highly activated areas. Then perform softmax operation and trilinear interpolation operation on the obtained attention map to obtain the processed attention map Then by minimizing the distillation losses Closer to the next layer of attention map The distillation loss is defined as: By optimizing distillation losses; Will gradually approach Allows attention to be transferred from the deepest layer to the shallowest layer; Establishing a tubular organ segmentation network: Inserting a tubular feature extraction module, a multi-scale information fusion module, and a self-attention distillation module into the established baseline network; the steps of establishing the tubular organ segmentation network are as follows: the tubular feature extraction module is placed in the last two stages of the encoder and the first two stages of the decoder. The tubular feature extraction module obtains a graph structure from the feature graph output by the previous encoding block / decoding block and uses it as its input. After four graph convolution operations, it outputs a graph with the same nodes as the input graph and the learned feature vector corresponding to each node. The output of the tubular feature extraction module serves as the input of the next encoding block / decoding block. The multi-scale information fusion module is then inserted at the connection between the encoder and decoder. It can capture multi-scale contextual information and reduce the loss of image details. The multi-scale information fusion module introduces a feature recalibration layer that can adaptively obtain the weight of each channel, and then use this weight to enhance useful features and suppress features that are less useful for the current task. Training the tubular organ segmentation network: The pre-processed original image and the segmentation gold standard are fed into the entire network for supervised learning; The segmentation network automatically segments out the lesions: After the network is trained, the network weights are saved. When testing the segmentation network, you only need to input the test image, and the segmentation network will automatically segment out the lesions based on the test image.

2. The medical image segmentation method according to claim 1, wherein: In the step of constructing the tubular feature extraction module, the initial feature map of the tubular feature extraction module is defined as: X 0 =f(H) (2) where H∈R M×F is the F-dimensional node feature matrix derived from the 3D U-Net, f has a linear rectifier unit with F input units and N1 output units and a normalization layer.

3. The medical image segmentation method according to claim 1, wherein: The method is to enhance useful features and suppress features that are less useful for the current task based on weights. Specifically, the activated feature map is broadcasted to restore it to its original size, the corresponding elements are added and then multiplied element-by-element with the original input feature map to obtain a recalibrated feature map. The extracted multi-scale feature map is connected to the feature recalibration layer, and different weights are assigned to information from different channels according to their importance, highlighting the channel information that is helpful for the segmentation task and suppressing the channel information that is not helpful for the segmentation task.

4. The medical image segmentation method according to claim 1, wherein: The loss function used in the process of calculating the boundary feature map is: L Edge =-G Edge log P Edge -(1-G Edge )log(1-P Edge ) (3) Among them, P Edge represents the boundary feature map, G Edge represents the gold standard of the boundary map, L Edge represents the cross entropy loss between the boundary feature map and the boundary map ground standard.

5. The medical image segmentation method according to claim 1, wherein: The network structure of the self-attention distillation module is as follows: the self-attention distillation module generates an activation-based attention map in the current layer, and uses the later layers as the "teacher layer" and the earlier layers as the "student layer". The attention map obtained by the "teacher layer" serves as the distillation target of the "student layer". Valuable contextual information can be used as a form of auxiliary supervision to further learn by performing attention distillation within the network.

6. The medical image segmentation method according to claim 1, wherein: During the training of the tubular organ segmentation network, the total network loss consists of three parts: dice loss is used to reflect the similarity between the network segmentation results and the gold standard; focal loss is used to improve the segmentation accuracy of difficult-to-distinguish samples in the trachea and pulmonary vessels; and radial loss is introduced to describe the spatial topological structure of the trachea and blood vessels. The sum of these three losses is used as the total loss, and backpropagation is used to update the parameters to guide the optimization of the entire network.

7. The medical image segmentation method according to claim 6, wherein: The loss function is defined as: Among them, L dice represents dice loss, i represents each training sample, N2 is the batch size, P i Represents the output probability map of the network, G i Indicates the gold standard of the sample, L focal Represents Focal loss, when G i When p is 1, t (i) = P i , otherwise p t (i) = 1-P i , L radial represents radial loss, p i,k represents the prediction result of the i-th voxel, k∈[0,1], d i,k represents the radial distance map of the i-th voxel, W is the inverse of the volume ratio of different categories, and the total loss is defined as: THE total =L dice +L focal +λL radial (9) Among them, λ is the weight parameter.

8. A medical image segmentation system, characterized by: The medical image segmentation system is used to implement the medical image segmentation method according to any one of claims 1 to 7.

9. An electronic device, characterized in that: include processor; a memory, the memory being communicatively connected to the processor; The memory stores instructions that can be executed by the processor, and the instructions are executed by the processor to implement the medical image segmentation method according to any one of claims 1 to 7.

Images