Joint segmentation method and system for left atrial and left atrial appendage tissue from ct images

Abstract

This invention provides a method and system for joint segmentation of the left atrium and left atrial appendage in CT images, comprising: constructing a first-stage dataset; building a first-stage neural network model N1 for joint segmentation of the left atrium and left atrial appendage; training the first-stage neural network model N1 using the first-stage dataset; testing the first-stage neural network model N1 using the first-stage dataset; sampling the coarse segmentation results of the first-stage neural network model N1 to construct a second-stage dataset; building a second-stage neural network model N2 for optimizing the coarse segmentation results; training the second-stage neural network model N2 using the second-stage data; and testing the second-stage neural network model N2 using the second-stage data. This invention achieves modeling of the relative positional relationship between the left atrium and left atrial appendage based on a neural network model, while simultaneously achieving accurate segmentation of the left atrium and left atrial appendage.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a method, system, medium, and terminal for joint segmentation of the left atrium and left atrial appendage tissue in CT images. Background Technology

[0002] Atrial fibrillation (AF) is one of the most common heart-related diseases, closely related to the left atrium. Adjacent to this cavity-like structure lies the more intricate left atrial appendage (LAA). The vast majority of strokes caused by AF are due to blood clots within the LAA. Anticoagulation therapy is a common treatment for stroke; however, many patients have contraindications to this therapy. Therefore, a more effective and feasible left atrial appendage occlusion technique has emerged. This method avoids most of the drawbacks of anticoagulation therapy.

[0003] The size of the occluder is closely related to the size of the interface between the left atrium and the left atrial appendage. Therefore, strengthening research on the topology of the left atrium and left atrial appendage will help in the research on the treatment of stroke caused by atrial fibrillation. Preoperative cardiac CT imaging is a typical interventional method for this disease. Therefore, fully automated and accurate segmentation of CT images of the left atrium and left atrial appendage is crucial for the diagnosis and treatment of this disease.

[0004] To date, much work has focused on the automatic segmentation of the left atrium; in contrast, research on the left atrial appendage is insufficient, especially on modeling the relative positional relationship between the two. The shapes and structures of the left atrium and left atrial appendage differ significantly. Furthermore, compared to the left and right atria and ventricles, the left atrial appendage has more indeterminate edges, particularly at its interface with the left atrium, making accurate segmentation of this region challenging.

[0005] Several works have been developed to address the problem of segmenting rough edges, and these works can be broadly categorized into three types.

[0006] The first type of strategy introduces the concept of multi-task learning, applying a strong constraint to the boundary region using a loss function. The second type optimizes the coarse segmentation results of the edge region through a complex post-processing procedure. However, both of these methods are mainly used to optimize regions with clear edges in high-quality images and are not suitable for uncertain edges in the left atrium and the left atrial appendage interface.

[0007] In contrast, the third approach works by enhancing the depth features corresponding to uncertain edge regions, thereby achieving precise localization of those regions. Current methods focus on implicitly incorporating edge region annotations into edge features through attention mechanisms, or on introducing uncertainty into edge pixels, but neither of these methods adequately describes the uncertainty of edges. Summary of the Invention

[0008] To address the shortcomings of existing technologies, the purpose of this invention is to provide a method and system for joint segmentation of the left atrium and left atrial appendage tissue in CT images.

[0009] According to one aspect of the present invention, a method for joint segmentation of the left atrium and left atrial appendage tissue in CT images is provided, comprising:

[0010] Construct the first-stage dataset;

[0011] A first-stage neural network model N1 for joint segmentation of the left atrium and left atrial appendage was constructed.

[0012] Using the first-stage dataset, train the first-stage neural network model N1;

[0013] Using the first-stage dataset, test the first-stage neural network model N1;

[0014] The coarse segmentation results of the first-stage neural network model N1 on the validation set are sampled to construct the second-stage dataset.

[0015] Construct a second-stage neural network model N2 to optimize the coarse segmentation results;

[0016] The second-stage neural network model N2 is trained using the data from the second stage.

[0017] The second-stage neural network model N2 was tested using the second-stage data.

[0018] The left atrium and left atrial appendage in CT images are segmented using the trained first-stage neural network model N1 and the second-stage neural network model N2.

[0019] Preferably, the first-stage neural network model N1 is a 3D medical image segmentation neural network model with multi-scale feature extraction, feature enhancement and feature fusion;

[0020] The operations in the first-stage neural network model N1 include:

[0021] A hierarchical neural network model is used to extract multi-scale image features from 3D CT images;

[0022] The multi-scale image features are enhanced by using a semantic difference module designed based on diffusion theory to obtain the enhanced features.

[0023] The enhanced features are then fused at multiple scales to obtain the final deep-coded features.

[0024] The deep encoded features are input into a segmentation network to segment the foreground target, resulting in a coarse segmentation result;

[0025] The training of the first-stage neural network model N1 includes:

[0026] Using the annotation results of the first-stage dataset and the coarse segmentation results, a loss function is constructed, and the model is trained to optimize the parameters of the first-stage neural network model N1.

[0027] Preferably, the enhanced features include two parts: one part is the original features from the hierarchical neural network model, and the other part is the enhanced edge features from the semantic difference module.

[0028] Preferably, the semantic difference module is designed based on diffusion theory and includes:

[0029] The diffusion theory is described using second-order partial differential equations:

[0030]

[0031] in, Let D represent the gradient operator, and let D represent the diffusion function, which determines the diffusion rate along a specific direction; the diffusion function D is... It exhibits nonlinear characteristics, employing deep features G from convolutional networks as the independent variable of the diffusion function. A semantic guidance graph representing the difference operator, where h represents the convolution-based mapping function;

[0032] Solving the second-order partial differential equation yields an approximate numerical solution:

[0033]

[0034]

[0035] Where p represents a specific pixel in the feature map, δ p λ represents a small neighborhood around pixel p, and v are weighting coefficients. Characterizes the original feature F t The value of the difference feature map at point p. Semantic difference map for deep features; Indicates the enhanced features, Indicates original features, This represents the edge features enhanced by the semantic difference module;

[0036] In the convolution-based mapping function h, a learnable edge operator is proposed, allowing the convolution kernel to have different values ​​at various positions while ensuring that the kernel has differential properties. The value at the exact center of the kernel is set to 1, resulting in the final enhanced edge features.

[0037]

[0038] in and Let F and G represent the learnable edge operators corresponding to feature F and deep semantic feature G, respectively. This represents the convolution kernel.

[0039] Preferably, the training process of the second-stage neural network model N2 includes:

[0040] The coarse segmentation results are sampled, including regions where the left atrium and left atrial appendage are not connected;

[0041] The images and coarse segmentation masks corresponding to the disconnected regions are extracted and simultaneously input into the second-stage neural network model N2;

[0042] Employing a connectivity loss function based on distance constraints Training is performed using the Dess loss function.

[0043] Preferably, the connectivity loss function based on distance constraints is adopted. Training using the Descein loss function includes:

[0044] Calculate the connectivity loss function

[0045] The left atrial appendage region is located by extracting the point-by-point coordinates of the left atrial appendage region through the pixel label information of the segmented truth mask;

[0046] Find the approximately smallest circumscribed hexahedron containing the left atrial appendage region, and determine the four vertices V on the hexahedron that are closest to the surface of the left atrium. i (i = 1, 2, 3, 4);

[0047] The connectivity loss function is calculated using the minimum distances from the four nearest vertices to the surface of the left atrium.

[0048]

[0049] Where σ represents the activation function, P represents the set of points on the surface of the left atrium, P j V represents an element in set P. i(i = 1, 2, 3, 4) represent the four vertices on the circumscribed hexahedron of the left atrial appendage that are closest to the surface of the left atrium, S represents the normalization coefficient (set to 20 in this experiment), and D represents the Euclidean distance;

[0050] Calculate the total loss function

[0051]

[0052] in, λ is the Descein loss function; when the number of training rounds of the second-stage network model N2 exceeds the set number, λ is 1; when the number of rounds is less than the set number, λ is 0.

[0053] Preferably, when testing the second-stage neural network model N2 using the second-stage data, channel correction is performed. Specifically, the image of the intercepted disconnected region and the coarse segmentation mask are simultaneously input into the second-stage neural network model N2, and the decoded features F are obtained through the encoder-decoder structure. d ; In the case of feature F d Before selecting the maximum activation value per pixel by channel, the maximum value of each channel in the feature is first unified.

[0054] According to a second aspect of the present invention, a joint segmentation system for left atrium and left atrial appendage tissue in CT images is provided, comprising:

[0055] The first data module constructs the first-stage dataset.

[0056] The first network model module, the first neural network model N1 built in this module is used for the joint segmentation of the left atrium and left atrial appendage;

[0057] The first training module uses the first-stage dataset to train the neural network model N1.

[0058] The first test module uses the first-stage dataset to test the neural network model N1.

[0059] The second data module samples the coarse segmentation results of the first-stage neural network model N1 to construct the second-stage dataset.

[0060] The second network model module is used to build a second-stage neural network model N2 for optimizing the coarse segmentation results.

[0061] The second training module uses the second-stage data to train the second-stage neural network model N2.

[0062] The second test module uses the second-stage data to test the second-stage neural network model N2.

[0063] According to a third aspect of the present invention, a terminal is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to execute any of the methods for joint segmentation of left atrium and left atrial appendage tissue in CT images, or to run the system for joint segmentation of left atrium and left atrial appendage tissue in CT images.

[0064] According to a fourth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, can be used to perform any of the methods for joint segmentation of left atrium and left atrial appendage tissue in CT images, or to run the system for joint segmentation of left atrium and left atrial appendage tissue in CT images.

[0065] Compared with the prior art, the embodiments of the present invention have at least one of the following beneficial effects:

[0066] This invention provides a method and system for joint segmentation of left atrium and left atrial appendage tissue in CT images. It is the first to realize the modeling of the relative positional relationship between left atrium and left atrial appendage tissue based on a neural network model, and to achieve accurate joint segmentation of left atrium and left atrial appendage tissue using labeled image data of left atrium and left atrial appendage.

[0067] The present invention provides a method and system for joint segmentation of the left atrium and left atrial appendage tissue in CT images, which can be applied to the clinical diagnosis and surgical treatment of diseases such as atrial fibrillation. Attached Figure Description

[0068] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0069] Figure 1 This is a flowchart of a method for joint segmentation of CT images of the left atrium and left atrial appendage according to an embodiment of the present invention;

[0070] Figure 2 This is a schematic diagram of the first-stage neural network model for segmenting the left atrium and left atrial appendage according to a preferred embodiment of the present invention.

[0071] Figure 3 This is a schematic diagram of a second-stage neural network model for segmenting the left atrium and left atrial appendage according to a preferred embodiment of the present invention.

[0072] Figure 4 This is a schematic diagram of the semantic difference module (SDM) according to a preferred embodiment of the present invention.

[0073] Figure 5 This is an iterative optimization diagram of the second-stage neural network model for the coarse segmentation result according to a preferred embodiment of the present invention;

[0074] Figure 6 This is a comparison diagram of the segmentation results of the method of the present invention and the existing method, which is a preferred embodiment of the present invention. Detailed Implementation

[0075] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0076] This invention employs the third strategy described in the background technology, namely, based on enhancing the depth features corresponding to uncertain edge regions to achieve precise localization of those regions. However, this invention differs from existing attention-based technologies by introducing a thermodynamic diffusion theory model into the enhancement module for uncertain regions for the first time. Specifically, edge formation is modeled as the final stable state of pixel diffusion, and the diffusion process itself is also a process of generating uncertainty. Therefore, diffusion theory has a stronger ability to model uncertainty compared to existing technologies.

[0077] Meanwhile, noise during imaging and blurred edges in the left atrial appendage region lead to a lack of communication between the left atrium and left atrial appendage in some cases. This problem hinders accurate segmentation of the interface between the two, reducing the efficiency of clinical surgical intervention. Therefore, for the joint segmentation task of the left atrium and left atrial appendage, especially for the need for accurate segmentation of uncertain edges, it is necessary to design appropriate deep learning methods.

[0078] Based on the above-described inventive concept, this invention provides an embodiment of a method for joint segmentation of the left atrium and left atrial appendage tissue in CT images, see [link to embodiment]. Figure 1 The specific process is as follows:

[0079] S11, Construct the first-stage dataset;

[0080] S12, Construct the first-stage neural network model N1 for joint segmentation of the left atrium and left atrial appendage;

[0081] S13, using the first-stage dataset built in S11, train the first-stage neural network model N1 built in S12;

[0082] S14, using the first-stage dataset built in S11, tests the first-stage neural network model N1 that was trained in S13;

[0083] S15, sample the coarse segmentation results of the first-stage neural network model N1 to construct the second-stage dataset;

[0084] S16, Build the second-stage neural network model N2 to optimize the coarse segmentation test results;

[0085] S17, use the second-stage data constructed in S15 to train the second-stage neural network model N2 built in S15;

[0086] S18 uses the second-stage data constructed in S15 to test the second-stage neural network model N2 trained in S17.

[0087] The left atrium and left atrial appendage in CT images are segmented using the trained first-stage neural network model N1 and the second-stage neural network model N2.

[0088] This embodiment realizes the modeling of the relative positional relationship between the left atrium and the left atrial appendage based on a neural network model, and at the same time achieves accurate segmentation of the left atrium and the left atrial appendage.

[0089] In a preferred embodiment of the present invention, S11 is implemented, in which a given cardiac CT three-dimensional image is divided into a training set, a validation set, and a test set.

[0090] In a preferred embodiment of the present invention, the first-stage neural network model N1 established in S12 is as follows: Figure 2 As shown, this is a 3D medical image segmentation neural network model with multi-scale feature extraction, feature enhancement, and feature fusion.

[0091] In a preferred embodiment, S12, the process of building the first-stage neural network model N1, includes:

[0092] S121 uses a hierarchical neural network model (encoder) to extract multi-scale image features from 3D CT images;

[0093] S122, the multi-scale features in S121 are enhanced by using a semantic difference module designed based on diffusion theory to obtain the enhanced features;

[0094] S123, perform multi-scale fusion on the enhanced features of S122 to obtain the final deep coding features;

[0095] S124: Input the deep encoding features of S123 into the segmentation network to segment the foreground target and obtain the segmentation result.

[0096] In a preferred embodiment, the specific process of implementing S13, namely training the first-stage neural network model N1, is as follows: construct a loss function (mask ground truth combined with Dessian loss + cross-entropy loss) using the expert annotation results and segmentation results, train the first neural network model N1 using the training set of S11, and optimize the neural network model parameters.

[0097] Furthermore, the enhanced features in S122 consist of two parts: one part is the original features from the hierarchical neural network model, and the other part is the semantic difference module from the semantic difference module, which enhances the edge features. Figure 4 As shown, this module is designed based on diffusion theory. Diffusion theory can be described by the following second-order partial differential equation:

[0098]

[0099] in Let F(t) denote the gradient operator, and D denote the diffusion function, which determines the diffusion rate along a specific direction. In this embodiment of the invention, D = D(F) has nonlinear characteristics, which results in a greater smoothing effect on regions parallel to the edge and a smaller smoothing effect on regions perpendicular to the edge during diffusion. The steady state of F(t) characterizes the better localization of the uncertain edge of the junction region between the left atrium and the left atrial appendage.

[0100] Specifically, for a given initial feature F, it is updated progressively with the diffusion process. In regions near the uncertainty edge, the diffusion motion is suppressed, while in regions far from the edge, the diffusion motion is enhanced.

[0101] In a preferred embodiment, the diffusion function D = D(F) exhibits nonlinear characteristics. Furthermore, since the localization of uncertain edges requires semantic information for guidance, deep features G from the convolutional network are introduced as the independent variable of the diffusion function. In summary, It is used as a diffusion function with nonlinear properties, where The semantic guidance graph represents the difference operator, and h represents the convolution-based mapping function. According to the theory of partial differential equations, the above second-order partial differential equation can be approximated numerically by the following equation:

[0102]

[0103]

[0104] Where p represents a specific pixel in the feature map, δ p λ represents a small neighborhood around pixel p, and v are weighting coefficients. Characterizes the original feature F tThe value of the difference feature map at point p contains rich boundary information. Clearly, this difference feature map includes both complex edge features and high-frequency noise. Therefore, a semantic difference map from deep features is introduced. As a guide, thus optimizing edge features Especially regarding the indeterminate boundary between the left atrium and the left atrial appendage. As time t increases, It will gradually diffuse into a stable state, which can accurately locate uncertain edges. Ultimately, the optimized feature F... t+1 From the original feature F t and enhanced edge features Add them together to get the result.

[0105] Meanwhile, due to the anisotropic distribution of data from the left atrium and left atrial appendage in the x, y, and z directions, traditional edge detection operators cannot effectively extract the difference map of feature F. Therefore, this embodiment proposes a learnable edge operator with a convolutional kernel that has different values ​​at various positions. To ensure the kernel possesses difference properties, the value at the very center of the kernel is set to -1. The final enhanced edge features... The following formula can be used to calculate:

[0106]

[0107] in and Let F and G represent the learnable edge operators corresponding to feature F and deep semantic feature G, respectively. This indicates a 3×3×3 convolution kernel.

[0108] The first-stage hierarchical neural network model N1 constructed above can yield a rough segmentation result for the test set.

[0109] Because the left atrial appendage region has blurred edges and there is noise during CT imaging, the coarse segmentation results generated by the first-stage network result in a disconnect between the left atrium and the left atrial appendage. This disconnection hinders accurate segmentation of the interface between the two, reducing the efficiency of clinical interventional treatment. To solve this problem, in another preferred embodiment of the present invention, step S16 is implemented to build a second-stage neural network model N2, also called a connectivity optimization network, the specific structure of which is as follows: Figure 3 As shown.

[0110] This invention uses 95% Hausdorff distance (hd95) as one of the performance indicators for segmentation, which characterizes the segmentation accuracy of the left atrium and left atrial appendage structures. Specifically, if there is a disconnect between the left atrium and the left atrial appendage, the hd95 value of the left atrial appendage region will be large, thus affecting the segmentation performance. Therefore, this embodiment proposes a novel connectivity loss function based on distance constraints. Implement S17.

[0111] Specifically, in the second stage of neural network training, this embodiment focuses on sampling the coarse segmentation results from the first stage, especially in regions where the left atrium and left atrial appendage are disconnected. Simultaneously, the image of the cropped region and the coarse segmentation mask are input into the network to enrich its input information. Finally, a loss function is used... Conduct training.

[0112] Furthermore, in calculating the loss function In the process, the left atrial appendage region is first located using the uniqueness of the labels. Then, to ensure the left atrium and left atrial appendage regions are adjacent, this embodiment aims to bring points in the left atrial appendage closest to the boundary as close as possible to the interface. Considering computational efficiency, this embodiment first encloses the left atrial appendage region with an approximate minimum circumscribed hexahedron C. Then, the four vertices Vi (i = 1, 2, 3, 4) on C closest to the left atrium are found, and the connectivity loss function is calculated using these four minimum distances. And Des lost It is also used as a regularization constraint for network training. The detailed calculation process is as follows:

[0113]

[0114] Total loss function It is calculated by the following formula:

[0115]

[0116] The value of λ is determined using the following strategy: when the number of training epochs in the second stage network exceeds 300, λ is set to 1; when the number of epochs is less than 300, λ is set to 0. Simultaneously, because this stage of the network focuses on optimizing cases with poor connectivity, sampling of cases with poor connectivity from the validation set data in the first stage is increased during training.

[0117] After the second-stage neural network iterative optimization, the connectivity of cases with poor connectivity was significantly improved, as shown in the specific results. Figure 5 As shown.

[0118] In one embodiment of the invention, S18 is implemented. During the testing phase, the final decoded feature F... d The softmax operator is applied to extract the predicted mask. However, F d Different channels often have different maximum activation values, so directly applying softmax will affect the final segmentation performance. Therefore, this embodiment proposes the concept of channel correction. Specifically, the image of the truncated disconnected regions and the coarse segmentation mask are simultaneously input into the second-stage neural network model N2, and the decoded features F are obtained through an encoder-decoder structure. d ; In the case of feature F d Before selecting the maximum activation value per pixel by channel, the maximum value of each channel in the feature is first unified.

[0119] Based on the same inventive concept, in other embodiments of the present invention, a joint segmentation system for left atrium and left atrial appendage tissue in CT images is provided, comprising:

[0120] The first data module constructs the first-stage dataset.

[0121] The first network model module, the first neural network model N1 built in this module is used for the joint segmentation of the left atrium and left atrial appendage;

[0122] The first training module uses the first-stage dataset to train the neural network model N1.

[0123] The first test module uses the first-stage dataset to test the neural network model N1.

[0124] The second data module samples the coarse segmentation results of the first-stage neural network model N1 to construct the second-stage dataset.

[0125] The second network model module is used to build a second-stage neural network model N2 to optimize the coarse segmentation results.

[0126] The second training module uses the second-stage data to train the second-stage neural network model N2.

[0127] The second test module uses the second-stage data to test the second-stage neural network model N2.

[0128] The specific implementation techniques of each module / unit in the above examples of the present invention can be referred to the steps of the joint segmentation method of the left atrium and left atrial appendage tissue in CT images in the above embodiments, and will not be repeated here.

[0129] Based on the same inventive concept, in other embodiments of the present invention, a terminal is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to execute any of the following methods for joint segmentation of the left atrium and left atrial appendage tissue in CT images, or to run a joint segmentation system for the left atrium and left atrial appendage tissue in CT images.

[0130] Based on the same inventive concept, in other embodiments of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, can be used to perform any of the methods for joint segmentation of the left atrium and left atrial appendage tissue in CT images, or to run a system for joint segmentation of the left atrium and left atrial appendage tissue in CT images.

[0131] To facilitate a more in-depth and detailed understanding of the technical solutions in the above embodiments, the present invention provides a specific application embodiment to verify the effects of the above embodiments.

[0132] 1. Experimental conditions

[0133] In this embodiment, the first stage uses 45 CT scan images as the training set for the model. The size of the image to be detected is z×512×512 pixels, where z represents the number of 2D slices. Additionally, 15 CT images outside the training set are used as a validation set to adjust the model's hyperparameters; 20 CT images outside the training and validation sets are used as the test set to verify the model's performance in joint segmentation of left atrium and left atrial appendage CT images. The second stage involves obtaining 50 coarse segmentation results from the validation set in the first stage, of which 30 results show connectivity issues. These 50 data points are then divided into a training set and a validation set, containing 35 and 15 data points respectively. The model is implemented using the PyTorch 1.8.0 platform, with two Nvidia Tesla V100 GPUs for computation.

[0134] The parameters during training are as follows: Both stages use the AdamW optimizer with an initial learning rate of 5e-4 and a cosine learning rate decay modulator with a weight decay coefficient of 1e-5. The batch size for each training iteration is set to 2, and the size of the pruned patch is 160×160×192. The first-stage 3D UNet model has a size of 23.47M and is trained for 2000 generations. The loss function is constructed using expert annotations and the segmentation results from the first stage, and is a weighted sum of Dessian loss and cross-entropy loss. The second-stage 3D UNet model has 1.92M parameters and is trained for 500 generations, using a weighted sum of Dessian loss and connectivity loss. The model with the highest performance on the validation set is selected as the final model.

[0135] 1. Experiment Content

[0136] The following section verifies the combined segmentation method of the left atrium and left atrial appendage proposed in this embodiment from both qualitative and quantitative perspectives.

[0137] 2.1 Qualitative Experimental Results

[0138] Figure 6 This embodiment uses full-image semantic segmentation of images in the test set and compares the results with those obtained by other methods.

[0139] Figure 6 The segmentation performance of 3D UNet, VNet, nnUNet, UNETR, SwinUNETR, UNeXt, and the method of this embodiment were compared under the same experimental conditions. It can be seen that the method proposed in this embodiment performs better in terms of localization performance on the uncertain boundary regions of the left atrium and left atrial appendage, as well as the connectivity between them, verifying that this embodiment has a good effect on the joint segmentation of CT images of the left atrium and left atrial appendage.

[0140] 2.2 Quantitative Analysis

[0141] Quantitative error analysis was performed on the semantic segmentation results of 20 CT images in the test set using the method of this embodiment. The methods compared were 3D UNet, ResUNet, VNet, TransBTS, UNETR, Swin UNETR, UNeXt, and nnUNet. The evaluation metrics were the Dice score coefficient (DSC (%)) and the 95% Hausdorff distance (hd95 (mm)).

[0142]

[0143] Where i represents the pixel index, N is the total number of pixels, pi represents the binary segmentation result of a specific point in the prediction result, and gi represents the binary segmentation result of a specific point in the expert annotation result. The value of DSC ranges from 0 to 1, and the closer it is to 1, the better the algorithm performance.

[0144]

[0145] P represents the set of voxels representing the model's predictions, and G represents the set of voxels representing the expert annotations. d represents the Euclidean distance. hd95 is a positive number; the smaller the value, the better the algorithm's performance.

[0146] The experimental results are shown in Table 1 after testing the images in the test set using different methods:

[0147] Table 1

[0148]

[0149]

[0150] As can be seen from the results in Table 1, the method proposed in this embodiment achieves better segmentation performance in both the left atrium and left atrial appendage, achieving a segmentation DSC of 88.91% and an HD95 of 7.75 mm on the entire test set. Compared with other classic network structures, these results demonstrate that the joint segmentation method proposed in this embodiment, based on uncertain edge enhancement and connectivity optimization between adjacent tissues, performs better on the test set, especially for the left atrial appendage region.

[0151] Meanwhile, on the images in the test set, the performance of the semantic difference module was compared with that of other modules used to handle uncertain edges, and the experimental results are shown in Table 2:

[0152] Table 2

[0153]

[0154] As can be seen from the results in Table 1, the semantic difference module (SDM) proposed in this embodiment has better generalization performance on the test set compared with other uncertain edge enhancement modules, especially for the left atrial appendage region.

[0155] Specific embodiments of the present invention have been described. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention. The preferred features described above can be used in any combination without conflict.

Claims

1. A method for joint segmentation of left atrium and left atrial appendage tissue in CT images, characterized in that, include: Construct the first-stage dataset; A first-stage neural network model N1 for joint segmentation of the left atrium and left atrial appendage was constructed. Using the first-stage dataset, train the first-stage neural network model N1; Using the first-stage dataset, test the first-stage neural network model N1; The coarse segmentation results of the first-stage neural network model N1 on the validation set are sampled to construct the second-stage dataset. Construct a second-stage neural network model N2 to optimize the coarse segmentation results; The second-stage neural network model N2 is trained using the data from the second stage. The second-stage neural network model N2 was tested using the second-stage data. The left atrium and left atrial appendage in CT images are segmented using the trained first-stage neural network model N1 and the second-stage neural network model N2. The first-stage neural network model N1 is a 3D medical image segmentation neural network model capable of multi-scale feature extraction, feature enhancement, and feature fusion. The operations in the first-stage neural network model N1 include: A hierarchical neural network model is used to extract multi-scale image features from CT images; The multi-scale image features are enhanced by using a semantic difference module designed based on diffusion theory to obtain the enhanced features. The enhanced features are then fused at multiple scales to obtain the final deep-coded features. The deep encoded features are input into a segmentation network to segment the foreground target, resulting in a coarse segmentation result; The training of the first-stage neural network model N1 includes: Using the annotation results of the first-stage dataset and the coarse segmentation results, a loss function is constructed, and the model is trained to optimize the parameters of the first-stage neural network model N1. The semantic difference module is designed based on diffusion theory and includes: The diffusion theory is described using second-order partial differential equations: ； in, Let D represent the gradient operator, and let D represent the diffusion function, which determines the diffusion rate along a specific direction; the diffusion function D is h(| |2), which has nonlinear characteristics, uses deep features G from the convolutional network as the independent variable of the diffusion function. A semantic guidance graph representing the difference operator, where h represents the convolution-based mapping function; Solving the second-order partial differential equation yields an approximate numerical solution: ； ； Where p represents a specific pixel in the feature map. This represents a small area surrounding pixel p. and These are weighting coefficients. Characterizes the original features The value of the difference feature map at point p. Semantic difference map for deep features; Indicates the enhanced features, Indicates original features, This represents the edge features enhanced by the semantic difference module; In the convolution-based mapping function h, a learnable edge operator is proposed, allowing the convolution kernel to have different values ​​at various positions while ensuring that the kernel has differential properties. The value at the exact center of the kernel is set to 1, resulting in the final enhanced edge features. , ； in and Let F and G represent the learnable edge operators corresponding to feature F and deep semantic feature G, respectively. Represents the convolution kernel; The training process of the second-stage neural network model N2 includes: The coarse segmentation results are sampled, including areas where there is no connection between the left atrium and the left atrial appendage; The images and coarse segmentation masks corresponding to the disconnected regions are extracted and simultaneously input into the second-stage neural network model N2; Employing a connectivity loss function based on distance constraints Training is performed using the Dess loss function.

2. The method for joint segmentation of left atrium and left atrial appendage tissue in CT images according to claim 1, characterized in that, The enhanced features consist of two parts: one part is the original features from the hierarchical neural network model, and the other part is the enhanced edge features from the semantic difference module.

3. The method for joint segmentation of left atrium and left atrial appendage tissue in CT images according to claim 1, characterized in that, The connectivity loss function based on distance constraints is adopted. Training using the Descein loss function includes: Calculate the connectivity loss function : The left atrial appendage region is located by extracting the point-by-point coordinates of the left atrial appendage region through the pixel label information of the segmented truth mask; Find the approximately smallest circumscribed hexahedron containing the left atrial appendage region, and determine the four vertices of the hexahedron that are closest to the surface of the left atrium. , i=1,2,3,4; The connectivity loss function is calculated using the minimum distances from the four nearest vertices to the surface of the left atrium. , ； in Let P represent the activation function, and let P represent the set of points on the surface of the left atrium. To represent an element in set P, , i=1,2,3,4; represent the four vertices on the circumscribed hexahedron of the left atrial appendage that are closest to the surface of the left atrium, S represents the normalization coefficient, S is set to 20, and D represents the Euclidean distance; Calculate the total loss function : ； Let be the Descein loss function; when the number of training rounds for the second-stage network model N2 exceeds a set number, Set to 1; when the number of rounds is less than the set number of times, Take 0.

4. The method for joint segmentation of left atrium and left atrial appendage tissue in CT images according to claim 3, characterized in that, When testing the second-stage neural network model N2 using the second-stage data, channel correction is performed. Specifically, the images of the intercepted disconnected regions and the coarse segmentation mask are simultaneously input into the second-stage neural network model N2, and the decoded features are obtained through the encoder-decoder structure. ; in terms of features Before selecting the maximum activation value per pixel by channel, the maximum value of each channel in the feature is first unified.

5. A joint segmentation system for left atrium and left atrial appendage tissue in CT images, used to implement the method of claim 1, characterized in that, include: The first data module constructs the first-stage dataset. The first network model module, the first neural network model N1 built in this module is used for the joint segmentation of the left atrium and left atrial appendage; The first training module uses the first-stage dataset to train the neural network model N1. The first test module uses the first-stage dataset to test the neural network model N1. The second data module samples the coarse segmentation results of the first-stage neural network model N1 to construct the second-stage dataset. The second network model module is used to build a second-stage neural network model N2 for optimizing the coarse segmentation results. The second training module uses the second-stage data to train the second-stage neural network model N2. The second test module uses the second-stage data to test the second-stage neural network model N2.

6. A terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it can be used to perform the method of any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, this program can be used to perform the method of any one of claims 1-4.

Images