Temporal bone CT automatic segmentation method based on convolutional neural network

Abstract

The invention discloses a temporal bone CT automatic segmentation method based on a convolutional neural network, and relates to the technical field of medical image processing, and the method comprises the steps: S1, data standardization; s2, a coarse segmentation stage; s3, a fine segmentation stage; and S4, a post-processing step. According to the temporal bone CT automatic segmentation method based on the convolutional neural network, through the four main steps of data standardization, coarse segmentation, fine segmentation and post-processing, rapid, accurate and automatic segmentation of the inner ear area in the temporal bone CT image can be achieved, the segmentation efficiency and accuracy are remarkably improved, and compared with the prior art, the method has the advantages that the segmentation efficiency is improved, and the segmentation efficiency is improved. According to the temporal bone CT key anatomical structure segmentation method, rapid and accurate segmentation of the temporal bone CT key anatomical structure is achieved through the two-stage convolutional neural network, the temporal bone CT key anatomical structure segmentation method has the advantages of being high in segmentation speed, precision and generalization ability and the like, important support is provided for clinical diagnosis, and the temporal bone CT key anatomical structure segmentation method has high clinical application value.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a temporal bone CT automatic segmentation method based on a convolutional neural network. Background Art

[0002] In medical imaging diagnosis, accurate segmentation of temporal bone CT images is crucial for the diagnosis and treatment of ear diseases.

[0003] At present, the segmentation methods of temporal bone CT images can be roughly divided into two categories: manual segmentation and automatic segmentation. However, in actual operation, the existing manual segmentation methods are not only time-consuming and error-prone, but the automatic segmentation methods are not accurate enough when dealing with small targets and complex backgrounds, and are slow and have limited generalization capabilities.

[0004] Therefore, it is urgent to improve this shortcoming. The present invention studies and improves the existing technology and its shortcomings, and provides a temporal bone CT automatic segmentation method based on convolutional neural network. Summary of the invention

[0005] The purpose of the present invention is to provide a temporal bone CT automatic segmentation method based on convolutional neural network to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a temporal bone CT automatic segmentation method based on convolutional neural network, comprising the following steps:

[0007] S1. Data standardization: Standardize the original data to unify the orientation, grayscale interval and voxel spacing. Data standardization can provide unified and standardized input data for the network model, thereby improving the accuracy and consistency of the segmentation results. The data standardization process specifically includes image orientation adjustment, grayscale value truncation, and resampling voxel spacing;

[0008] S2, coarse segmentation stage: Use the Attention U-Net 3D architecture to perform coarse temporal bone CT segmentation to quickly locate the inner ear area in the CT image and provide a preliminary region of interest (ROI) for the next stage of fine segmentation. Through this process, the amount of calculation is effectively reduced, and the processing speed is significantly improved, laying the foundation for subsequent analysis;

[0009] S3, fine segmentation stage: Segment the key anatomical structures in the temporal bone ROI area, and use the UNETR architecture to further refine the segmentation within the coarse segmentation area to accurately locate and extract complex anatomical structures such as the cochlea, vestibule, semicircular canals, auditory ossicles, facial nerve, round window niche and external auditory canal. These structures are crucial for clinical diagnosis, and accurate segmentation provides reliable data support for subsequent surgical planning and treatment evaluation;

[0010] S4. Post-processing steps: solve the problem of over-segmentation or under-segmentation, further optimize and improve the quality of the segmentation results to ensure that they are more suitable for clinical applications, use morphological operations (such as erosion, dilation, opening and closing operations) to improve the smoothness of the segmentation boundary and repair small defects and holes, and after binarization, perform connected domain analysis to identify and retain larger connected areas while removing small isolated areas or noise. For the segmentation of tubular structures in medical images, extract the center line of the structure through refinement processing and use the center line points for curve fitting.

[0011] Furthermore, in step S1, the data standardization process is specifically as follows:

[0012] Image orientation adjustment: adjust the image data to a unified anatomical coordinate system. The anatomical coordinate system uses three directional axes, namely, the X-axis, the Y-axis and the Z-axis, which represent the left-right, front-back and top-bottom directions of the human body, respectively. Due to different scanning angles, body positions and equipment settings of different patients, the directions of the original images may vary. By aligning the patient's image data to these anatomical axes, it is ensured that the image data of different patients have the same reference when analyzing. For example, if the LPS coordinate system is used, regardless of whether the patient has different postures or body positions during CT image acquisition, after the image is adjusted, the structures such as the cochlea and inner ear will be located in relatively consistent anatomical positions. Specific operations: use the DICOM orientation information to perform affine transformation, and the DICOM coordinate system uses the RAS (Right-Anterior-Superior) coordinate system as the standard;

[0013] Grayscale value truncation: limit the grayscale value of the CT image to between -1000 and 3000 Hounsfield Unit (HU) to improve the segmentation effect of the model. This range effectively includes the CT value distribution of the temporal bone and its surrounding tissues, helps to remove irrelevant background noise, and enhances the capture of important structures (such as the inner ear, external auditory canal, cochlea, etc.). Specific operations: use image processing software or programming libraries (such as Python's SimpleITK, Nibabel, etc.) to read CT image data, and ensure that the read image data contains grayscale value information (in Hounsfield Unit (HU)). After setting the lower limit of the grayscale value to -1000HU and the upper limit to 3000HU, traverse each pixel in the image and check the grayscale value. If the grayscale value of the pixel is lower than the lower limit (-1000HU), set it to the lower limit value (-1000HU). If the grayscale value of the pixel is higher than the upper limit (3000HU), set it to the upper limit value (3000HU). Keep the pixels with grayscale values ​​within the range unchanged.

[0014] Resample voxel spacing: Unify the original CT scans with different resolutions and voxel spacing to the same voxel spacing so that the CT images can be resampled at the same voxel spacing for more refined segmentation in the convolutional neural network. Specific operations: Use linear interpolation or nearest neighbor interpolation to resample the voxel spacing of the image to 0.3mm to ensure that small and delicate anatomical structures are clearly captured by the model. This step improves the spatial resolution of the image and provides more accurate data input for subsequent deep learning tasks.

[0015] Furthermore, in step S2, the Attention U-Net 3D architecture has an encoder-decoder symmetric structure, which enables the network to extract feature information in the image layer by layer, and introduces an attention mechanism, so that the network can focus on important areas in the image, thereby improving the accuracy of segmentation and enhancing the ability to understand complex anatomical structures. In addition, Attention U-Net 3D uses skip connections to retain detailed information lost during the encoding process and effectively restores subtle features in the image in the decoding stage.

[0016] Furthermore, the Attention U-Net 3D architecture training process specifically includes:

[0017] 1) Network input processing: The temporal bone CT images are adjusted to a uniform size, such as a 160×160×64 three-dimensional volume, and data enhancement techniques such as random rotation, scaling, and flipping are introduced. By increasing the diversity of data in the training set, the risk of overfitting of the model in the test phase is effectively avoided, further improving the generalization ability of the network;

[0018] 2) Loss function selection: Considering the common category imbalance problem in segmentation tasks, Dice Loss is selected as the loss function. Dice Loss accurately measures the model performance by calculating the overlap between the segmentation results predicted by the model and the true annotations, thereby improving the segmentation accuracy of small volume structures (such as the inner ear). Its core idea is to optimize the performance of the model when facing unbalanced data through similarity calculation;

[0019] 3) Optimizer and parameter adjustment: Select the Adam optimizer and use the adaptive learning rate feature of the Adam optimizer to speed up the convergence of the network and reduce the parameter tuning time during model training. At the same time, appropriate learning rate settings (such as 1e-4) and the application of weight decay parameters can help control the overfitting problem of the model and ensure the stability of the training process.

[0020] Furthermore, in step S3, the UNETR architecture combines the advantages of Transformers and convolutional neural networks (Transformers are good at processing long-distance dependencies and capturing global information, while CNNs are good at locating and retaining high-resolution features), captures image details and spatial context information through a self-attention mechanism to process the spatial structure of the inner ear, and introduces depthwise separable convolutional layers and dilated convolutions to reduce the amount of computation and model parameters while maintaining good performance. At the same time, an attention gating mechanism is introduced into the encoder-decoder structure of the Transformer (the encoder is responsible for mapping the input sequence into a continuous representation, and the decoder gradually generates an output sequence based on the output of the encoder) to dynamically adjust the importance of different features to ensure high resolution and accuracy within a small region of interest (ROI).

[0021] Furthermore, the depthwise separable convolutional layer decomposes the standard convolution into depthwise convolution and pointwise convolution, as follows:

[0022] Deep convolution: Use different convolution kernels to perform convolution on each input channel. This operation decouples channel correlation and spatial correlation, making the convolution operations on channel relationship and planar spatial relationship independent of each other, thereby reducing the computational cost;

[0023] Point-by-point convolution: Perform point-by-point (i.e., 1x1 convolution) operations on the output of the depthwise convolution to combine the feature information of different channels, which is equivalent to weighted summation of the output of the depthwise convolution to generate a new feature map.

[0024] Furthermore, the dilated convolution increases the actual size of the convolution kernel by inserting holes (i.e., zero values) into the convolution kernel, thereby expanding the receptive field (the size of the area that the convolution kernel can cover on the input feature map) without increasing the amount of computation, so that the model can capture contextual information in a larger range without losing the spatial information of the input feature map.

[0025] Furthermore, the attention gating mechanism calculates the importance weight of each position in the input sequence and dynamically adjusts the feature representation accordingly, so that the model pays more attention to local details that are crucial to the segmentation task when processing complex anatomical structures, thereby enhancing the model's ability to capture key information. The implementation method of introducing the attention gating mechanism in the encoder-decoder structure of the Transformer is as follows:

[0026] Design attention gating layer: Introduce attention gating layer in each sublayer of encoder and decoder to calculate the importance weight of each position in the input sequence;

[0027] Combined with multi-head attention mechanism: The multi-head attention mechanism is used to capture the dependencies between different positions in the input sequence, and combined with the weights of the output of the attention gating layer for weighted summation to obtain a more accurate feature representation;

[0028] Training and optimization: The model parameters are optimized through the training process so that the attention gating mechanism can accurately identify key features and dynamically adjust the feature importance.

[0029] Furthermore, the UNETR network training process specifically includes:

[0030] 1) Network input processing: The inner ear region obtained by the coarse segmentation network is used as the ROI for processing, and a sliding window processing strategy is introduced to decompose the larger image block into several overlapping small blocks (96×96×64), so as to ensure that no details are lost when processing small anatomical structures (such as the cochlea and facial nerve). At the same time, data enhancement such as random rotation, scaling and flipping of the ROI area is performed to increase data diversity and enhance the generalization ability of the model;

[0031] 2) Loss function selection: DiceFocalLoss is selected as the loss function. DiceFocalLoss combines the advantages of Dice Loss and Focal Loss, and can effectively deal with the segmentation problem of small volume targets and category imbalance, focusing on improving the prediction accuracy of difficult-to-segment areas (such as facial nerve and auditory ossicles) and enhancing the robustness of segmentation;

[0032] 3) Optimizer adjustment: Dynamic adjustment is performed according to the training situation. Specifically, a higher learning rate is used in the early stage to accelerate convergence, and the learning rate is gradually reduced in the later stage to avoid overfitting and stabilize performance. Optimization techniques such as learning rate scheduling and weight decay help the model converge to the global optimal state and ensure the accuracy and consistency of the segmentation results.

[0033] Furthermore, in step S4, the morphological operation helps optimize the boundary shape by defining the relationship between pixels, making the segmentation result clearer, and specifically includes: erosion, expansion, opening operation and closing operation;

[0034] Erosion: Eliminate boundary points and shrink the boundary inward. It is used to eliminate small, meaningless objects or disconnect objects. The erosion operation selects a suitable structural element, traverses the image, and compares the structural element with each position in the image. If the structural element is completely within the image, the pixel value at that position is retained, otherwise the pixel value at that position is set to the background value.

[0035] Dilation: merge all background points that are in contact with the object into the object, so that the boundary expands outward, which is used to fill holes or narrow parts in the object and connect adjacent objects. The dilation operation also requires selecting a structural element and traversing the image. If the structural element intersects with a certain position in the image, the pixel value at that position is set as the foreground value.

[0036] Opening operation: Erosion is performed first and then dilation is performed to eliminate small noise points or isolated pixels in the image, while smoothing the edges of objects and retaining the overall structure of the objects. The opening operation can remove objects smaller than the structural elements while keeping larger objects unaffected.

[0037] Closing operation: dilation is performed first and then erosion is performed to fill small holes or broken parts in the image, connect tight but discontinuous object parts, and smooth the boundaries of objects. The closing operation can fill holes or breaks that are smaller than structural elements while keeping the overall shape of the object unaffected. For example, the use of the closing operation can effectively fill small holes in the segmented area to ensure the integrity of the segmentation result.

[0038] The present invention provides a temporal bone CT automatic segmentation method based on convolutional neural network, which has the following advantages:

[0039] Beneficial effects:

[0040] The present invention first ensures the normalization of input data by adjusting the image orientation, truncating the grayscale value, and resampling the voxel spacing. Then, the Attention U-Net 3D network is used for coarse segmentation. After quickly locating the inner ear area, the UNETR network is used to further finely segment the cochlea and other small structures to ensure the accuracy and resolution of the segmentation. Finally, the segmentation results are optimized, noise is removed, and the boundaries are repaired through morphological operations and connected domain analysis. Through the four main steps of data standardization, coarse segmentation, fine segmentation, and post-processing, this method can achieve fast, accurate, and automatic segmentation of the inner ear area in temporal bone CT images, significantly improving the efficiency and accuracy of segmentation. Compared with the existing technology, it has the advantages of fast segmentation speed, high accuracy, and strong generalization ability, providing important support for clinical diagnosis and having high clinical application value.

[0041] The present invention realizes fast and accurate segmentation of key anatomical structures of temporal bone CT through a two-stage convolutional neural network. Specifically, in the coarse segmentation stage, the Attention U-Net 3D architecture is used to perform coarse segmentation of temporal bone CT, quickly locate the inner ear area in the CT image, and provide a preliminary ROI area for the fine segmentation in the next stage, thereby reducing the amount of calculation; while in the fine segmentation stage, the UNETR architecture is used, which not only combines the advantages of Transformers and convolutional neural networks, but also introduces deep separable convolutional layers and dilated convolutions to enhance the model's ability to capture local details. In combination with the attention gating mechanism introduced in the encoder-decoder structure of the Transformer, the importance of different features can be dynamically adjusted, further enhancing the model's ability to capture local details and improving cutting accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of the automatic segmentation process of a temporal bone CT automatic segmentation method based on a convolutional neural network of the present invention;

[0043] Figure 2 The image is a normalized original image of an automatic temporal bone CT segmentation method based on a convolutional neural network according to the present invention;

[0044] Figure 3 It is an image normalized original histogram of a temporal bone CT automatic segmentation method based on a convolutional neural network of the present invention;

[0045] Figure 4 A standardized processed image of a temporal bone CT automatic segmentation method based on a convolutional neural network according to the present invention;

[0046] Figure 5 A standardized processing histogram of a temporal bone CT automatic segmentation method based on a convolutional neural network according to the present invention;

[0047] Figure 6 This is a rough segmentation result diagram of the inner ear of a temporal bone CT automatic segmentation method based on a convolutional neural network according to the present invention;

[0048] Figure 7 This is a graph showing the inner ear ROI extraction result of a temporal bone CT automatic segmentation method based on a convolutional neural network according to the present invention;

[0049] Figure 8 A temporal bone CT fine segmentation image of a temporal bone CT automatic segmentation method based on a convolutional neural network according to the present invention;

[0050] Fig. 9 This is a cochlear round window segmentation image of a temporal bone CT automatic segmentation method based on a convolutional neural network in the present invention. DETAILED DESCRIPTION

[0051] The following embodiments of the present invention are described in further detail in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0052] like Figure 1-Figure 9 As shown, a temporal bone CT automatic segmentation method based on convolutional neural network includes the following steps:

[0053] S1. Data standardization: Standardize the original data to unify the orientation, grayscale interval and voxel spacing. Data standardization can provide unified and standardized input data for the network model, thereby improving the accuracy and consistency of the segmentation results. The data standardization process specifically includes image orientation adjustment, grayscale value truncation, and resampling voxel spacing;

[0054] Image orientation adjustment: adjust the image data to a unified anatomical coordinate system. Due to different scanning angles, body positions, and equipment settings of different patients, the orientation of the original image may be different. By aligning the patient's image data to these anatomical axes, it is ensured that the image data of different patients have the same reference when analyzing. For example, if the LPS coordinate system is used, regardless of whether the patient has different postures or body positions during CT image acquisition, after the image is adjusted, the cochlea, inner ear and other structures will be located in relatively consistent anatomical positions. Specific operation: use DICOM orientation information to perform affine transformation, and the DICOM coordinate system uses the RAS (Right-Anterior-Superior) coordinate system as the standard. Specifically, first read the orientation information of the DICOM file, which is stored in the Image Orientation (Patient) and Image Position (Patient) fields, and then according to the target coordinate system (such as RAS), reverse or adjust the coordinate axes in the orientation matrix (for example, reverse the X and Y axes when LPS is converted to RAS), and finally set the adjusted orientation matrix back to the image;

[0055] Grayscale value truncation: The grayscale value of the CT image is limited to between -1000 and 3000 Hounsfield Unit (HU) to improve the segmentation effect of the model. This range effectively includes the CT value distribution of the temporal bone and its surrounding tissues, helps to remove irrelevant background noise, and enhances the capture of important structures (such as the inner ear, external auditory canal, cochlea, etc.). Figure 4-Figure 5,Specific operations: Use image processing software or programming libraries (such as Python's SimpleITK, Nibabel, etc.) to read CT image data, and ensure that the read image data contains gray value information (in Hounsfield Unit (HU)). After setting the lower limit of the gray value to -1000HU and the upper limit to 3000HU, traverse each pixel in the image and check the gray value. If the gray value of the pixel is lower than the lower limit (-1000HU), set it to the lower limit value (-1000HU). If the gray value of the pixel is higher than the upper limit (3000HU), set it to the upper limit value (3000HU). Keep the pixels with gray values ​​within the range unchanged;

[0056] Resample voxel spacing: Unify the original CT scans with different resolutions and voxel spacing to the same voxel spacing so that the CT images can be resampled at the same voxel spacing for more refined segmentation in the convolutional neural network. Specific operations: Use linear interpolation or nearest neighbor interpolation to resample the voxel spacing of the image to 0.3mm to ensure that small and delicate anatomical structures are clearly captured by the model. This step improves the spatial resolution of the image and provides more accurate data input for subsequent deep learning tasks;

[0057] S2, coarse segmentation stage: Use the Attention U-Net 3D architecture to perform coarse temporal bone CT segmentation to quickly locate the inner ear area in the CT image and provide a preliminary region of interest (ROI) for the next stage of fine segmentation. Through this process, the amount of calculation is effectively reduced, and the processing speed is significantly improved, laying the foundation for subsequent analysis;

[0058] In this embodiment, the Attention U-Net 3D architecture has an encoder-decoder symmetric structure, which enables the network to extract feature information in the image layer by layer, and introduces an attention mechanism, so that the network can focus on important areas in the image, thereby improving the accuracy of segmentation and enhancing the ability to understand complex anatomical structures. The Attention U-Net 3D uses skip connections to retain the detailed information lost in the encoding process and effectively restore the subtle features in the image in the decoding stage. The Attention U-Net 3D architecture training process specifically includes:

[0059] 1) Network input processing: The temporal bone CT images are adjusted to a uniform size, such as a 160×160×64 three-dimensional volume, and data enhancement techniques such as random rotation, scaling, and flipping are introduced. By increasing the diversity of data in the training set, the risk of overfitting of the model in the test phase is effectively avoided, further improving the generalization ability of the network;

[0060] 2) Loss function selection: Considering the common category imbalance problem in segmentation tasks, Dice Loss is selected as the loss function. Dice Loss accurately measures the model performance by calculating the overlap between the segmentation results predicted by the model and the true annotations, thereby improving the segmentation accuracy of small volume structures (such as the inner ear). Its core idea is to optimize the performance of the model when facing unbalanced data through similarity calculation;

[0061] 3) Optimizer and parameter adjustment: Select the Adam optimizer and use the adaptive learning rate feature of the Adam optimizer to speed up the convergence of the network and reduce the parameter tuning time during model training. At the same time, the appropriate learning rate setting (such as 1e-4) and the application of weight decay parameters help control the overfitting problem of the model and ensure the stability of the training process;

[0062] In this embodiment, after completing the rough segmentation of the temporal bone CT, the system will calculate the geometric center of the area, and this center point will be used to define the region of interest (ROI) in the fine segmentation stage, thereby further improving the accuracy and computational efficiency of the next step of segmentation. Through these steps, the temporal bone CT rough segmentation network can provide a solid foundation for subsequent image processing, and the effect is shown in FIG. Figure 6-Figure 7 ;

[0063] S3, fine segmentation stage: Segment the key anatomical structures in the temporal bone ROI area, and use the UNETR architecture to further refine the segmentation within the coarse segmentation area to accurately locate and extract complex anatomical structures such as the cochlea, vestibule, semicircular canals, auditory ossicles, facial nerve, round window niche and external auditory canal. These structures are crucial for clinical diagnosis, and accurate segmentation provides reliable data support for subsequent surgical planning and treatment evaluation;

[0064] In this embodiment, the UNETR architecture combines the advantages of Transformers and convolutional neural networks (Transformers are good at processing long-distance dependencies and capturing global information; while CNNs are good at locating and retaining high-resolution features), captures image details and spatial context information through a self-attention mechanism to process the spatial structure of the inner ear, and introduces depthwise separable convolutional layers and dilated convolutions to reduce the amount of computation and model parameters while maintaining good performance. At the same time, an attention gating mechanism is introduced into the encoder-decoder structure of the Transformer (the encoder is responsible for mapping the input sequence into a continuous representation, and the decoder gradually generates an output sequence based on the output of the encoder) to dynamically adjust the importance of different features to ensure high resolution and accuracy in a small region of interest (ROI), and the fine segmentation results are shown in Figure 2. Figure 8 ; The UNETR network training process specifically includes:

[0065] 1) Network input processing: The inner ear region obtained by the coarse segmentation network is used as the ROI for processing, and a sliding window processing strategy is introduced to decompose the larger image block into several overlapping small blocks (96×96×64), so as to ensure that no details are lost when processing small anatomical structures (such as the cochlea and facial nerve). At the same time, data enhancement such as random rotation, scaling and flipping of the ROI area is performed to increase data diversity and enhance the generalization ability of the model;

[0066] 2) Loss function selection: DiceFocalLoss is selected as the loss function. DiceFocalLoss combines the advantages of Dice Loss and Focal Loss, and can effectively deal with the segmentation problem of small volume targets and category imbalance, focusing on improving the prediction accuracy of difficult-to-segment areas (such as facial nerve and auditory ossicles) and enhancing the robustness of segmentation;

[0067] 3) Optimizer adjustment: Dynamic adjustment is performed according to the training situation. Specifically, a higher learning rate is used in the early stage to accelerate convergence, and the learning rate is gradually reduced in the later stage to avoid overfitting and stabilize performance. Optimization techniques such as learning rate scheduling and weight decay help the model converge to the global optimal state, ensuring the accuracy and consistency of the segmentation results;

[0068] In this embodiment, the depthwise separable convolution layer decomposes the standard convolution into depthwise convolution and pointwise convolution, as follows:

[0069] Deep convolution: Use different convolution kernels to perform convolution on each input channel. This operation decouples channel correlation and spatial correlation, making the convolution operations on channel relationship and planar spatial relationship independent of each other, thereby reducing the computational cost;

[0070] Point-by-point convolution: Perform point-by-point (i.e., 1x1 convolution) operations on the output of the depth convolution to combine the feature information of different channels, which is equivalent to weighted summation of the output of the depth convolution to generate a new feature map;

[0071] In this embodiment, the dilated convolution increases the actual size of the convolution kernel by inserting holes (i.e., zero values) into the convolution kernel, thereby expanding the receptive field (the size of the area that the convolution kernel can cover on the input feature map) without increasing the amount of computation, so that the model can capture contextual information in a larger range without losing the spatial information of the input feature map; in the UNETR architecture, the application of dilated convolution helps the model reduce the amount of computation and memory consumption while maintaining a high segmentation accuracy;

[0072] In this embodiment, the attention gating mechanism calculates the importance weight of each position in the input sequence and dynamically adjusts the feature representation accordingly, so that the model pays more attention to local details that are critical to the segmentation task when processing complex anatomical structures, thereby enhancing the model's ability to capture key information. The implementation method of introducing the attention gating mechanism into the encoder-decoder structure of the Transformer is as follows:

[0073] Design attention gating layer: Introduce attention gating layer in each sublayer of encoder and decoder to calculate the importance weight of each position in the input sequence;

[0074] Combined with multi-head attention mechanism: The multi-head attention mechanism is used to capture the dependencies between different positions in the input sequence, and combined with the weights of the output of the attention gating layer for weighted summation to obtain a more accurate feature representation;

[0075] Training and optimization: Optimize model parameters through the training process so that the attention gating mechanism can accurately identify key features and dynamically adjust feature importance;

[0076] The round window is an important connecting part between the inner ear and the middle ear, and its position and shape are extremely complex. In this embodiment, the anatomical feature of the inner ear surrounded by hard bones is fully utilized. By identifying and automatically segmenting the non-bone voxels adjacent to the inner ear, the accurate segmentation of the round window is successfully achieved, and satisfactory results are produced. The specific method is: first, the inner ear structure mask image is binary expanded to expand the area of ​​the inner ear; then, the original inner ear mask is subtracted from the expanded inner ear mask, and the voxels with HU values ​​less than -500 in the area are extracted. These voxels are defined as the round window area; the schematic diagram of round window segmentation is attached Fig. 9 , where green represents the extended inner ear area and red represents the empty round window area.

[0077] S4, post-processing steps: solve the problem of over-segmentation or under-segmentation, further optimize and improve the quality of the segmentation results to ensure that they are more suitable for clinical applications, use morphological operations (such as corrosion, expansion, opening and closing operations) to improve the smoothness of the segmentation boundary and repair small defects and holes, and after binarization, perform connected domain analysis to identify and retain larger connected areas while removing small isolated areas or noise. For medical image segmentation of tubular structures, extract the center line of the structure through thinning processing and use the center line points for curve fitting;

[0078] In this embodiment, the morphological operation helps optimize the boundary shape by defining the relationship between pixels, making the segmentation result clearer, and specifically includes: erosion, expansion, opening operation and closing operation;

[0079] Erosion: eliminate boundary points and shrink the boundary inwards, which is used to eliminate small and meaningless objects or disconnect the connection between objects. In this embodiment, the erosion operation selects a suitable structural element, traverses the image, and compares the structural element with each position in the image. If the structural element is completely within the image, the pixel value of the position is retained, otherwise the pixel value of the position is set to the background value.

[0080] Dilation: merge all background points that are in contact with the object into the object, so that the boundary expands outward, which is used to fill holes or narrow parts in the object and connect adjacent objects. In this embodiment, the dilation operation also needs to select a structural element and traverse the image. If the structural element intersects with a certain position in the image, the pixel value of the position is set as the foreground value.

[0081] Opening operation: Erosion is performed first and then dilation is performed to eliminate small noise points or isolated pixels in the image, while smoothing the edges of objects and retaining the overall structure of the objects. The opening operation can remove objects smaller than the structural elements while keeping larger objects unaffected.

[0082] Closing operation: dilation is performed first and then erosion is performed to fill small holes or broken parts in the image, connect the parts of objects that are tightly but discontinuous, and smooth the boundaries of objects. Closing operation can fill holes or breaks that are smaller than structural elements while keeping the overall shape of the object unaffected. For example, using closing operation can effectively fill small holes in the segmented area to ensure the integrity of the segmentation result.

[0083] In this embodiment, the connected domain analysis ensures that only anatomical structures with biological significance are retained by finding continuous pixel blocks. For example, the segmented cochlear region may contain some noise or artifacts, and the connected domain processing can effectively remove these unnecessary parts, thereby improving the reliability of the segmentation result.

[0084] In this embodiment, the specific processing flow of medical image segmentation of tubular structures is as follows: first, the skeleton or center line of the structure is extracted, and then curve fitting is performed based on these points to generate a smooth path to fill in the missing parts in the segmentation result; this not only enhances the coherence of the structure, but also effectively solves the problem of under-segmentation caused by noise or fuzzy boundaries, improves the accuracy and completeness of the segmentation, and provides more reliable information for clinical diagnosis.

[0085] The embodiments of the present invention are given for the purpose of illustration and description, and are not intended to be exhaustive or to limit the invention to the disclosed forms. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present invention and to enable those of ordinary skill in the art to understand the present invention and thereby design various embodiments with various modifications suitable for specific uses.

Claims

1. A temporal bone CT automatic segmentation method based on convolutional neural network, characterized in that: The following steps are involved: S1. Data standardization: Standardize the original data to unify the orientation, grayscale interval and voxel spacing, and the data standardization process specifically includes image orientation adjustment, grayscale value truncation, and resampling voxel spacing; S2, coarse segmentation stage: Use the Attention U-Net 3D architecture to perform coarse temporal bone CT segmentation to quickly locate the inner ear area in the CT image and provide a preliminary region of interest for the next stage of fine segmentation; S3, fine segmentation stage: using the UNETR architecture, further refine the segmentation within the coarse segmentation area to accurately locate and extract complex anatomical structures; S4. Post-processing steps: Morphological operations are used to improve the smoothness of the segmentation boundary and repair small defects and holes. After binarization, connected domain analysis is performed to identify and retain larger connected areas. For medical image segmentation of tubular structures, the center line of the structure is extracted through refinement processing and the center line points are used for curve fitting.

2. The method for automatic temporal bone CT segmentation based on convolutional neural network according to claim 1, characterized in that: In step S1, the data standardization process is specifically as follows: Image orientation adjustment: adjust the image data to a unified anatomical coordinate system. Specific operation: use DICOM orientation information to perform affine transformation, and the DICOM coordinate system uses the RAS coordinate system as the standard; Grayscale value truncation: limit the grayscale value of the CT image to between -1000 and 3000HU. Specific operation: use image processing software or programming library to read CT image data, and ensure that the read image data contains grayscale value information. After setting the lower limit of the grayscale value to -1000HU and the upper limit to 3000HU, traverse each pixel in the image and check the grayscale value. If the grayscale value of the pixel is lower than the lower limit, set it to the lower limit value. If the grayscale value of the pixel is higher than the upper limit, set it to the upper limit value. Keep the pixels with grayscale values ​​within the range unchanged. Resample voxel spacing: Unify the original CT scans with different resolutions and voxel spacing to the same voxel spacing. The specific operation is to resample the voxel spacing of the image to 0.3mm using linear interpolation or nearest neighbor interpolation.

3. The method for automatic temporal bone CT segmentation based on convolutional neural network according to claim 1, characterized in that: In step S2, the Attention U-Net 3D architecture has an encoder-decoder symmetric structure and introduces an attention mechanism. The Attention U-Net 3D retains the detail information lost in the encoding process through skip connections and effectively restores the subtle features in the image in the decoding stage.

4. The temporal bone CT automatic segmentation method based on convolutional neural network according to claim 3 is characterized in that: The Attention U-Net 3D architecture training process specifically includes: 1) Network input processing: The temporal bone CT images are adjusted to a uniform size, and data augmentation technology is introduced to effectively avoid the risk of overfitting of the model in the testing phase by increasing the diversity of data in the training set, further improving the generalization ability of the network; 2) Loss function selection: Dice Loss is selected as the loss function. Dice Loss accurately measures the model performance by calculating the overlap between the segmentation results predicted by the model and the true annotations, thereby improving the segmentation accuracy of small volume structures. 3) Optimizer and parameter adjustment: Select the Adam optimizer and use the adaptive learning rate feature of the Adam optimizer to speed up the convergence of the network and reduce the parameter tuning time during model training.

5. The temporal bone CT automatic segmentation method based on convolutional neural network according to claim 1, characterized in that: In step S3, the UNETR architecture combines Transformers and convolutional neural networks, captures image details and spatial context information through a self-attention mechanism to process the spatial structure of the inner ear, and introduces depthwise separable convolutional layers and dilated convolutions to reduce the amount of computation and model parameters. At the same time, an attention gating mechanism is introduced in the encoder-decoder structure of the Transformer to dynamically adjust the importance of different features.

6. The method for automatic temporal bone CT segmentation based on convolutional neural network according to claim 5, characterized in that: The depthwise separable convolutional layer decomposes the standard convolution into depthwise convolution and pointwise convolution as follows: Depth convolution: Use different convolution kernels for convolution on each input channel; Point-wise convolution: The output of the depthwise convolution is operated point-by-point to combine feature information from different channels.

7. The method for automatic temporal bone CT segmentation based on convolutional neural network according to claim 5, characterized in that: The dilated convolution increases the actual size of the convolution kernel by inserting holes into the convolution kernel, thereby expanding the receptive field without increasing the amount of computation, so that the model can capture contextual information in a larger range without losing the spatial information of the input feature map.

8. The method for automatic temporal bone CT segmentation based on convolutional neural network according to claim 5, characterized in that: The attention gating mechanism calculates the importance weight of each position in the input sequence and dynamically adjusts the feature representation accordingly. The implementation method of introducing the attention gating mechanism into the encoder-decoder structure of the Transformer is as follows: Design attention gating layer: Introduce attention gating layer in each sublayer of encoder and decoder to calculate the importance weight of each position in the input sequence; Combined with multi-head attention mechanism: The multi-head attention mechanism is used to capture the dependencies between different positions in the input sequence, and combined with the weights of the output of the attention gating layer for weighted summation to obtain a more accurate feature representation; Training and optimization: The model parameters are optimized through the training process so that the attention gating mechanism can accurately identify key features and dynamically adjust the feature importance.

Images