Facial nerve and drumhead nerve automatic segmentation method based on temporal bone CT

Abstract

The invention relates to a temporal bone CT-based facial nerve and drumhead nerve automatic segmentation method, and belongs to the technical field of nerve segmentation, and the method comprises the steps: obtaining a temporal bone CT image, inputting the temporal bone CT image to a layer of encoder, and changing the channel number of the temporal bone CT image into 32; the signals are sequentially input into a second layer encoder, a third layer encoder and a fourth layer encoder for processing; sequentially inputting the output of the fourth-layer encoder into the fifth-layer encoder, the sixth-layer encoder and the last-layer encoder to be processed, wherein the fifth-layer encoder, the sixth-layer encoder and the last-layer encoder are added with fast discrete wavelet transform convolution fusion modules; key features of the temporal bone CT image are extracted; the outputs of the second-layer encoder, the third-layer encoder, the fourth-layer encoder, the fifth-layer encoder and the sixth-layer encoder are respectively subjected to jump connection processing through the high-frequency attention guidance module and then are input into the second-layer decoder, the third-layer decoder, the fourth-layer decoder, the fifth-layer decoder and the sixth-layer decoder, and the key features of the last-layer encoder are input into the last-layer decoder; and the decoders of the last layer, the sixth layer, the fifth layer, the fourth layer, the third layer, the second layer and the first layer sequentially perform up-sampling to realize feature recovery and decoding, and facial nerve and drumhead nerve segmentation prediction results are generated.

Description

Technical Field

[0001] The present invention belongs to the technical field of nerve segmentation, and in particular relates to an automatic segmentation method of facial nerve and chorda tympani nerve based on temporal bone CT. Background Art

[0002] The facial nerve (FN) is one of the most important cranial nerves in the human body. The chorda tympani nerve (CTN) is an important branch of the facial nerve in the temporal bone. It is responsible for the taste of the anterior two-thirds of the tongue and controls the secretion of the submandibular and sublingual glands. Its course in the temporal bone is extremely complex and its anatomical structure varies greatly.

[0003] In otology and skull base surgery, such as cochlear implants, radical mastoidectomy, ossicular chain reconstruction, middle ear cholesteatoma resection, and facial schwannoma resection, precise positioning and protection of the facial nerve and chorda tympani nerve are key to surgical success, as well as key difficulties and risks. Intraoperative damage to the facial nerve can lead to serious complications such as facial paralysis and hemifacial spasm; damage to the chorda tympani nerve can cause persistent taste disorders and dry mouth, severely impacting the patient's quality of life. Therefore, accurate preoperative identification and three-dimensional reconstruction of the course and adjacent relationship of the facial nerve and chorda tympani nerve using imaging methods is of vital clinical significance for developing personalized surgical plans, predicting surgical risks, shortening operative time, and protecting nerve function.

[0004] High-resolution temporal bone CT (HRCT) is currently the preferred imaging method for clinically assessing the fine anatomy of the temporal bone. However, accurately tracing these neural structures across hundreds of CT slices presents significant challenges: 1. The anatomical structures are complex and tiny: the diameter of the facial nerve canal is only approximately 1-2 mm, and the chorda tympani nerve is even thinner. Surrounded by the dense temporal bone, they have low contrast with surrounding structures such as the auditory ossicles, blood vessels, and air cells, resulting in blurred boundaries. Their anatomical morphology and positional variability are particularly pronounced in the presence of pathological conditions (such as inflammation, tumors, and trauma), making identification extremely challenging. 2. Limitations of manual segmentation: Currently, the gold standard in clinical practice still relies on manual segmentation by experienced radiologists or otologists. This method has the following inherent drawbacks: It is time-consuming and labor-intensive: completing a complete three-dimensional segmentation of the facial nerve and chorda tympani nerve in the temporal bone typically requires several hours, resulting in low efficiency and insufficient clinical application. High subjectivity and poor repeatability: The segmentation results are highly dependent on the doctor's professional knowledge, experience and subjective judgment. There are significant differences in the segmentation results between different observers and even the same observer at different times, lacking objectivity and consistency. High technical threshold: The operator is required to have an extremely in-depth understanding of the complex three-dimensional anatomy of the temporal bone. The training cost is high and it is difficult to popularize in primary hospitals. In order to overcome the shortcomings of manual segmentation, researchers have developed a variety of semi-automatic or fully automatic segmentation algorithms. Early semi-automatic methods, such as those based on region growing, dynamic contour models, and level sets, although they can improve some efficiency, still require manual interaction (such as selecting seed points and setting initial contours). The segmentation results are still affected by the operator's subjectivity and are prone to failure where the neural boundaries are blurred or interrupted.

[0005] In recent years, with the development of artificial intelligence, deep learning-based automatic segmentation techniques, particularly convolutional neural networks (CNNs) such as the U-Net architecture, have made breakthrough progress in the field of medical image segmentation. Several studies, both domestic and international, have attempted to apply these techniques to the automatic segmentation of the temporal facial nerve.

[0006] However, the current approach still has the following urgent issues and significant shortcomings: 1. Ignoring key nerve branches: Most existing automatic segmentation algorithms focus solely on the relatively large trunk of the facial nerve, completely ignoring the equally functionally important chorda tympani nerve. Because the chorda tympani nerve is much smaller on CT images, its signal characteristics are extremely weak, and it is almost indistinguishable from surrounding tissue. Its automatic segmentation is considered an extremely challenging task in this field. The shortcomings of existing technologies prevent clinicians from obtaining precise preoperative information on the location and variation of the chorda tympani nerve, leaving a blind spot for surgical risk. 2. Insufficient segmentation accuracy for small targets: The facial nerve itself is a typical "small target" segmentation task. Standard deep learning networks are prone to segmentation discontinuities, blurred edges, and loss of detail when processing such slender, tortuous, and easily interrupted structures. In particular, in areas of high anatomical variation such as the geniculate ganglion, labyrinthine segment, and vertical segment of the facial nerve, the robustness and accuracy of existing models often fail to meet clinical requirements. 3. Limited model generalization and robustness: Many research results are trained on highly homogeneous datasets from a single center and a single device. This results in significant performance degradation when applied to external data from different hospitals and with varying CT scan parameters (slice thickness, resolution, and noise level). Furthermore, when faced with cases characterized by significant anatomical changes due to pathological conditions such as temporal bone fractures, cholesteatoma erosion, and tumor compression, the segmentation accuracy of existing models can drop dramatically or even fail completely.

[0007] In summary, there is an urgent need for an algorithm that can simultaneously, fully automatically, and accurately segment the facial nerve and its important branch, the chorda tympani nerve, within the temporal bone. This algorithm must not only overcome the technical challenges of the tiny nerve structure and low contrast, but also address individual anatomical variations and pathological conditions. This algorithm can provide clinicians with comprehensive, reliable, and efficient preoperative 3D navigation information, truly improving the precision and safety of otologic and skull base surgeries. Summary of the Invention

[0008] In view of the above-mentioned deficiencies in the existing technology, the purpose of the invention is to provide an automatic segmentation method for the facial nerve and chorda tympani nerve based on temporal bone CT, which not only overcomes the technical difficulties of tiny nerve structure and low contrast, but also can cope with the challenges of individual anatomical variations and pathological conditions, thereby providing clinicians with comprehensive, reliable and efficient preoperative three-dimensional navigation information, and truly improving the accuracy and safety of otolaryngology and skull base surgery.

[0009] The present invention proposes an automatic segmentation method for the facial nerve and chorda tympani nerve based on temporal bone CT. The method is applied to a model consisting of a multi-layer encoder, a multi-layer decoder, a fast discrete wavelet transform convolution fusion module, a wavelet transform-based Mamba module, and a high-frequency attention guidance module with skip connections. The method includes:

[0010] Acquire a temporal bone CT image, input the temporal bone CT image into a first-layer encoder to downsample the temporal bone CT image so that the number of channels of the temporal bone CT image is changed to 32;

[0011] The temporal bone CT image with 32 channels is sequentially input into the second, third and fourth encoder layers for downsampling to achieve dimensionality reduction and temporal bone CT image feature extraction.

[0012] The temporal bone CT image features output by the fourth encoder are sequentially input into the fifth encoder, the sixth encoder, and the last encoder with a fast discrete wavelet transform convolution fusion module for downsampling to achieve dimensionality reduction and enhance the temporal bone CT image features;

[0013] The enhanced temporal bone CT image features in the last encoder layer are input into the Mamba module based on wavelet transform to extract the key features of the temporal bone CT image;

[0014] The temporal bone CT image features extracted from the second, third, and fourth encoder layers, as well as the temporal bone CT image features enhanced by the fifth and sixth encoder layers, are processed by the high-frequency attention guidance module through skip connections and then input into the second, third, fourth, fifth, and sixth decoders, respectively.

[0015] The temporal bone CT image features enhanced by the last encoder layer and the key features of the temporal bone CT image are input into the last decoder layer. The last decoder layer, the sixth decoder layer, the fifth decoder layer, the fourth decoder layer, the third decoder layer, the second decoder layer and the first decoder layer are upsampled in sequence to achieve feature recovery and decoding;

[0016] The output of the first layer decoder is binarized to generate the facial nerve and chorda tympani nerve segmentation prediction results.

[0017] Furthermore, in the above-mentioned method for automatic segmentation of the facial nerve and chorda tympani nerve based on temporal bone CT, the temporal bone CT image with 32 channels is sequentially input into the second encoder, the third encoder, and the fourth encoder for downsampling to achieve dimensionality reduction and extract temporal bone CT image features, including:

[0018] The second-layer encoder doubles the number of channels through convolutional normalization activation operations, and the doubled number of channels is less than the preset value;

[0019] The third layer encoder extracts CT image features through convolutional normalization activation operations;

[0020] The fourth layer encoder improves model performance through residual connections.

[0021] Furthermore, in the above-mentioned method for automatic segmentation of the facial nerve and chorda tympani nerve based on temporal bone CT, the temporal bone CT image features output by the fourth-layer encoder are sequentially input into the fifth-layer encoder, the sixth-layer encoder, and the last-layer encoder to which a fast discrete wavelet transform convolution fusion module is added for downsampling to achieve dimensionality reduction and enhance the temporal bone CT image features, including:

[0022] Step 1: Divide the temporal bone CT image features into two parts with the same batch, channel, height and width. The first part stores local features, and the second part stores global features.

[0023] Step 2: Perform 3*3 convolution on local features and global features to extract local features, perform 3*3 wavelet convolution on global features, and then use frequency space blocks to extract global features.

[0024] Step 3: The global features and local features are fused through the spatial attention module, and the global features and local features are processed through batch normalization and activation function;

[0025] Step 4: Concatenate the global features and local features along the channel direction and pass them through the Sigmoid function;

[0026] Step 5: Add the residual of the output of the Sigmoid function to the temporal bone CT image features output by the fourth encoder to obtain the output of the fifth encoder;

[0027] The output of the fifth encoder is downsampled and input into the sixth encoder layer, and the above five steps are repeated to obtain the output of the sixth encoder layer;

[0028] The output of the sixth encoder is downsampled and input into the last encoder, and the above five steps are repeated to obtain the output of the last encoder.

[0029] Furthermore, in the above-mentioned automatic segmentation method of facial nerve and chorda tympani nerve based on temporal bone CT, the enhanced temporal bone CT image features in the last layer of encoder are input into the Mamba module based on wavelet transform to extract the key features of the temporal bone CT image, including:

[0030] The enhanced temporal bone CT image feature layer in the latter encoder is normalized and then separated into high-frequency-high-frequency component, high-frequency-low-frequency component, low-frequency-high-frequency component and low-frequency-low-frequency component through discrete wavelet transform;

[0031] After ordinary convolution, activation function, convolution, and layer normalization, the high-frequency-high-frequency component, high-frequency-low-frequency component, and low-frequency-high-frequency component pass through the Mamba module based on wavelet transform to obtain long-range dependencies, enhancing the model's ability to understand global changes. The low-frequency-low-frequency component undergoes convolution to extract local features.

[0032] The high-frequency-high-frequency component, high-frequency-low-frequency component, low-frequency-high-frequency component and low-frequency-low-frequency component are spliced ​​along the 0-dimensional direction, restored to the input size after inverse wavelet transform, and finally the key features of the temporal bone CT image are extracted through residual connection.

[0033] Furthermore, in the above-mentioned method for automatic segmentation of the facial nerve and chorda tympani nerve based on temporal bone CT, the processing steps of the high-frequency attention guidance module skip connection include:

[0034] After performing discrete wavelet transform on the input features, convolution, batch normalization, activation, convolution, and batch normalization operations are performed on the high-frequency components;

[0035] The high-frequency components are processed by the convolution block attention module to optimize the context semantics, and then convolved with the high-frequency components after discrete wavelet transformation. After batch normalization, residual connection is performed;

[0036] The original input feature map is concatenated, convolved, batch normalized, and the output feature map is activated.

[0037] Furthermore, in the above-mentioned method for automatic segmentation of the facial nerve and chorda tympani nerve based on temporal bone CT, the number of channels of the second-layer decoder is consistent with that before the second-layer encoder downsampling, the number of channels of the third-layer decoder is consistent with that before the third-layer encoder downsampling, the number of channels of the fourth-layer decoder is consistent with that before the fourth-layer encoder downsampling, the number of channels of the fifth-layer decoder is consistent with that before the fifth-layer encoder downsampling, and the number of channels of the sixth-layer decoder is consistent with that before the sixth-layer encoder downsampling.

[0038] Furthermore, in the above-mentioned method for automatic segmentation of facial nerve and chorda tympani nerve based on temporal bone CT, upsampling is to increase the corresponding image to twice its original size.

[0039] Furthermore, in the above-mentioned method for automatic segmentation of facial nerve and chorda tympani nerve based on temporal bone CT, downsampling is to reduce the corresponding image to half of its original size.

[0040] Furthermore, in the above-mentioned automatic segmentation method of the facial nerve and chorda tympani nerve based on temporal bone CT, the last layer decoder, the sixth layer decoder, the fifth layer decoder, the fourth layer decoder, the third layer decoder, the second layer decoder and the first layer decoder are used to perform convolution normalization activation and residual connection operations to achieve the restoration and reconstruction of the facial nerve and chorda tympani nerve.

[0041] Furthermore, in the above-mentioned method for automatic segmentation of facial nerve and chorda tympani nerve based on temporal bone CT, the transformation for restoring the transformed image to the input size is an inverse wavelet transform.

[0042] The beneficial effects of the present invention are as follows: The present invention proposes an automatic segmentation method for facial nerve and chorda tympani nerve based on temporal bone CT, the method comprising: acquiring a temporal bone CT image, inputting the temporal bone CT image into a first-layer encoder to downsample the temporal bone CT image and change the number of channels of the temporal bone CT image to 32; inputting the temporal bone CT image with a channel number of 32 into a second-layer encoder, a third-layer encoder and a fourth-layer encoder in sequence for downsampling to achieve dimensionality reduction and extraction of temporal bone CT image features; inputting the temporal bone CT image features output by the fourth-layer encoder into a fifth-layer encoder, a sixth-layer encoder and a last-layer encoder to which a fast discrete wavelet transform convolution fusion module is added in sequence for downsampling to achieve dimensionality reduction and enhance the temporal bone CT image features; inputting the enhanced temporal bone CT image features in the last-layer encoder into a wavelet transform-based encoder. The key features of the temporal bone CT image are extracted in the Mamba module; the temporal bone CT image features extracted from the second, third, and fourth layer encoders and the temporal bone CT image features enhanced by the fifth and sixth layer encoders are respectively input into the second, third, fourth, fifth, and sixth layer decoders through the high-frequency attention guidance module for skip connection processing; the temporal bone CT image features enhanced by the last layer encoder and the key features of the temporal bone CT image are input into the last layer decoder, and the last layer decoder, sixth layer decoder, fifth layer decoder, fourth layer decoder, third layer decoder, second layer decoder, and first layer decoder are upsampled in sequence to achieve feature recovery and decoding; the output of the first layer decoder is binarized to generate facial nerve and chorda tympani nerve segmentation prediction results. The present invention not only overcomes the technical difficulties of small neural structure and low contrast, but also can cope with the challenges of individual anatomical variation and pathological state, thereby providing clinicians with comprehensive, reliable, and efficient preoperative three-dimensional navigation information, truly improving the accuracy and safety of otolaryngology and skull base surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings are only for the purpose of illustrating specific embodiments and are not to be considered as limiting the present invention. Throughout the drawings, the same reference numerals represent the same components. Obviously, the drawings described below are only some of the embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings.

[0044] Figure 1 A schematic diagram of a method for automatic segmentation of the facial nerve and chorda tympani nerve based on temporal bone CT provided by an embodiment of the present invention;

[0045] Figure 2 A schematic diagram of an automatic segmentation architecture of the facial nerve and chorda tympani nerve based on temporal bone CT provided by an embodiment of the present invention;

[0046] Figure 3 A diagram of a method for downsampling to achieve dimensionality reduction and extracting temporal bone CT image features provided by an embodiment of the present invention;

[0047] Figure 4 A diagram of a method for downsampling to achieve dimensionality reduction and enhance temporal bone CT image features provided by an embodiment of the present invention;

[0048] Figure 5 A diagram of a method for extracting key features of a temporal bone CT image provided by an embodiment of the present invention;

[0049] Figure 6 A diagram of a jump connection processing method for a high-frequency attention guidance module provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are part of the embodiments of the present invention, rather than all of the embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work should fall within the scope of protection of the present invention.

[0051] Furthermore, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts disclosed in the present invention.

[0052] In the description of this invention, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The terms "mounted," "connected," and "connected" should be interpreted broadly, meaning, for example, fixed, removable, or integral; mechanical or electrical; direct or indirect through an intermediary; and internal communication between two components. Those skilled in the art will understand the specific meanings of these terms in this invention on a case-by-case basis.

[0053] Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, like numbers in different figures represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present invention. Rather, they are merely examples of methods and systems consistent with certain aspects of the present invention, as detailed in the appended claims.

[0054] The present invention proposes an automatic segmentation method for the facial nerve and chorda tympani nerve based on temporal bone CT. This method not only overcomes the technical difficulties of tiny nerve structure and low contrast, but also can cope with the challenges of individual anatomical variations and pathological conditions, thereby providing clinicians with comprehensive, reliable and efficient preoperative three-dimensional navigation information, truly improving the accuracy and safety of otolaryngology and skull base surgery.

[0055] Method Example

[0056] Figure 1 A schematic diagram of a method for automatic segmentation of the facial nerve and chorda tympani nerve based on temporal bone CT provided by an embodiment of the present invention.

[0057] Figure 2 A schematic diagram of an automatic segmentation architecture of the facial nerve and chorda tympani nerve based on temporal bone CT provided by an embodiment of the present invention.

[0058] The present invention proposes an automatic segmentation method of facial nerve and chorda tympani nerve based on temporal bone CT, which is applied to a model consisting of a multi-layer encoder, a multi-layer decoder, a fast discrete wavelet transform convolution fusion module, a wavelet transform-based Mamba module and a high-frequency attention guidance module jump connection, combined with Figure 1 , the method includes six steps from S1 to S6:

[0059] S1: Acquire a temporal bone CT image, input the temporal bone CT image into the first layer encoder, downsample the temporal bone CT image, and change the number of channels of the temporal bone CT image to 32.

[0060] Specifically, in the embodiment of the present invention, combined with Figure 2 The temporal bone CT image is the original temporal bone CT image. Here, the size of the temporal bone CT image can be standardized and then input into the first layer encoder 201 to downsample the temporal bone CT image and change the number of channels of the temporal bone CT image to 32.

[0061] S2: The temporal bone CT image with 32 channels is sequentially input into the second, third, and fourth encoder layers for downsampling to achieve dimensionality reduction and temporal bone CT image feature extraction.

[0062] Specifically, in the embodiment of the present invention, combined with Figure 2 The temporal bone CT image with 32 channels is sequentially input into the second layer encoder 202, the third layer encoder 203 and the fourth layer encoder 204 for downsampling to achieve dimensionality reduction and the method for extracting temporal bone CT image features is described in detail below.

[0063] S3: The temporal bone CT image features output by the fourth encoder are sequentially input into the fifth encoder, the sixth encoder and the last encoder with the fast discrete wavelet transform convolution fusion module for downsampling to achieve dimensionality reduction and enhance the temporal bone CT image features.

[0064] Specifically, in the embodiment of the present invention, combined with Figure 2 The temporal bone CT image features output by the fourth-layer encoder 204 are sequentially input into the fifth-layer encoder 205, the sixth-layer encoder 206, and the last-layer encoder 207 to which the fast discrete wavelet transform convolution fusion module is added for downsampling to achieve dimensionality reduction and enhance the temporal bone CT image features. The method is described in detail below. The English abbreviation of the fast discrete wavelet transform convolution fusion module here is FDWTC fusion module, and the corresponding full English name is Fast Discrete Wavelet Transform Convolution fusion module. The FDWTC fusion module is added to the fifth-layer encoder 205, the sixth-layer encoder 206, and the last-layer encoder 207.

[0065] S4: The enhanced temporal bone CT image features in the last layer of encoder are input into the Mamba module based on wavelet transform to extract the key features of the temporal bone CT image.

[0066] Specifically, in the embodiment of the present invention, combined with Figure 2 The enhanced temporal bone CT image features in the last layer of encoder 207 are input into the wavelet transform-based Mamba module to extract the key features of the temporal bone CT image. The method is described in detail below. The wavelet transform Mamba module here is abbreviated as WMB module, and its full name is Wavelet-based MambaBlock module. Mamba is a new sequence modeling architecture based on a selective state space model.

[0067] S5: The temporal bone CT image features extracted from the second, third and fourth encoders, as well as the temporal bone CT image features enhanced by the fifth and sixth encoders, are processed by jump connection in the high-frequency attention guidance module and then input into the second, third, fourth, fifth and sixth decoders.

[0068] Specifically, in the embodiment of the present invention, combined with Figure 2The temporal bone CT image features extracted from the second-layer encoder 202, the third-layer encoder 203, and the fourth-layer encoder 204, as well as the temporal bone CT image features enhanced by the fifth-layer encoder 205 and the sixth-layer encoder 206, are respectively processed by the high-frequency attention guidance module jump connection and input into the second-layer decoder 212, the third-layer decoder 213, the fourth-layer decoder 214, the fifth-layer decoder 215, and the sixth-layer decoder 216. The English abbreviation of the high-frequency attention guidance module jump connection is HGAM jump connection, and its full name is High-frequency Guided Attention Module jump connection. The HGAM jump connection retains key semantic information after upsampling and maintains spatial resolution in the gradual restoration process. The high-frequency attention guidance module jump connection processing method is described in detail below.

[0069] S6: The temporal bone CT image features enhanced by the last layer encoder and the key features of the temporal bone CT image are input into the last layer decoder. The last layer decoder, the sixth layer decoder, the fifth layer decoder, the fourth layer decoder, the third layer decoder, the second layer decoder and the first layer decoder are upsampled in sequence to achieve feature recovery and decoding.

[0070] Specifically, in the embodiment of the present invention, combined with Figure 2 The temporal bone CT image features enhanced by the last layer encoder and the key features of the temporal bone CT image are input into the last layer decoder 217. The last layer decoder 217, the sixth layer decoder 216, the fifth layer decoder 215, the fourth layer decoder 214, the third layer decoder 213, the second layer decoder 212 and the first layer decoder 211 perform upsampling in sequence to achieve feature recovery and decoding.

[0071] It should be understood that the last layer decoder 217, the sixth layer decoder 216, the fifth layer decoder 215, the fourth layer decoder 214, the third layer decoder 213, and the second layer decoder 212 all have two inputs, one of which is the input from the upper layer decoder, and the other is the temporal bone CT image features extracted from the second layer encoder 202, the third layer encoder 203, and the fourth layer encoder 204, and the temporal bone CT image features enhanced by the fifth layer encoder 205 and the sixth layer encoder 206, respectively, after being processed by the high-frequency attention guidance module through jump connection and input into the corresponding decoder. For example, the input of the second layer decoder 212 is the features of the second layer encoder 202 after jump connection processing through the high-frequency attention guidance module and the features output after upsampling by the third layer decoder 213.

[0072] S7: The output of the first layer decoder is binarized to generate the facial nerve and chorda tympani nerve segmentation prediction results.

[0073] Specifically, in the embodiment of the present invention, combined with Figure 2 ,Through upsampling, the output of the first layer decoder 211 is binarized to generate the facial nerve and chorda tympani nerve segmentation prediction results.

[0074] Figure 3 A diagram of a method for downsampling to achieve dimensionality reduction and extracting temporal bone CT image features provided in an embodiment of the present invention.

[0075] Furthermore, in the above-mentioned automatic segmentation method of facial nerve and chorda tympani nerve based on temporal bone CT, the temporal bone CT image with 32 channels is sequentially input into the second encoder, the third encoder and the fourth encoder for downsampling to achieve dimensionality reduction and temporal bone CT image feature extraction, combined with Figure 3 , including three steps from S31 to S33:

[0076] S31: The second-layer encoder doubles the number of channels through convolution normalization activation operation, and the doubled number of channels is less than the preset value;

[0077] S32: The third encoder layer extracts CT image features through convolutional normalization activation operations;

[0078] S33: The fourth encoder layer improves model performance through residual connections.

[0079] Specifically, in the embodiment of the present invention, in the second layer encoder, the third layer encoder and the fourth layer encoder, the second layer encoder first doubles the number of channels through the convolution normalization activation operation, but the number of channels cannot exceed the specified maximum value. Then, the third layer encoder extracts the temporal bone CT image features through the convolution normalization activation operation, and the fourth layer encoder improves the model performance through the residual connection.

[0080] Figure 4 A method for downsampling to achieve dimensionality reduction and enhance temporal bone CT image features provided in an embodiment of the present invention.

[0081] Furthermore, in the above-mentioned automatic segmentation method of facial nerve and chorda tympani nerve based on temporal bone CT, the temporal bone CT image features output by the fourth-layer encoder are sequentially input into the fifth-layer encoder, the sixth-layer encoder and the last-layer encoder with the fast discrete wavelet transform convolution fusion module for downsampling to achieve dimensionality reduction and enhance the temporal bone CT image features, and combined with Figure 4 , including seven steps from S41 to S47:

[0082] S41: Divide the temporal bone CT image features into two parts with the same batch, channel, height and width, the first part stores local features, and the second part stores global features.

[0083] Specifically, in an embodiment of the present invention, the temporal bone CT image features are divided into two parts with the same shape (Batchsize, Channels, Height, Weight). The left part is used to store local features with a shape of (B, C / 2, H, W), and the right part is used to store global features with a shape of (B, C / 2, H, W).

[0084] S42: Perform 3*3 convolution on the local features and the global features to extract the local features, perform 3*3 wavelet convolution on the global features, and then use the local features to extract the global features through frequency space blocks.

[0085] Specifically, in an embodiment of the present invention, local features and global features are subjected to 3x3 convolution to extract local features, local features are subjected to 3x3 wavelet convolution and global features are subjected to frequency space blocks to extract global features. The English abbreviation of frequency space block is SFB, and the full English name is Spatial Frequency Block. Wavelet convolution with a convolution kernel of 3x3 has a larger receptive field than ordinary convolution with a convolution kernel of 3x3, and can extract global features.

[0086] Here, the frequency branch in the frequency space block is transformed by discrete wavelet transform (DWT), in which the Haar transform is used to establish long-distance dependencies and extract global features. The spatial branch uses a residual module based on a convolutional neural network to enhance the representation of spatial local information. The convolutional neural network is abbreviated as CNN, in which the full name is Convolutional Neural Network.

[0087] S43: Global features and local features are fused through the spatial attention module, and global features and local features are processed through batch normalization and activation function.

[0088] Specifically, in an embodiment of the present invention, global features and local features are fused through a spatial attention module. The English abbreviation of the spatial attention module is SAM, and its full English name is Spatial Attention Module. Then, batch normalization and Relu activation function are used to process the global features and local features.

[0089] S44: The global features and local features are concatenated along the channel direction and then passed through the Sigmoid function.

[0090] Specifically, in the embodiment of the present invention, the global features and the local features are spliced ​​along the channel direction, and finally passed through the Sigmoid function to prevent gradient explosion.

[0091] The global features and local features are concatenated along the channel direction using the following formula:

[0092] Output=Concat((local,global),dim=1)

[0093] Among them, concat(.) represents the concatenation operation, dim=1 represents the concatenation along 1 dimension (i.e., the channel dimension), and the above formula represents the concatenation of local and global along the channel direction. Output Output∈X B,C,H,W .

[0094] S45: Perform residual addition on the output of the Sigmoid function and the temporal bone CT image feature output by the fourth encoder to obtain the output of the fifth encoder.

[0095] Specifically, in the embodiment of the present invention, the residual is added to the original input feature after Sigmoid, which can enhance the model's expressive power.

[0096] S46: downsample the output of the fifth encoder and input it to the sixth encoder, repeat the above five steps S41 to S45 to obtain the output of the sixth encoder;

[0097] S47: downsample the output of the sixth layer encoder and input it to the last layer encoder, repeat the above five steps S41 to S45 to obtain the output of the last layer encoder.

[0098] Figure 5 A diagram of a method for extracting key features of temporal bone CT images provided by an embodiment of the present invention.

[0099] Furthermore, in the above-mentioned automatic segmentation method of facial nerve and chorda tympani nerve based on temporal bone CT, the enhanced temporal bone CT image features in the last layer of encoder are input into the Mamba module based on wavelet transform to extract the key features of the temporal bone CT image, and the temporal bone CT image features are extracted by combining the temporal bone CT image features with the Mamba module based on wavelet transform. Figure 5 , including three steps from S51 to S53:

[0100] S51: The enhanced temporal bone CT image feature layer in the last encoder layer is normalized and separated into high-frequency-high-frequency component, high-frequency-low-frequency component, low-frequency-high-frequency component and low-frequency-low-frequency component through discrete wavelet transform.

[0101] Specifically, in the embodiment of the present invention, at the intersection of the encoder and the decoder, the discrete wavelet transform will not change the number of channels or lose information, and the layer is first normalized and then the discrete wavelet transform is used to separate the input into four parts: HH (high-frequency-high-frequency component), HL (high-frequency-low-frequency component), LH (low-frequency-high-frequency component), and LL (low-frequency-low-frequency component).

[0102] S52: After ordinary convolution, activation function, convolution and layer normalization, the high-frequency-high-frequency component, high-frequency-low-frequency component and low-frequency-high-frequency component are passed through the Mamba module based on wavelet transform to obtain long-distance dependencies, enhancing the model's ability to understand global changes. The low-frequency-low-frequency component is convolved to extract local features.

[0103] Specifically, in the embodiment of the present invention, the three parts of high-frequency-high-frequency component, high-frequency-low-frequency component, and low-frequency-high-frequency component represent edge and detail features. The high-frequency-high-frequency component, high-frequency-low-frequency component, and low-frequency-high-frequency component are subjected to Conv (ordinary convolution), Relu (activation function), Conv (convolution), LayerNorm (layer normalization) and then passed through the Mamba module based on wavelet transform to obtain long-distance dependencies, thereby enhancing the model's ability to understand global changes. The low-frequency-low-frequency component is subjected to convolution to extract local features.

[0104] S53: The high-frequency-high-frequency component, high-frequency-low-frequency component, low-frequency-high-frequency component, and low-frequency-low-frequency component are concatenated along the 0-dimensional direction, restored to the input size after transformation, and finally the key features of the temporal bone CT image are extracted through residual connection.

[0105] Specifically, in the embodiment of the present invention, the formula for splicing the high-frequency-high-frequency component, the high-frequency-low-frequency component, the low-frequency-high-frequency component, and the low-frequency-low-frequency component along the 0-dimensional direction is expressed as:

[0106] X=Concat(((HH,HL,LH),LL),dim=0)

[0107] Here, after transformation, the input size is restored, and finally the key features of the temporal bone CT image are extracted through residual connection.

[0108] It should be understood that discrete wavelet transform and the wavelet transform-based Mamba module can effectively capture long-distance dependencies, enhance the model's ability to understand global changes, and enhance the ability to extract key features.

[0109] Figure 6 A diagram of a jump connection processing method for a high-frequency attention guidance module provided in an embodiment of the present invention.

[0110] Furthermore, in the above-mentioned automatic segmentation method of facial nerve and chorda tympani nerve based on temporal bone CT, Figure 6 The processing method of the high-frequency attention guidance module jump connection includes three steps from S61 to S63:

[0111] S61: After performing discrete wavelet transform on the input feature map, convolution, batch normalization, activation, convolution, and batch normalization operations are performed on the high-frequency components;

[0112] S62: The high-frequency components are processed by the convolutional block attention module to optimize the contextual semantics, and then convolved with the high-frequency components after discrete wavelet transformation. After batch normalization, residual connection is performed.

[0113] S63: Concatenate the input feature map, perform convolution, batch normalization, and activate the output feature map.

[0114] Specifically, in an embodiment of the present invention, after the input feature map is subjected to discrete wavelet transform, ConvBnRelu (convolution, batch normalization, Relu activation) and ConvBn (convolution, batch normalization) operations are performed on the high-frequency components, and then the high-frequency components are processed through the convolution block attention module to optimize the contextual semantics. The residual connection is performed after the ConvBnConvBn (convolution, batch normalization) operation with the original high-frequency components, and finally the Concat (splicing) operation is performed with the original input feature map, and the feature map is output through ConvBnRelu (convolution, batch normalization, Relu activation), which enhances the feature extraction capability, enhances the information integration effect, and highlights the key information.

[0115] Furthermore, in the above-mentioned method for automatic segmentation of the facial nerve and chorda tympani nerve based on temporal bone CT, the number of channels of the second-layer decoder is consistent with that before the second-layer encoder downsampling, the number of channels of the third-layer decoder is consistent with that before the third-layer encoder downsampling, the number of channels of the fourth-layer decoder is consistent with that before the fourth-layer encoder downsampling, the number of channels of the fifth-layer decoder is consistent with that before the fifth-layer encoder downsampling, and the number of channels of the sixth-layer decoder is consistent with that before the sixth-layer encoder downsampling.

[0116] Specifically, in the embodiment of the present invention, the number of channels of the decoder is consistent with that of the encoder of the same layer before downsampling.

[0117] Furthermore, in the above-mentioned method for automatic segmentation of facial nerve and chorda tympani nerve based on temporal bone CT, upsampling is to increase the corresponding image to twice its original size.

[0118] Specifically, in the embodiment of the present invention, upsampling improves the resolution of the feature map, doubles the size, and the number of channels corresponds to the encoder to reconstruct detail information.

[0119] Furthermore, in the above-mentioned method for automatic segmentation of facial nerve and chorda tympani nerve based on temporal bone CT, downsampling is to reduce the corresponding image to half of its original size.

[0120] Specifically, in the embodiment of the present invention, downsampling reduces the size of the feature map, halves the size, increases the number of channels, does not exceed a specified value, and extracts important features.

[0121] Furthermore, in the above-mentioned automatic segmentation method of the facial nerve and chorda tympani nerve based on temporal bone CT, the last layer decoder, the sixth layer decoder, the fifth layer decoder, the fourth layer decoder, the third layer decoder, the second layer decoder and the first layer decoder are used to perform convolution normalization activation and residual connection operations to achieve the restoration and reconstruction of the facial nerve and chorda tympani nerve.

[0122] Specifically, in an embodiment of the present invention, the last layer decoder, the sixth layer decoder, the fifth layer decoder, the fourth layer decoder, the third layer decoder and the second layer decoder are jump-connected to the frequency attention guidance module, and convolution normalization activation and residual connection operations are performed to achieve the restoration and reconstruction of the facial nerve and chorda tympani nerve. The output of the second layer decoder is input into the first layer decoder to obtain the segmented facial nerve and chorda tympani nerve.

[0123] Furthermore, in the above-mentioned method for automatic segmentation of facial nerve and chorda tympani nerve based on temporal bone CT, the transformation for restoring the transformed image to the input size is an inverse wavelet transform.

[0124] Specifically, in the embodiment of the present invention, the English abbreviation of inverse wavelet transform is IWT, and the full English name is Inverse Wavelet Transform.

[0125] Those skilled in the art will appreciate that although some embodiments described herein include some features and not others included in other embodiments, the combination of features from different embodiments is intended to be within the scope of the invention and to form different embodiments.

[0126] Those skilled in the art will understand that the description of each embodiment has its own focus, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0127] Although the embodiments of the present invention are described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations shall fall within the scope defined by the appended claims. The above are only specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or substitutions within the technical scope disclosed by the present invention, and such modifications or substitutions shall be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

[0128] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for automatic segmentation of facial nerve and chorda tympani nerve based on temporal bone CT, characterized in that: Applied to a model consisting of a multi-layer encoder, a multi-layer decoder, a fast discrete wavelet transform convolution fusion module, a wavelet transform-based Mamba module, and a high-frequency attention guidance module jump connection, the method includes: Acquire a temporal bone CT image, input the temporal bone CT image into a first-layer encoder to downsample the temporal bone CT image so that the number of channels of the temporal bone CT image is changed to 32; The temporal bone CT image with 32 channels is sequentially input into the second, third and fourth encoder layers for downsampling to achieve dimensionality reduction and temporal bone CT image feature extraction. The temporal bone CT image features output by the fourth encoder are sequentially input into the fifth encoder, the sixth encoder, and the last encoder with a fast discrete wavelet transform convolution fusion module for downsampling to achieve dimensionality reduction and enhance the temporal bone CT image features; The enhanced temporal bone CT image features in the last encoder layer are input into the Mamba module based on wavelet transform to extract the key features of the temporal bone CT image; The temporal bone CT image features extracted from the second, third, and fourth encoder layers, as well as the temporal bone CT image features enhanced by the fifth and sixth encoder layers, are processed by the high-frequency attention guidance module through skip connections and then input into the second, third, fourth, fifth, and sixth decoders, respectively. The temporal bone CT image features enhanced by the last encoder layer and the key features of the temporal bone CT image are input into the last decoder layer. The last decoder layer, the sixth decoder layer, the fifth decoder layer, the fourth decoder layer, the third decoder layer, the second decoder layer and the first decoder layer are upsampled in sequence to achieve feature recovery and decoding; The output of the first layer decoder is binarized to generate the facial nerve and chorda tympani nerve segmentation prediction results.

2. The method for automatic segmentation of facial nerve and chorda tympani nerve based on temporal bone CT according to claim 1, characterized in that: The temporal bone CT image with 32 channels is sequentially input into the second layer encoder, the third layer encoder and the fourth layer encoder for downsampling to achieve dimensionality reduction and extract features of the temporal bone CT image, including: The second-layer encoder doubles the number of channels through convolutional normalization activation operations, and the doubled number of channels is less than the preset value; The third layer encoder extracts CT image features through convolutional normalization activation operations; The fourth layer encoder improves model performance through residual connections.

3. The method for automatic segmentation of facial nerve and chorda tympani nerve based on temporal bone CT according to claim 1, characterized in that: The temporal bone CT image features output by the fourth encoder are sequentially input into the fifth encoder, the sixth encoder, and the final encoder with the fast discrete wavelet transform convolution fusion module for downsampling to achieve dimensionality reduction and enhance the temporal bone CT image features, including: Step 1: Divide the temporal bone CT image features into two parts with the same batch, channel, height and width. The first part stores local features, and the second part stores global features. Step 2: Perform 3*3 convolution on local features and global features to extract local features, perform 3*3 wavelet convolution on global features, and then use frequency space blocks to extract global features. Step 3: The global features and local features are fused through the spatial attention module, and the global features and local features are processed through batch normalization and activation function; Step 4: Concatenate the global features and local features along the channel direction and pass them through the Sigmoid function; Step 5: Add the residual of the output of the Sigmoid function to the temporal bone CT image features output by the fourth encoder to obtain the output of the fifth encoder; The output of the fifth encoder is downsampled and input into the sixth encoder layer, and the above five steps are repeated to obtain the output of the sixth encoder layer; The output of the sixth encoder is downsampled and input into the last encoder, and the above five steps are repeated to obtain the output of the last encoder.

4. The method for automatic segmentation of facial nerve and chorda tympani nerve based on temporal bone CT according to claim 1, characterized in that: The enhanced temporal bone CT image features in the last encoder layer are input into the Mamba module based on wavelet transform to extract the key features of the temporal bone CT image, including: The enhanced temporal bone CT image feature layer in the last encoder layer is normalized and then separated into high-frequency-high-frequency component, high-frequency-low-frequency component, low-frequency-high-frequency component and low-frequency-low-frequency component through discrete wavelet transform; After ordinary convolution, activation function, convolution, and layer normalization, the high-frequency-high-frequency component, high-frequency-low-frequency component, and low-frequency-high-frequency component pass through the Mamba module based on wavelet transform to obtain long-range dependencies, enhancing the model's ability to understand global changes. The low-frequency-low-frequency component undergoes convolution to extract local features. The high-frequency-high-frequency component, high-frequency-low-frequency component, low-frequency-high-frequency component and low-frequency-low-frequency component are spliced ​​along the 0-dimensional direction, restored to the input size after transformation, and finally the key features of the temporal bone CT image are extracted through residual connection.

5. The method for automatic segmentation of facial nerve and chorda tympani nerve based on temporal bone CT according to claim 1, characterized in that: The processing steps of the high-frequency attention guidance module skip connection include: After performing discrete wavelet transform on the input feature map, convolution, batch normalization, activation, convolution, and batch normalization operations are performed on the high-frequency components; The high-frequency components are processed by the convolution block attention module to optimize the context semantics, and then convolved with the high-frequency components after discrete wavelet transformation. After batch normalization, residual connection is performed; The feature map is concatenated with the input, and after convolution and batch normalization, the output feature map is activated.

6. The method for automatic segmentation of facial nerve and chorda tympani nerve based on temporal bone CT according to claim 1, characterized in that: The number of channels of the second-layer decoder is the same as that of the second-layer encoder before downsampling, the number of channels of the third-layer decoder is the same as that of the third-layer encoder before downsampling, the number of channels of the fourth-layer decoder is the same as that of the fourth-layer encoder before downsampling, the number of channels of the fifth-layer decoder is the same as that of the fifth-layer encoder before downsampling, and the number of channels of the sixth-layer decoder is the same as that of the sixth-layer encoder before downsampling.

7. The method for automatic segmentation of facial nerve and chorda tympani nerve based on temporal bone CT according to claim 1, characterized in that: Upsampling is to increase the size of the corresponding image to twice its original size.

8. The method for automatic segmentation of facial nerve and chorda tympani nerve based on temporal bone CT according to claim 1, characterized in that: Downsampling is reducing the corresponding image to half of its original size.

9. The method for automatic segmentation of facial nerve and chorda tympani nerve based on temporal bone CT according to claim 1, characterized in that: The last decoder layer, the sixth decoder layer, the fifth decoder layer, the fourth decoder layer, the third decoder layer, the second decoder layer and the first decoder layer are used to perform convolution normalization activation and residual connection operations to achieve the restoration and reconstruction of the facial nerve and chorda tympani nerve.

10. The method for automatic segmentation of facial nerve and chorda tympani nerve based on temporal bone CT according to claim 4, characterized in that: The transformation to restore the input size after the transformation is an inverse wavelet transform.

Images