Lightweight inner ear segmentation method based on channel-space decoupled non-local attention

Abstract

The present application relates to the technical field of computer, and proposes a light-weight inner ear segmentation method based on channel-space decoupled non-local attention, which comprises: obtaining an ear computer tomography image to be segmented; inputting the image into an inner ear segmentation network to obtain a segmentation result of an inner ear structure; the inner ear segmentation network adopts an encoder-decoder structure, and in the encoding and decoding processes, feature processing is performed through a depth separable convolution containing a dynamic routing mechanism, and a channel-space dimension decoupled non-local attention operation is performed on the bottleneck layer between the encoding and decoding. While significantly reducing the model parameter quantity and the calculation overhead, the present application improves the accuracy of segmentation of the complex and tiny structure of the inner ear.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a lightweight inner ear segmentation method based on channel-space decoupled nonlocal attention. Background Technology

[0002] Clear imaging and accurate analysis of the inner ear structure are of great significance for its functional research and related engineering applications. With advancements in imaging technology, ultra-high resolution computed tomography (CT) scanners can clearly display sub-millimeter-level structures in the ear. However, this significant increase in imaging resolution has also led to a dramatic increase in image data volume, making traditional manual processing methods inefficient and unable to meet the demands of processing massive amounts of data.

[0003] Currently, deep learning-based inner ear segmentation technology has made significant progress, with encoder-decoder architectures represented by U-Net and their derivative models being widely adopted. However, when facing ultra-high-resolution inner ear image segmentation tasks, existing inner ear segmentation methods have the following limitations: most models have complex structures and a large number of parameters, resulting in a heavy computational burden when processing high-resolution images, making them difficult to deploy and apply in resource-constrained real-world scenarios; due to the small scale, complex structure, and blurred edges of the inner ear target, existing models often fail to fully capture its global semantics and spatial relationships, easily leading to problems such as segmentation region breakage or adhesion, affecting the accuracy of inner ear segmentation.

[0004] Therefore, there is an urgent need for an inner ear segmentation method that can significantly reduce the computational complexity of the model and enhance the accuracy of inner ear segmentation. Summary of the Invention

[0005] To address the technical problems of existing methods for processing fine inner ear structures in high-resolution CT images, such as excessively complex computational models leading to deployment difficulties and insufficient global context modeling capabilities for complex and minute inner ear structures resulting in poor inner ear segmentation accuracy, this invention proposes a lightweight inner ear segmentation method based on channel-space decoupled nonlocal attention, which includes: Acquire computed tomography images of the ear to be segmented; The image is input into the inner ear segmentation network to obtain the segmentation results of the inner ear structure; The inner ear segmentation network adopts an encoder-decoder structure, and performs feature processing through depthwise separable convolutions with dynamic routing mechanisms in both the encoding and decoding processes. It also performs nonlocal attention operations to decouple channel and spatial dimensions in the bottleneck layer between encoding and decoding.

[0006] In some embodiments, the feature processing via depthwise separable convolutions incorporating dynamic routing mechanisms includes: Dynamically route the features input to the encoding or decoding process to obtain the filtered key features; Perform depthwise separable convolution operations on the key features to obtain multi-scale fused features; The multi-scale fusion features are subjected to channel attention enhancement processing to obtain enhanced features; The enhanced features are residually concatenated with the features input to the encoding or decoding process to obtain the output features of the encoding or decoding process.

[0007] In some embodiments, the dynamic routing operation on the features input to the encoding or decoding process to obtain the filtered key features includes: Global average pooling is performed on the features input to the encoding or decoding process. Then, the pooling results are sequentially subjected to convolutional dimensionality reduction, nonlinear activation, convolutional mapping, and normalization to generate routing weights. The features input to the encoding or decoding process are subjected to grouped convolution, batch normalization, and feature reshaping to obtain grouped features; The grouping features are weighted and fused based on the routing weights to obtain the filtered key features.

[0008] In some embodiments, performing a depthwise separable convolution operation on the key features to obtain multi-scale fused features includes: The key features are input into at least two parallel branches, and each branch is processed by convolution, batch normalization and nonlinear activation using depth-separable convolution kernels of different scales before output. The features output from all branches are concatenated to obtain multi-scale fused features.

[0009] In some embodiments, the nonlocal attention operation that decouples the channel from the spatial dimension includes: Channel nonlocal attention operation is performed on the features input to the bottleneck layer to obtain channel semantic enhancement features, and spatial nonlocal attention operation is performed on the features input to the bottleneck layer to obtain spatial location enhancement features. The channel semantic enhancement features and the spatial location enhancement features are concatenated, and then convolution, batch normalization and nonlinear activation are performed to obtain the features output by the bottleneck layer.

[0010] In some embodiments, performing channel nonlocal attention operation on the features input to the bottleneck layer to obtain channel semantically enhanced features includes: The features input to the bottleneck layer are subjected to convolutional transformation to generate query features and key features, and the features input to the bottleneck layer are simultaneously mapped to values ​​that can be used for aggregation. The query features and the key features are spatially compressed, and the similarity between the compressed features is calculated to obtain the channel attention weights. Based on the channel attention weights, the values ​​are weighted and aggregated and inversely transformed to obtain the initial channel enhancement features; The initial channel enhancement features are weighted and fused with the features input to the bottleneck layer to obtain channel semantic enhancement features.

[0011] In some embodiments, performing a spatial nonlocal attention operation on the features input to the bottleneck layer to obtain spatially enhanced features includes: The features input to the bottleneck layer are subjected to convolution transformation and dimension permutation along the height and width dimensions of space, respectively, to generate query features in the horizontal and vertical directions, and at the same time, the features input to the bottleneck layer are mapped to value features in the horizontal and vertical directions. Calculate the similarity between query features in the horizontal direction and between query features in the vertical direction to obtain spatial attention weights in the height and width dimensions. Based on the spatial attention weights, the value features in the corresponding directions are weighted and aggregated, and the channel dimension is restored through inverse permutation operation to obtain the initial spatial enhancement features in the horizontal and vertical directions. The initial spatial enhancement features in the horizontal and vertical directions are weighted and fused with the features input to the bottleneck layer to obtain spatial location enhancement features.

[0012] In some embodiments, the lightweight inner ear segmentation method based on channel-space decoupled nonlocal attention further includes: Construct a dataset of computed tomography images of the ear containing labeled inner ear structure masks; The initial inner ear segmentation network is iteratively trained using the dataset. In each iteration, the prediction error is calculated based on a composite loss function consisting of binary cross-entropy loss and Dice loss, and the parameters of the inner ear segmentation network are updated based on the prediction error until the inner ear segmentation network converges, thus obtaining the inner ear segmentation network.

[0013] In some embodiments, the lightweight inner ear segmentation method based on channel-space decoupled nonlocal attention further includes: A deep supervision strategy is employed during the iterative training process.

[0014] In some embodiments, the deep supervision strategy includes: Based on the intermediate feature maps output by the decoder at each stage of the inner ear segmentation network, corresponding auxiliary segmentation results are generated. Based on the binary cross-entropy loss and the Dice loss, the auxiliary segmentation loss between each of the auxiliary segmentation results and the true labeled inner ear structure mask is calculated respectively. The auxiliary segmentation loss is added to the main segmentation loss calculated based on the composite loss function to obtain the total loss; The parameters of the inner ear segmentation network are updated based on the total loss.

[0015] The present invention has at least the following beneficial effects: The present invention proposes a lightweight inner ear segmentation method based on channel-space decoupled nonlocal attention, which significantly reduces the number of model parameters and computational overhead while improving the accuracy of segmenting complex and tiny structures of the inner ear.

[0016] Specifically, by employing depthwise separable convolutions with dynamic routing mechanisms for feature processing in both encoding and decoding, the depthwise separable convolutions significantly reduce the model's parameter size. The dynamic routing mechanism adaptively selects key features and suppresses redundancy. Together, they achieve a lightweight network model, reducing computational burden and making it more suitable for deployment and application in scenarios with limited computing resources. This solves the problem of complexity and deployment difficulties in existing inner ear segmentation network models. By performing nonlocal attention operations at the bottleneck layer between encoding and decoding to decouple channel and spatial dimensions, global context modeling is decoupled into two independent processes: semantic dependency modeling in the channel dimension and long-range positional dependency modeling in the spatial dimension. This reduces the computational complexity of traditional nonlocal operations and can more comprehensively and efficiently capture the global semantic information and spatial structural relationships of images. This significantly enhances the network's ability to identify and segment small-scale and vaguely defined inner ear structures, effectively reducing over-segmentation and under-segmentation, and improving the accuracy of the segmentation results. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0018] Figure 1 The flowchart shown is a lightweight inner ear segmentation method based on channel-space decoupling nonlocal attention, provided as an embodiment of the present invention. Figure 2 The diagram shown is an overall structural schematic of a lightweight inner ear segmentation network provided in another embodiment of the present invention. Figure 3 The diagram shown is a structural schematic of a dynamic routing depth-separable convolutional module provided in another embodiment of the present invention; Figure 4 The diagram shown is a schematic representation of a channel-space decoupling nonlocal attention mechanism provided in another embodiment of the present invention. Figure 5 The diagram shown is a flowchart of the training and inference process of a lightweight inner ear segmentation network provided in another embodiment of the present invention. Figure 6 The diagram shown is a schematic of the inner ear segmentation result output by a lightweight inner ear segmentation network provided in another embodiment of the present invention. Detailed Implementation

[0019] The following describes embodiments of the present invention. However, it should be understood that the disclosed embodiments are merely examples, and other embodiments may take various alternative forms.

[0020] Furthermore, it should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or apparatus that comprises a list of elements may include not only those elements but also elements not expressly listed or inherent to such process, method, article, or apparatus.

[0021] One or more embodiments of the present invention will now be described with reference to the accompanying drawings.

[0022] Based on the above objectives, a first aspect of the present invention proposes an embodiment of a lightweight inner ear segmentation method based on channel-space decoupled nonlocal attention. For example... Figure 1 As shown, Figure 1 The flowchart shown is a lightweight inner ear segmentation method based on channel-space decoupled nonlocal attention, provided as an embodiment of the present invention, which includes: S1. Obtain the computed tomography image of the ear to be segmented; S2. Input the image into the inner ear segmentation network to obtain the segmentation result of the inner ear structure. The inner ear segmentation network adopts an encoder-decoder structure, and performs feature processing through depthwise separable convolution with dynamic routing mechanism in both encoding and decoding processes, as well as nonlocal attention operation to decouple channel and spatial dimensions in the bottleneck layer between encoding and decoding.

[0023] Among them, reference Figure 2 The inner ear segmentation network uses a U-Net type encoder-decoder architecture as its basic structure. The input is the computed tomography (CT) image of the ear to be segmented, and the output is the corresponding inner ear structure segmentation map.

[0024] The encoder consists of four stages, with the number of output channels in each stage being {24, 48, 96, 192} respectively. Each stage of the encoder is composed of a dynamically routed depthwise separable convolutional module and a downsampling module connected in series. Specifically, the first stage consists of two cascaded depthwise separable convolutional blocks and a downsampling module; the second to fourth stages each consist of two cascaded depthwise separable convolutional blocks with a dynamic routing mechanism and a downsampling module. Let the input image size be H×W×3, then the size of the output feature map of the i-th stage is... , where C=12.

[0025] The decoder section is symmetrical to the encoder and also comprises four stages. The decoder achieves feature reconstruction and restoration symmetrically through an upsampling module and a similar convolutional module. First, a 3×3 convolution and bilinear interpolation upsampling operation doubles the spatial resolution of the input feature map and halves the number of channels. Then, two cascaded depthwise separable convolutional blocks with dynamic routing mechanisms refine the features, aligning their scale with the feature map of the corresponding stage in the encoder. These depthwise separable convolutional blocks with dynamic routing mechanisms effectively suppress feature redundancy and perform efficient feature extraction with a low parameter count, controlling computational overhead while maintaining strong representational power.

[0026] The bottleneck layer is located between the deepest layer of the encoder (where the feature map size is smallest) and the shallowest layer of the decoder. A nonlocal attention mechanism, decoupled from the channel and spatial dimensions, is embedded in the bottleneck layer to fuse global contextual information and enhance the model's ability to perceive complex structures. By modeling long-range dependencies in both the channel and spatial dimensions, the ability to capture global contextual information is enhanced, thereby reducing missegmentation and improving the accuracy of inner ear structure segmentation.

[0027] The inner ear segmentation network architecture described above is a lightweight network structure. Depthwise separable convolution significantly reduces the model's parameter size, while the dynamic routing mechanism adaptively selects key features and suppresses redundancy. These two mechanisms work synergistically to achieve a lightweight network model, reducing computational burden and making it more suitable for deployment and application in scenarios with limited computing resources. At the bottleneck layer between encoding and decoding, a nonlocal attention operation decouples the channel and spatial dimensions, decoupling global context modeling into two independent processes: semantic dependency modeling in the channel dimension and long-range positional dependency modeling in the spatial dimension. This reduces the computational complexity of traditional nonlocal operations and can more comprehensively and efficiently capture the global semantic information and spatial structural relationships of images. This significantly enhances the network's ability to identify and segment small-scale and vaguely defined inner ear structures, effectively reducing over-segmentation and under-segmentation, and improving the accuracy of inner ear segmentation results.

[0028] Among them, reference Figure 3The processing steps of the depthwise separable convolutional module with dynamic routing mechanism in this invention include: dynamic route filtering, multi-scale depthwise separable convolution, channel attention enhancement, and residual connections, to efficiently handle redundancy in the feature map. Specifically, refer to... Figure 3 (a) Through a dynamic routing mechanism, key information in the input features is adaptively filtered to suppress redundancy and reduce the computational burden on subsequent tasks; (Refer to...) Figure 3 (b) Multi-scale depthwise separable convolution operations are performed on the selected features to effectively control the number of model parameters while extracting multi-scale contextual features; feature channels are enhanced through 1×1 convolution and efficient channel attention (ECA) mechanism to highlight important feature responses; Reference Figure 3 (c) By fusing the original and processed features through residual connections, smooth information transfer and stable training are ensured. Multi-scale depthwise separable convolution, as a lightweight convolution operation to replace standard convolution, can reduce the computational complexity and number of parameters of the model. It decomposes standard convolution into two independent steps: depthwise convolution and pointwise convolution. Depthwise convolution is used to perform spatial convolution on each input channel individually, while pointwise convolution, for example, is a 1×1 convolution used to combine channel information. Compared with standard convolution, the multi-scale depthwise separable convolution of this invention can significantly reduce the amount of computation.

[0029] The inner ear structure is intricate in shape and exhibits high local feature similarity; relying solely on local convolution can easily lead to missegmentation or discontinuous segmentation. Introducing global contextual information can provide the model with spatial structural relationships and semantic constraints, helping to distinguish adjacent anatomical structures such as the cochlea, vestibule, and semicircular canals. Non-local attention operations, as a type of attention mechanism, can be used to model the dependencies between any two locations in the feature map, capturing long-range contextual information, rather than focusing solely on the local neighborhood as in traditional convolution. (Reference) Figure 4 The nonlocal attention mechanism of this invention, which decouples the channel and spatial dimensions, decouples the global dependency modeling into two parallel and independent sub-processes: (See reference...) Figure 4 (a) Channel-based nonlocal attention calculates the correlation between all channels along the channel dimension, performing semantic dependency modeling along the channel dimension; Reference Figure 4 (b) Spatial nonlocal attention calculates the correlation between all locations in the spatial dimension, performing spatial location dependency modeling. (See reference) Figure 4(c) The features output from parallel processing of channel non-local attention and spatial non-local attention are concatenated and then processed through 1×1 convolution, batch normalization, and activation functions to finally output features enhanced by global context, achieving synergistic enhancement of channel semantic modeling and spatial location modeling. This design significantly reduces the computational complexity of global attention and effectively improves the model's ability to capture global contextual information by separately enhancing the long-range dependencies of channel semantics and spatial location, thereby improving the accuracy of inner ear segmentation.

[0030] In some embodiments, model training is required to obtain a trained inner ear segmentation network. Please refer to [reference needed]. Figure 5 The training process includes dataset construction, loss function design, and optimization strategies. One approach is to collect a batch of ear CT images and have experts annotate the precise contours of the inner ear structures to create pixel-level annotation masks, thus constructing the training dataset. As a feasible implementation, the loss function used during training can combine binary cross-entropy loss and Dice loss. The former focuses on pixel-level classification accuracy, while the latter focuses on the overlap between the predicted and real regions. Combining the two helps improve the boundary accuracy and region integrity of the segmentation. In other embodiments, to further improve training performance, a deep supervision strategy can be employed, applying auxiliary loss function constraints to multiple intermediate layers of the decoder to enhance gradient propagation and feature learning. The inner ear segmentation network undergoes multiple rounds of iterative optimization on the training data using backpropagation algorithms and optimizers (such as AdamW) until the model performance converges, resulting in a well-trained inner ear segmentation network suitable for inference.

[0031] As a specific implementation, inner ear segmentation inference can be performed based on the trained inner ear segmentation network described above. Please continue to refer to... Figure 5 When segmenting a new ear CT image, necessary preprocessing (such as size normalization and intensity standardization) is performed first, followed by input into the trained inner ear segmentation network. This trained network automatically performs forward propagation, sequentially passing through steps such as dynamic routing convolutional feature extraction in the encoder, channel-space decoupling and non-local attention enhancement in the bottleneck layer, and feature restoration in the decoder. Finally, it outputs a segmentation result image at the end of the trained inner ear segmentation network, as shown below. Figure 6 As shown, the pixel regions of the inner ear structure are clearly identified, and this result can be directly used for subsequent analysis or visualization.

[0032] The aforementioned lightweight inner ear segmentation method based on channel-space decoupled nonlocal attention significantly reduces the number of model parameters and computational overhead while improving the accuracy of segmenting complex and minute structures of the inner ear. By employing depthwise separable convolutions with dynamic routing mechanisms for feature processing in both encoding and decoding, the method significantly reduces the model's parameter size. The dynamic routing mechanism adaptively selects key features and suppresses redundancy; the two work synergistically to achieve a lightweight network model, reducing computational burden and making it more suitable for deployment and application in scenarios with limited computing resources. This solves the problem of existing inner ear segmentation network models being complex and difficult to deploy. By performing nonlocal attention operations that decouple the channel and spatial dimensions at the bottleneck layer between encoding and decoding, global context modeling is decoupled into two independent processes: semantic dependency modeling in the channel dimension and long-range positional dependency modeling in the spatial dimension. This reduces the computational complexity of traditional nonlocal operations and can more comprehensively and efficiently capture the global semantic information and spatial structural relationships of images. This significantly enhances the network's ability to identify and segment minute and blurred-boundary inner ear structures, effectively reducing over-segmentation and under-segmentation, and improving the accuracy of inner ear segmentation results.

[0033] According to several embodiments of the present invention, feature processing is performed by depthwise separable convolution with dynamic routing mechanism, including: performing dynamic routing operation on features input to the encoding or decoding process to obtain filtered key features; performing depthwise separable convolution operation on the key features to obtain multi-scale fused features; performing channel attention enhancement processing on the multi-scale fused features to obtain enhanced features; and performing residual connection between the enhanced features and features input to the encoding or decoding process to obtain output features of the encoding or decoding process.

[0034] As one specific embodiment, please refer to Figure 2-3 This invention includes a module with dynamically routed depthwise separable convolutions that replaces the standard convolutional blocks in traditional U-Net and is used to construct the feature extraction parts of the encoder and decoder. This module can be decomposed into several sub-steps during feature processing: adaptively filtering the input features through a dynamic routing mechanism to obtain a set of key features; performing multi-scale depthwise separable convolution operations on these key features to extract and fuse feature information at different scales; finally, channel attention enhancement is applied to the fused features, and they are fused with the original input features of this module through residual connections to obtain the output features. By using this module, the network achieves efficient and targeted feature learning with low parameter counts in each feature transformation step of encoding and decoding.

[0035] According to several embodiments of the present invention, dynamic routing operations are performed on the features input to the encoding or decoding process to obtain filtered key features, including: performing global average pooling on the features input to the encoding or decoding process, and then sequentially performing convolutional dimensionality reduction, nonlinear activation, convolutional mapping and normalization on the pooling results to generate routing weights; performing grouped convolution, batch normalization and feature reshaping on the features input to the encoding or decoding process to obtain grouped features; and performing weighted fusion of the grouped features based on the routing weights to obtain filtered key features.

[0036] As one specific embodiment, please refer to Figure 3 (a) The dynamic routing operation comprises two parallel processing branches: route weight generation and group feature extraction. Specifically, the dynamic route weight generation branch aims to adaptively assign importance weights to different groups of input features, given the input feature map. First, generate a set of route weight vectors. ,in G To determine the number of groups, the process is as follows: First, for... Global average pooling is performed along the spatial dimension to extract global statistical information for each channel. Then, a 1×1 convolution is used to reduce the channel dimension, followed by a ReLU activation function to introduce non-linearity. Finally, another 1×1 convolution maps the features to... G The initial grouping weights are formed in a 3D space; finally, they are normalized using the Softmax function to ensure that the sum of the weights of each group is 1, thus obtaining the final routing weights. This weight This is used for subsequent channel-level recalibration of features to enhance important features and suppress redundant information. This process can be represented by Equation 1: (Formula 1) in, This is a global average pooling operation. This is a 1×1 convolution operation.

[0037] The grouping feature extraction branch aims to structure the input features into a structure with routing weights. The corresponding grouping structure. Specifically, firstly, for the input feature map... Perform 1×1 grouped convolution, with the number of groups being... G To achieve feature interaction within the group; batch normalization (BN) is performed on the convolutional output to accelerate training convergence and stabilize the optimization process; the normalized feature map is reshaped along the channel dimension. G There are 1 group, each containing 1 group. Each channel is used to obtain grouping features. This process can be represented by Formula 2: (Formula 2) in, Indicates grouping characteristics, This represents a 1×1 grouped convolution. This indicates batch normalization processing. Indicates shape reshaping. .

[0038] The outputs of the routing weight branch and the group feature extraction branch are weighted and fused together to filter out key information. Specifically, the routing weights generated above... Features of each group We perform a weighted summation to obtain the filtered key features. This process can be represented by Formula 3: (Formula 3) This fusion operation enables the network to adaptively emphasize the most discriminative feature groups, thereby suppressing redundancy and improving the compactness and effectiveness of feature representation.

[0039] According to several embodiments of the present invention, performing depthwise separable convolution operations on key features to obtain multi-scale fused features includes: inputting key features into at least two parallel branches, each branch being processed by convolution, batch normalization and nonlinear activation processing using depthwise separable convolution kernels of different scales, and then outputting the results; and concatenating the features output from all branches to obtain multi-scale fused features.

[0040] As one specific embodiment, please refer to Figure 3 (b) Multi-scale depthwise separable convolution is used for efficient feature extraction and fusion of the selected key features. Specifically, the feature map obtained from dynamic routing processing is... The input is fed to two parallel branches. It is understood that, depending on the actual processing needs, the input can also be fed to more than two parallel branches for processing, in order to improve processing efficiency. In this embodiment, the example of inputting to two parallel branches is used for illustration.

[0041] Branch 1: Use a 3×3 depthwise separable convolution kernel After processing, batch normalization and ReLU activation are performed to output features. .

[0042] Branch 2: Process using concatenated 1×5 and 5×1 asymmetric depthwise separable convolutional kernels to obtain a wider range of contextual information, followed by batch normalization (BN) and ReLU activation to output features. .

[0043] The two sets of feature maps obtained and Concat along the channel dimension to obtain features that fuse multi-scale contextual information. The above process can be represented by formula 4-6: (Formula 4) (Formula 5) (Formula 6) in, and These represent 3×3 depthwise separable convolution branches and 1×5 and 5×1 asymmetric depthwise separable convolution operations, respectively. This is for splicing operations.

[0044] To further enhance feature representation capabilities, please refer to Figure 3 (c) Features after splicing Perform 1×1 convolution and efficient channel attention (ECA) processing, then connect the input features via residual connections. Add them together to obtain the final output features of this module. This process can be represented by Formula 7: (Formula 7) in, For splicing operations, For efficient channel attention mechanisms, This is a function mapping operation. This design achieves effective fusion and enhancement of multi-scale features while significantly reducing the number of parameters.

[0045] According to several embodiments of the present invention, a nonlocal attention operation to decouple the channel and spatial dimensions includes: performing a channel nonlocal attention operation on the features input to the bottleneck layer to obtain channel semantic enhancement features, and simultaneously performing a spatial nonlocal attention operation on the features input to the bottleneck layer to obtain spatial location enhancement features; concatenating the channel semantic enhancement features and the spatial location enhancement features, and then performing convolution, batch normalization and nonlinear activation processing to obtain the features output by the bottleneck layer.

[0046] As one specific embodiment, please refer to Figure 4This invention provides a nonlocal attention mechanism that decouples channel and spatial dimensions. This mechanism decouples the long-range dependency modeling advantage of nonlocal attention into two parallel and independent processes: channel nonlocal attention models semantic dependencies between channels, while spatial nonlocal attention models spatial location (height and width) dependencies. Finally, the enhanced features output from these two parallel sub-branches are concatenated and passed through a lightweight fusion layer (e.g., 1×1 convolution, batch normalization, and activation functions), ultimately outputting a bottleneck layer feature enhanced with global context for use by subsequent decoders. This decoupling design, which models semantics through channels and location through spatial dimensions, significantly reduces computational complexity while more efficiently enhancing the global contextual information of features.

[0047] According to several embodiments of the present invention, channel nonlocal attention operations are performed on the features input to the bottleneck layer to obtain channel semantic enhancement features, including: performing convolutional transformation on the features input to the bottleneck layer to generate query features and key features, and simultaneously mapping the features input to the bottleneck layer to values ​​that can be aggregated; compressing the spatial dimensions of the query features and key features, and calculating the similarity between the compressed features to obtain channel attention weights; performing weighted aggregation and inverse shape transformation on the values ​​based on the channel attention weights to obtain initial channel enhancement features; and performing weighted fusion of the initial channel enhancement features and the features input to the bottleneck layer to obtain channel semantic enhancement features.

[0048] As one specific embodiment, please refer to Figure 4 (a) The channel nonlocal attention mechanism specifically includes: given the input feature map First, query features are generated using two independent 1×1 convolutions. Q Bond features K And then perform global average pooling compression on both in terms of spatial dimension. Q,K ∈ B×C′. Then, calculate... Q and K The scaled dot product similarity is used to obtain the channel attention weight matrix after Softmax normalization. This matrix explicitly encodes the global dependencies between channels. Simultaneously, it incorporates the input features... Map the values ​​to V using another 1×1 convolution and adjust their shape. Then apply the attention weights. AND value V After aggregation, the original size is restored through inverse shape transformation, resulting in semantically enhanced features. Finally, the enhanced features are improved using a learnable scalar parameter λ. Adjustments are made, and residual connections are used to connect the original input features. Fusion, output channel semantic enhancement features The above process can be represented by formula 8-13: (Formula 8) (Formula 9) (Formula 10) (Formula 11) (Formula 12) (Formula 13) in, This represents a global average pooling operation. , and They are respectively represented as calculations , and 1×1 convolution operation, and These are the forward and inverse operations of shape transformation. These are learnable scalar parameters. This ensures that original detailed information is preserved while enhancing global semantic dependencies.

[0049] According to several embodiments of the present invention, spatial nonlocal attention operations are performed on the features input to the bottleneck layer to obtain spatial location enhancement features, including: performing convolution transformation and dimension permutation on the features input to the bottleneck layer along the height and width dimensions of space respectively to generate query features in the horizontal and vertical directions, and simultaneously mapping the features input to the bottleneck layer to value features in the horizontal and vertical directions; calculating the similarity between query features in the horizontal direction and between query features in the vertical direction respectively to obtain spatial attention weights in the height and width dimensions; performing weighted aggregation on the value features in the corresponding directions based on the spatial attention weights, and restoring the channel dimension through inverse permutation operation to obtain initial spatial enhancement features in the horizontal and vertical directions; and performing weighted fusion of the initial spatial enhancement features in the horizontal and vertical directions with the features input to the bottleneck layer respectively to obtain spatial location enhancement features.

[0050] As one specific embodiment, please refer to Figure 4 (b) Spatial nonlocal attention mechanisms specifically include: for the same input features Modeling is performed along the height and width directions of the space, respectively. First, query features in the horizontal and vertical directions are generated through two independent 1×1 convolutions. and The shape is then adjusted using a dimension permutation operation. Subsequently, the similarity matrix of the query features in each direction is calculated, and the high-dimensional attention weights are obtained after Softmax normalization. Attention weights in width dimension Simultaneously, the input features are mapped to the corresponding value features. and . Utilize respectively , right , We perform weighted aggregation and then restore the channel dimension through inverse permutation to obtain spatially enhanced features in two directions. and Finally, the fused result is added to the original features as a residual using a learnable parameter γ to output the spatial location enhancement feature. The above process can be represented by formula 14-20: (Formula 14) (Formula 15) (Formula 16) (Formula 17) (Formula 18) (Formula 19) (Formula 20) in, , This represents a 1×1 convolution operation in the horizontal and vertical directions. For the forward and inverse operations of dimension permutation, It is a learnable parameter scalar.

[0051] Please refer to Figure 4 (c) Enhance the semantic features of the above channels Spatial location enhancement features Channel concatenation is performed, followed by fusion and nonlinear transformation using a 1×1 convolution, batch normalization (BN), and ReLU activation function to finally obtain the features output by this attention mechanism, which are rich in global contextual information. This mechanism decouples channel and spatial attention computations, reducing computational costs while accurately enhancing global semantic and spatial structural information, thereby effectively improving the model's accuracy in locating and segmenting complex inner ear structures.

[0052] According to several embodiments of the present invention, a lightweight inner ear segmentation method based on channel-space decoupled nonlocal attention further includes: constructing an ear computed tomography image dataset containing labeled inner ear structure masks; iteratively training an initial inner ear segmentation network using the dataset, wherein, in each iteration, the prediction error is calculated based on a composite loss function consisting of binary cross-entropy loss and Dice loss, and the parameters of the inner ear segmentation network are updated based on the prediction error until the inner ear segmentation network converges, thus obtaining a trained inner ear segmentation network.

[0053] As one specific embodiment, please refer to Figure 5-6 The lightweight inner ear segmentation network of this invention can be implemented in the PyTorch framework. The experimental hardware environment consisted of an Intel i7 CPU (3.6GHz), 12GB of system memory, and an NVIDIA GeForce GTX 1080 Ti GPU (11GB of VRAM). During training, the AdamW optimizer was used, with an initial learning rate of 1e-3, and cosine annealing for decay (maximum period t=20). The momentum parameter was 0.9, and the weight decay coefficient was 1e-6. The batch size was set to 4, and the total number of training epochs was 300. To improve the model's generalization ability, data augmentation methods such as random horizontal flipping, normalization, random scaling, and random pruning were used during training.

[0054] The specific training process of the inner ear segmentation network is as follows: (1) Construction of Inner Ear Segmentation Dataset: To achieve model training and evaluation, this invention constructs a dedicated inner ear structure segmentation dataset. This dataset is derived from clinical images acquired by U-HRCT equipment, including 86 three-dimensional samples. Each sample contains approximately 370 two-dimensional slices. All slices are annotated with pixel-level masks by senior experts for the inner ear structures (such as the cochlea, vestibule, and semicircular canals) in each image slice, forming high-quality segmentation masks, i.e., ground truth labels. Preferably, the size of all images and corresponding annotations is unified, for example, uniformly adjusted to 512×512 pixels, and divided into training and testing sets, for example, randomly divided into training and testing sets in an 8:2 ratio.

[0055] (2) Iterative training: The inner ear segmentation network is trained in multiple epochs using the training set data. In each iteration, a batch of data is input into the inner ear segmentation network for forward propagation to obtain the predicted segmentation map.

[0056] (3) Loss Calculation and Parameter Update: A composite loss function is used to calculate the prediction error between the predicted map and the ground truth label. This composite loss function consists of binary cross-entropy loss and Dice loss. Binary cross-entropy loss is used to provide pixel-by-pixel classification supervision, and Dice loss is used to optimize the overlap between the predicted and ground truth regions. The combination of the two can effectively handle class imbalance and improve boundary accuracy. The gradient of the loss with respect to the network parameters is calculated using the backpropagation algorithm, and an optimizer (such as AdamW) is used to update the network parameters.

[0057] (4) Model convergence: Repeat the above training process until the performance index of the inner ear segmentation network on the validation set tends to be stable, that is, the model converges. The model saved at this time is the trained inner ear segmentation network.

[0058] According to several embodiments of the present invention, a lightweight inner ear segmentation method based on channel-space decoupled nonlocal attention further includes: employing a deep supervision strategy during iterative training. This deep supervision strategy includes: generating corresponding auxiliary segmentation results based on the intermediate feature maps output at each stage of the decoder of the inner ear segmentation network; calculating the auxiliary segmentation loss between each auxiliary segmentation result and the truth-labeled inner ear structure mask based on binary cross-entropy loss and Dice loss; adding the auxiliary segmentation loss to the main segmentation loss calculated based on the composite loss function to obtain the total loss; and updating the parameters of the inner ear segmentation network based on the total loss.

[0059] To further improve the segmentation performance of the inner ear's fine structures, this invention also introduces a deep supervision strategy. An auxiliary segmentation result is output at each stage of the decoder, and it is supervised and constrained using a ground truth mask. The total loss function... It consists of the main segmentation loss at the final output and the auxiliary segmentation losses of each intermediate layer, with the core computational unit being the binary cross-entropy loss. With Dice loss The weighted sum, the total loss function The calculation formula is Formula 21: (Formula 21) in, This represents the binary cross-entropy loss. This indicates Dice's loss. Represents the internal Ground Truth map (pixel values: target is 1, target is 0). The model predicts probability mappings with values ​​ranging from 0 to 1. Each auxiliary loss is calculated in the same way and weighted and summed with the main loss before being used for backpropagation, thereby promoting the effective fusion and optimization of features at each level of the decoder.

[0060] The reasoning process based on the pre-trained inner ear segmentation network includes: first, preprocessing the input ear U-HRCT images, including unifying the size to the size used during training and standardizing the pixel values ​​to ensure that the input and training data distributions are consistent; then, inputting the preprocessed images into the trained lightweight inner ear segmentation network, and directly outputting the corresponding inner ear structure segmentation results after forward propagation of the network. Figure 6 An example of inner ear segmentation is shown, from left to right: the original input image, the expert-annotated ground truth mask, and the segmentation result obtained from the inner ear segmentation network trained based on this invention. To clearly show the details, the segmented inner ear region is magnified in the figure. It can be seen that this invention can achieve complete and precise inner ear structure segmentation, and the results highly match the ground truth annotations, verifying the effectiveness of this invention.

[0061] The following is in conjunction with the appendix Figure 1-6 A specific embodiment of the technical solution of the present invention is given to further understand the method of the present invention.

[0062] Step 1: Construction of the Inner Ear Segmentation Task Dataset To achieve accurate segmentation of the inner ear structure, a dedicated inner ear structure segmentation dataset was first constructed. This embodiment utilizes ultra-high resolution computed tomography (U-HRCT) to acquire images, containing 86 sample data points. Each sample's original 3D image contains approximately 370 2D slices. All slices were meticulously annotated manually by experienced medical imaging experts to generate a segmentation mask (Ground Truth) for the inner ear structure. All images were uniformly resized to 512×512 pixels and randomly divided into a training set (69 cases) and a test set (17 cases) at an 8:2 ratio for model training and evaluation.

[0063] Step 2: Design of a lightweight inner ear segmentation network 2.1 Overall Network Architecture The overall architecture of the lightweight inner ear segmentation network constructed in this invention is as follows: Figure 2 As shown, the classic encoder-decoder (U-Net) structure is used as the basic framework.

[0064] The encoder consists of four stages for progressive downsampling to extract multi-level features. The number of output channels for each stage are {24, 48, 96, 192}. The first stage uses two cascaded depthwise separable convolutional blocks for processing, followed by downsampling; the second to fourth stages each use two cascaded depthwise separable convolutional blocks with dynamic routing mechanisms for processing, followed by downsampling. If the input image size is H×W×3, then the feature map size output from the i-th stage is... , where C=12.

[0065] Decoder: Symmetrical to encoder, it also consists of 4 stages. In each stage, the resolution of the input feature map is doubled and the number of channels is halved through 3×3 convolution and bilinear interpolation upsampling operations. Then, the features are refined through two cascaded depthwise separable convolutional blocks with dynamic routing mechanisms to align them with the scale of the feature map output by the corresponding stage of the encoder.

[0066] Bottleneck layer and attention mechanism: In the bottleneck layer between the encoder and decoder, nonlocal attention operations that decouple the channel and spatial dimensions are embedded to model global context dependencies and improve the model’s ability to locate and segment complex inner ear structures.

[0067] 2.2 Depthwise separable convolutional blocks with dynamic routing mechanisms The module structure is as follows: Figure 3 As shown, the specific processing steps include: (1) Dynamic routing operation to obtain the filtered key features Generate route weights: given input features First, global average pooling (GAP) is performed to obtain channel-level statistics. Then, 1×1 convolution dimensionality reduction, ReLU nonlinear activation, 1×1 convolution mapping, and Softmax normalization are performed sequentially to generate the routing weight vector. , where G is the number of groups. This process can be represented by the aforementioned Formula 1, and will not be elaborated further.

[0068] Generate grouped features: for input features Perform 1×1 grouped convolutions (with G groups), followed by batch normalization (BN), and finally reshape the features into G groups along the channel dimension to obtain the grouped features. Where C' = C / G. This process can be represented by the aforementioned formula 2, and will not be elaborated further.

[0069] Weighted fusion: based on route weight Grouping features Perform weighted summation to filter out key features. This process can be represented by the aforementioned Formula 3, and will not be elaborated further.

[0070] (2) Depthwise separable convolution operation yields multi-scale fused features. Key features Parallel input to two branches for multi-scale feature extraction: Branch 1: Employs a 3×3 depthwise separable convolution followed by BN and ReLU to output features. .

[0071] Branch 2: Employs concatenated 1×5 and 5×1 asymmetric depthwise separable convolutions, followed by BN and ReLU, to output features. .

[0072] The output features of the two branches are concatenated along the channel dimension to obtain multi-scale fused features. The above process can be represented by the aforementioned formula 4-6, and will not be repeated here.

[0073] (3) Channel attention enhancement processing and residual connection Multi-scale fusion features A 1×1 convolution is performed, and an efficient channel attention (ECA) module is used for enhancement. Simultaneously, the original input feature X is processed through a mapping function (such as a 1×1 convolution or identity mapping). Finally, the two are summed to obtain the final output feature of the module. This process can be represented by the aforementioned Formula 7, and will not be elaborated further.

[0074] 2.3 Nonlocal attention operations decoupled from channel and spatial dimensions The mechanism structure is as follows: Figure 4 As shown, it is implemented at the bottleneck layer and includes two parts: channel nonlocal attention and spatial nonlocal attention, which are implemented in parallel.

[0075] (1) Channel non-local attention operation to obtain channel semantic enhancement features For input features Query features are generated using 1×1 convolutions respectively. Q Key features K Then, it is subjected to global average pooling compression along the spatial dimension to obtain Q,K∈B×C′. Query features are then calculated. Q Key features K The similarity matrix is ​​normalized using Softmax to obtain the channel attention weights. Simultaneously, input features Map the result to a value V using another 1×1 convolution and adjust its shape. Utilize attention weights. Weighted aggregation of value V is performed, and the value is restored to its original form through inverse shape transformation. Enhanced features Compared with the original input features The channel semantic enhancement features are obtained by weighted fusion using learnable parameters λ. The above process can be represented by the aforementioned formula 8-13, and will not be repeated here.

[0076] (2) Spatial nonlocal attention operation to obtain spatial location enhancement features For input features 1×1 convolution transformations are performed along the height (h) and width (w) dimensions respectively, and horizontal query features are generated through dimension permute. and vertical query features Calculate separately High-dimensional attention weights are obtained based on the similarity to themselves. ,calculate The width-dimensional attention weight is obtained from the similarity with itself. At the same time, input features Mapped to horizontal value features and vertical direction value features . Utilize respectively , right , We perform weighted aggregation and restore the channel dimension through inverse permutation to obtain enhanced features. and The enhanced features from both directions are weighted and fused with the original input features using a learnable parameter γ to obtain the spatial location enhanced features. The above process can be represented by the aforementioned formulas 14-20, and will not be repeated here.

[0077] (3) Feature fusion Channel semantic enhancement features Spatial location enhancement features The data is concatenated and then processed through a 1×1 convolution, batch normalization (BN), and ReLU activation function to finally obtain the bottleneck layer output, which incorporates features with global context information.

[0078] Step 3: Training of the lightweight inner ear segmentation network The training process for a lightweight inner ear segmentation network can be found in [reference needed]. Figure 5 The training process specifically includes: 3.1 Training Environment and Settings The training process of the inner ear segmentation network of this invention can be implemented on the PyTorch open-source framework. The hardware environment is: Intel i7 CPU (3.6GHz), 12GB RAM, and NVIDIA GeForce GTX 1080 Ti GPU (11GB VRAM). Training uses the AdamW optimizer with an initial learning rate of 1e-3, a cosine annealing scheduling strategy (period t=20), momentum set to 0.9, and a weight decay coefficient of 1e-6. The batch size is set to 4, and the total number of training epochs is 300. During training, data augmentation strategies such as random horizontal flipping, normalization, random scaling, and random pruning are used to improve the model's generalization ability.

[0079] 3.2 Loss Function and Deep Supervision Strategy To improve the learning performance of the network, especially the features of the decoder, this embodiment adopts a deep supervision strategy.

[0080] Main segmentation loss: The loss between the network's final output and the ground truth labeled mask y is determined by the binary cross-entropy loss. and Dice loss The composite loss function Calculate the total loss function. The calculation formula is the aforementioned Formula 21, and will not be repeated here.

[0081] Auxiliary segmentation loss: At each stage of the decoder, an auxiliary segmentation result is generated from the intermediate feature map output by that stage. The same composite loss function described above is used. Calculate the loss between each auxiliary segmentation result and the true mask y.

[0082] Total loss: The main segmentation loss is weighted and summed with all auxiliary segmentation losses to obtain the total loss used for backpropagation, and the network parameters are updated based on this.

[0083] The network is iteratively trained using the above method until the model converges, resulting in a well-trained inner ear segmentation network.

[0084] Step 4: Inference of the Lightweight Inner Ear Segmentation Network The reasoning process for the lightweight inner ear segmentation network can be found in [reference needed]. Figure 5 Its reasoning process specifically includes: In practical applications, the ear CT images to be segmented are preprocessed, including size normalization to 512×512 and pixel value standardization, to make them consistent with the training data specifications. Subsequently, the preprocessed images are input into the trained inner ear segmentation network. After forward propagation, the network directly outputs the segmentation result probability map of the inner ear structure. The final segmentation mask can be obtained by thresholding (e.g., 0.5).

[0085] Step 5: Segmentation Results Figure 6 An example of inner ear segmentation is shown, with the original input image, expert-annotated ground truth mask, and the segmentation result obtained by the method of this embodiment shown from left to right. For clarity, the segmented inner ear region is magnified in the figure. It can be seen that the method proposed in this invention can achieve complete and precise segmentation of the inner ear structure, and the results are highly consistent with the ground truth mask, verifying the effectiveness of the method.

[0086] Finally, it should be noted that those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program for setting system parameters can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the above methods. The storage medium for the program can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. The above computer program embodiments can achieve the same or similar effects as any of the corresponding foregoing method embodiments.

[0087] Furthermore, the method disclosed in the embodiments of the present invention can also be implemented as a computer program executed by a processor, which may be stored in a computer-readable storage medium. When the computer program is executed by the processor, it performs the functions defined in the method disclosed in the embodiments of the present invention.

[0088] Furthermore, the above-described method steps and system units can also be implemented using a controller and a computer-readable storage medium for storing a computer program that enables the controller to perform the functions of the above-described steps or units.

[0089] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the functionality of various illustrative components, blocks, modules, circuits, and steps has been generally described. Whether this functionality is implemented as software or as hardware depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the functionality in various ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the embodiments disclosed herein.

[0090] The above are exemplary embodiments disclosed in this invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of this invention as defined by the claims. The functions, steps, and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. Furthermore, although the elements disclosed in the embodiments of this invention may be described or claimed individually, they may be understood as multiple unless explicitly limited to a singular number.

[0091] It should be understood that, as used herein, the singular form “a” is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, “and / or” refers to any and all possible combinations of one or more of the associated listed items.

[0092] The embodiment numbers disclosed in the above embodiments of the present invention are merely for description and do not represent the superiority or inferiority of the embodiments.

[0093] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0094] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

Claims

1. A lightweight inner ear segmentation method based on channel-space decoupled nonlocal attention, characterized in that, include: Acquire computed tomography images of the ear to be segmented; The image is input into the inner ear segmentation network to obtain the segmentation results of the inner ear structure; The inner ear segmentation network adopts an encoder-decoder structure, and performs feature processing through depthwise separable convolutions with dynamic routing mechanisms in both the encoding and decoding processes. It also performs nonlocal attention operations to decouple channel and spatial dimensions in the bottleneck layer between encoding and decoding.

2. The lightweight inner ear segmentation method based on channel-space decoupled nonlocal attention as described in claim 1, characterized in that, The feature processing via depthwise separable convolutions with dynamic routing mechanisms includes: Dynamically route the features input to the encoding or decoding process to obtain the filtered key features; Perform depthwise separable convolution operations on the key features to obtain multi-scale fused features; The multi-scale fusion features are subjected to channel attention enhancement processing to obtain enhanced features; The enhanced features are residually concatenated with the features input to the encoding or decoding process to obtain the output features of the encoding or decoding process.

3. The lightweight inner ear segmentation method based on channel-space decoupling nonlocal attention as described in claim 2, characterized in that, The dynamic routing operation on the features input to the encoding or decoding process to obtain the filtered key features includes: Global average pooling is performed on the features input to the encoding or decoding process. Then, the pooling results are sequentially subjected to convolutional dimensionality reduction, nonlinear activation, convolutional mapping, and normalization to generate routing weights. The features input to the encoding or decoding process are sequentially subjected to grouped convolution, batch normalization, and feature reshaping to obtain grouped features; The grouping features are weighted and fused based on the routing weights to obtain the filtered key features.

4. The lightweight inner ear segmentation method based on channel-space decoupling nonlocal attention as described in claim 3, characterized in that, The process of performing depthwise separable convolution on the key features to obtain multi-scale fused features includes: The key features are input into at least two parallel branches, and each branch is processed by convolution, batch normalization and nonlinear activation using depth-separable convolution kernels of different scales before output. The features output from all branches are concatenated to obtain multi-scale fused features.

5. The lightweight inner ear segmentation method based on channel-space decoupled nonlocal attention as described in claim 1, characterized in that, The nonlocal attention operation that decouples the channel and spatial dimensions includes: Channel nonlocal attention operation is performed on the features input to the bottleneck layer to obtain channel semantic enhancement features, and spatial nonlocal attention operation is performed on the features input to the bottleneck layer to obtain spatial location enhancement features. The channel semantic enhancement features are concatenated with the spatial location enhancement features, and then convolution, batch normalization and nonlinear activation are performed sequentially to obtain the features output by the bottleneck layer.

6. The lightweight inner ear segmentation method based on channel-space decoupled nonlocal attention as described in claim 5, characterized in that, The process of performing channel nonlocal attention operation on the features input to the bottleneck layer to obtain channel semantically enhanced features includes: The features input to the bottleneck layer are subjected to convolutional transformation to generate query features and key features, and the features input to the bottleneck layer are simultaneously mapped to values ​​that can be used for aggregation. The query features and the key features are spatially compressed, and the similarity between the compressed features is calculated to obtain the channel attention weights. Based on the channel attention weights, the values ​​are weighted and aggregated and subjected to inverse shape transformation to obtain initial channel enhancement features; The initial channel enhancement features are weighted and fused with the features input to the bottleneck layer to obtain channel semantic enhancement features.

7. The lightweight inner ear segmentation method based on channel-space decoupled nonlocal attention as described in claim 5, characterized in that, The step of performing a spatial nonlocal attention operation on the features input to the bottleneck layer to obtain spatially enhanced features includes: The features input to the bottleneck layer are subjected to convolution transformation and dimension permutation along the height and width dimensions of space, respectively, to generate query features in the horizontal and vertical directions, and at the same time, the features input to the bottleneck layer are mapped to value features in the horizontal and vertical directions. Calculate the similarity between query features in the horizontal direction and between query features in the vertical direction to obtain spatial attention weights in the height and width dimensions. Based on the spatial attention weights, the value features in the corresponding directions are weighted and aggregated, and the channel dimension is restored through inverse permutation operation to obtain the initial spatial enhancement features in the horizontal and vertical directions. The initial spatial enhancement features in the horizontal and vertical directions are weighted and fused with the features input to the bottleneck layer to obtain spatial location enhancement features.

8. The lightweight inner ear segmentation method based on channel-space decoupled nonlocal attention as described in claim 1, characterized in that, Also includes: Construct a dataset of computed tomography images of the ear containing labeled inner ear structure masks; The initial inner ear segmentation network is iteratively trained using the dataset. In each iteration, the prediction error is calculated based on a composite loss function consisting of binary cross-entropy loss and Dice loss, and the parameters of the inner ear segmentation network are updated based on the prediction error until the inner ear segmentation network converges, thus obtaining the inner ear segmentation network.

9. The lightweight inner ear segmentation method based on channel-space decoupling nonlocal attention as described in claim 8, characterized in that, Also includes: A deep supervision strategy is employed during the iterative training process.

10. The lightweight inner ear segmentation method based on channel-space decoupled nonlocal attention according to claim 9, characterized in that, The deep supervision strategy includes: Based on the intermediate feature maps output by the decoder at each stage of the inner ear segmentation network, corresponding auxiliary segmentation results are generated. Based on the binary cross-entropy loss and the Dice loss, the auxiliary segmentation loss between each of the auxiliary segmentation results and the true labeled inner ear structure mask is calculated respectively. The auxiliary segmentation loss is added to the main segmentation loss calculated based on the composite loss function to obtain the total loss; The parameters of the inner ear segmentation network are updated based on the total loss.

Images