MRI (Magnetic Resonance Imaging) image eye muscle segmentation method based on VMUnet + + model

Abstract

The invention provides an MRI (Magnetic Resonance Imaging) image eye muscle segmentation method based on a VMUnet + + model, which is used for assisting thyroid-related eye disease diagnosis. The method comprises the following steps: acquiring eye medical image data of a thyroid-related eye disease patient; preprocessing the medical image data; an improved VMUnet + + network architecture is constructed; taking the preprocessed eye image data as the input of a VMUnet + + model, taking the eye muscle region multi-class segmentation result as the output, and adopting a CeDiceLoss loss function to train a segmentation model to obtain a trained eye muscle segmentation model; and inputting an eye medical image of a patient to be diagnosed into the trained model to obtain an accurate eye muscle segmentation result, thereby providing auxiliary support for clinical diagnosis of thyroid-related eye diseases. By introducing a Vision Mama mechanism, the modeling ability of the model for the complex form of the eye muscle is enhanced, and the precision and robustness of eye muscle segmentation are remarkably improved by combining the dense connection and deep supervision strategy of UNet + +.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a deep learning-based method and system for automatic ocular muscle segmentation for the auxiliary diagnosis of thyroid-related eye diseases. Background Technology

[0002] Thyroid-associated ophthalmopathy (TAO), also known as Graves' ophthalmopathy, is an autoimmune disease associated with thyroid dysfunction. Its main pathological features include inflammation, edema, enlargement, and even fibrosis of the extraocular muscles and orbital soft tissues. These lesions can lead to proptosis, diplopia, and limited eye movement; in severe cases, they can even compress the optic nerve, causing irreversible visual impairment. Therefore, accurate assessment of the extraocular muscle condition is crucial for the early diagnosis, disease classification, treatment planning, and monitoring of TAO efficacy.

[0003] Magnetic resonance imaging (MRI), due to its excellent soft tissue resolution, has become the "gold standard" for assessing extraocular muscle lesions, clearly displaying the morphology, volume, and inflammatory activity of the extraocular muscles. However, in current clinical practice, the analysis of MRI images mainly relies on manual interpretation and delineation by radiologists or ophthalmologists. This traditional method has many limitations: First, it is inefficient, as manually segmenting and measuring multiple extraocular muscles (such as the medial rectus, lateral rectus, superior rectus, and inferior rectus) is time-consuming and laborious; second, it is highly subjective, with segmentation results affected by factors such as the doctor's experience and fatigue level, leading to differences between different doctors and even between the same doctor at different times, lacking consistency and repeatability; finally, it lacks quantitative precision, as early, subtle changes in volume or morphological abnormalities of the extraocular muscles are difficult to accurately capture through manual observation, easily leading to missed diagnoses or delayed diagnosis.

[0004] In recent years, deep learning models, represented by UNet, have achieved great success in the field of medical image segmentation. UNet and its variants, through their classic encoder-decoder structure and skip connections, can effectively fuse multi-scale features and perform excellently in various segmentation tasks. However, standard UNet-like models still face challenges when dealing with structures like the eye muscles—slender, complex in shape, and whose boundaries are easily confused with surrounding adipose tissue. On the one hand, traditional convolutional neural networks (CNNs) have limited receptive fields, making it difficult to effectively capture the long-range dependencies and global morphological features of the eye muscles in three-dimensional space. On the other hand, existing models still have room for optimization in their feature fusion mechanisms to better address the morphological diversity of the eye muscles in the course of TAO (Transcatheter Osteoarthritis).

[0005] To overcome the aforementioned technical challenges, there is an urgent need for an automated, high-precision, and efficient method for ocular muscle segmentation, thereby providing objective and quantifiable imaging indicators for clinical practice. Therefore, this paper proposes an improved segmentation network that integrates long-range dependency modeling capabilities and multi-scale features, which has significant clinical value and practical implications for improving the diagnosis and treatment of thyroid-related ophthalmopathy. Summary of the Invention

[0006] This invention addresses the shortcomings of existing technologies by providing a method, system, and storage medium for ocular muscle segmentation based on VM-UNet++. This invention constructs a novel VM-UNet++ network architecture by combining the state-space model of Vision Mamba (VMamba) with the dense skip connections and deep supervision strategy of UNet++. This architecture can effectively capture the long-range dependencies and global morphological features of ocular muscles in medical images. Furthermore, through deep fusion of multi-scale features, it significantly improves the segmentation accuracy for complex boundaries and subtle lesion features of ocular muscles.

[0007] To achieve the above-mentioned objectives, the technical solution adopted by the present invention is as follows:

[0008] An eye muscle segmentation method based on VM-UNet++ includes the following steps:

[0009] S1. Obtain ocular medical imaging data of patients with thyroid-associated ophthalmopathy, in the form of three-dimensional medical images in NIfTI compressed file format;

[0010] S2. Preprocessing of medical image data, including standardizing image data to the [0,1] range, processing multi-category label formats, random data augmentation, and resizing;

[0011] S3. Construct an improved VM-UNet++ network architecture that integrates Vision Mamba's long-distance dependency modeling capabilities with UNet++'s multi-scale feature fusion mechanism;

[0012] S4. The preprocessed eye image data is used as the input to the VM-UNet++ model, and the multi-class segmentation results of the eye muscle region are used as the output. The CeDiceLoss loss function is used to train the segmentation model.

[0013] S5. Input the ocular medical images of the patient to be diagnosed into the trained VM-UNet++ model to obtain accurate ocular muscle segmentation results.

[0014] Further, step S1 includes the following sub-steps:

[0015] S101. Collect three-dimensional ocular medical imaging data of patients with thyroid-related ophthalmopathy, ensuring that the data format is NIfTI compressed file format;

[0016] S102. Perform quality checks on the image data and remove images with severe artifacts or poor quality.

[0017] S103. Randomly divide the image data into training set, validation set and test set in a ratio of 8:1:1.

[0018] Further, step S2 includes the following sub-steps:

[0019] S201. Perform intensity standardization on the three-dimensional medical image, normalizing the pixel values ​​to the range of [0,1].

[0020] S202. Process multi-category labels, convert floating-point labels into integer categories and limit them to a valid range (0-22 categories), and achieve fine segmentation of 23 categories of eye muscles;

[0021] S203. Perform random data augmentation, including random rotation, flipping, scaling, and elastic deformation;

[0022] S204. Adjust the image size to the fixed size required by the network input;

[0023] S205. Perform slice extraction to convert the three-dimensional image into a two-dimensional slice sequence.

[0024] Furthermore, in step S3, constructing the improved VM-UNet++ network architecture specifically involves:

[0025] S301. Construct the Vision Mamba encoder and use the state-space model to model long-distance dependencies.

[0026] S302. Design a multi-scale dense jump connection structure to achieve effective fusion of features at different levels;

[0027] S303. Introduce an intermediate module based on the Mamba architecture between the encoder and decoder, combining low-level and high-level feature details.

[0028] S304. A deep supervision strategy is adopted, and auxiliary losses are added to multiple decoding layers.

[0029] Furthermore, the CeDiceLoss loss function is defined as follows:

[0030] L total =α×L CE +β×L Dice

[0031] Among them, L CE For multivariate cross-entropy loss, L Dice For Dice loss, α and β are weighting factors used to balance the contributions of the two losses.

[0032] This invention also discloses an eye muscle segmentation system based on VM-UNet++, which can be used to implement the above-mentioned eye muscle segmentation method, specifically including:

[0033] The data acquisition module is used to collect and manage three-dimensional ocular medical imaging data of patients with thyroid-associated eye diseases;

[0034] The data preprocessing module performs standardization, label conversion, data augmentation, and resizing of medical images.

[0035] The VM-UNet++ network module integrates the Vision Mamba and UNet++ architectures to achieve high-precision eye muscle segmentation;

[0036] The model training module uses preprocessed data to train the VM-UNet++ segmentation model;

[0037] The segmentation prediction module segments the extraocular muscle region in new ocular medical images;

[0038] The results display module visualizes the segmentation results and provides auxiliary information for clinical diagnosis.

[0039] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0040] 1. Enhanced long-distance dependency modeling capability: By introducing Vision Mamba as the core of the encoder, and utilizing its state-space model and selective scanning mechanism, the problem of limited receptive field in traditional CNNs is effectively overcome. It can capture global morphological information of slender structures such as eye muscles, and improve the ability to understand complex structures.

[0041] 2. Improve the efficiency of multi-scale feature fusion: By drawing on the dense skip connections and redesigned intermediate modules of UNet++, this invention achieves deeper and more comprehensive feature fusion between the encoder and decoder, effectively combining high-level semantic information with low-level detailed features, thereby accurately delineating the boundaries of the eye muscles.

[0042] 3. Improved segmentation accuracy and robustness: Combining the global modeling capabilities of Vision Mamba and the fine feature fusion of UNet++, this invention demonstrates higher accuracy and robustness in ocular muscle segmentation tasks, especially when dealing with ocular muscles with irregular shapes and blurred boundaries caused by lesions, where it outperforms traditional segmentation models.

[0043] 4. Optimize the training process: The CeDiceLoss hybrid loss function is adopted, which effectively balances pixel-level classification accuracy and region segmentation similarity, and alleviates the problem of imbalance between the foreground (eye muscle) and background regions in medical images; at the same time, the introduction of deep supervision strategy accelerates model convergence and improves the model's generalization ability. Attached Figure Description

[0044] Figure 1 This is a flowchart of the eye muscle segmentation method based on VM-UNet++ according to an embodiment of the present invention;

[0045] Figure 2 This is a schematic diagram of the VM-UNet++ network model structure according to an embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram of the Vision Mamba module structure according to an embodiment of the present invention;

[0047] Figure 4 This is a schematic diagram of the dense skip connection structure of UNet++ according to an embodiment of the present invention;

[0048] Figure 5 This is an example diagram of ocular MRI image input according to an embodiment of the present invention;

[0049] Figure 6 This is an example diagram showing the output of eye muscle segmentation results in an embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and examples.

[0051] like Figure 1 As shown, this invention provides an eye muscle segmentation method based on VM-UNet++, comprising the following steps:

[0052] S1. Obtain ocular medical imaging data of patients with thyroid-associated ophthalmopathy, in the form of three-dimensional medical images in NIfTI compressed file format;

[0053] S2. Preprocess medical image data;

[0054] S3. Construct an improved VM-UNet++ network architecture;

[0055] S4. Train the VM-UNet++ segmentation model;

[0056] S5. Input the ocular medical image to be segmented into the trained model to obtain the segmentation result.

[0057] In one specific embodiment, the detailed execution process of steps S1 and S2 is as follows:

[0058] S101. Collect ocular magnetic resonance imaging (MRI) data from multiple patients with thyroid-associated ophthalmopathy, in the form of three-dimensional medical images in NIfTI compressed files.

[0059] S102. Perform quality checks on the image data to exclude data with severe motion artifacts or poor scan quality.

[0060] S103. Randomly shuffle the selected image data and divide it into training set, validation set and test set in a ratio of 8:1:1.

[0061] S201. Intensity standardization is performed on the three-dimensional medical images, and the voxel values ​​are linearly normalized to the range of [0,1] to eliminate imaging differences caused by different devices.

[0062] S202. Process the multi-category labels manually drawn by the doctor, round the floating-point labels to integer categories and limit them to a valid range (0-22 categories), and maintain the multi-category label format to achieve fine segmentation of 23 categories of eye muscles, where category 0 is the background and categories 1-22 are different eye muscle regions.

[0063] S203. Perform online random data augmentation on the training set, including random rotation, horizontal / vertical flipping, scaling, and elastic deformation within a certain range, to improve the model's generalization ability.

[0064] S204. Adjust the image size to the fixed size required for network input, such as 512×512 pixels.

[0065] S205. Extract the three-dimensional NIfTI compressed image data along a specific axis (such as the coronal plane) into a two-dimensional slice sequence for subsequent model training.

[0066] like Figure 2 As shown, the VM-UNet++ network model constructed in this invention mainly consists of a Vision Mamba encoder, dense hop connection paths, and a decoder.

[0067] like Figure 3 As shown, the Vision Mamba encoder constructed in step S3 has the VSS (Visual StateSpace) Block at its core. This module utilizes a state-space model (SSM) and a selective scan mechanism (SS2D) to capture long-range dependencies in the image. The input feature map is first flattened into a sequence, and then processed by scanning in four directions (up, down, left, and right), effectively capturing global morphological information of slender structures such as eye muscles, overcoming the problem of limited receptive field in traditional convolution.

[0068] like Figure 4As shown, the dense skip connection structure designed in step S3 draws inspiration from UNet++. Dense feature fusion paths are constructed between the encoder and decoder, and within the decoder itself. Feature maps from different levels of the encoder (e.g., X0,0, X1,0, etc.) are upsampled and concatenated to achieve deep fusion with features from the corresponding levels of the decoder. This design allows the decoder to utilize both high-level semantic information and low-level detailed texture information when generating segmentation maps, thus more accurately restoring the boundaries of the eye muscles. Simultaneously, prediction results are output and auxiliary losses are calculated at multiple intermediate nodes (e.g., X0,1, X0,2, etc.) to achieve deep supervision and accelerate model convergence.

[0069] In one specific embodiment, the CeDiceLoss hybrid loss function used in step S4 is defined as follows:

[0070] The Dice loss function measures the overlap between the predicted segmentation map and the ground truth label, and it is defined as follows:

[0071]

[0072] Where P is the probability map predicted by the model, G is the true binary mask, and ε is a very small smoothing constant used to prevent the denominator from being zero.

[0073] Multivariate cross-entropy loss (CE Loss) penalizes the prediction results at the pixel level.

[0074] The final hybrid loss function is:

[0075] L total =α×L CE +β×L Dice

[0076] Here, α and β are weighting factors used to adjust the relative importance of the two losses. In this embodiment, α = 0.5 and β = 0.5 can be set.

[0077] like Figure 5 and Figure 6 As shown, in step S5, a slice of an ocular MRI image to be segmented is prepared. Figure 5 The input is fed into the trained VM-UNet++ model, and the model outputs a segmentation map of the same size as the input. Figure 6 The colored areas precisely mark the location and outline of the eye muscles.

[0078] In another embodiment of the present invention, an eye muscle segmentation system based on VM-UNet++ is provided. This system can be used to implement the above-described eye muscle segmentation method, specifically including:

[0079] The data acquisition module is used to collect and manage ocular medical image data in NIfTI compressed file format;

[0080] The data preprocessing module performs operations such as standardization, label conversion, data augmentation, resizing, and slice extraction.

[0081] The VM-UNet++ network module, as the core segmentation engine, integrates the VisionMamba and UNet++ architectures to achieve high-precision eye muscle segmentation;

[0082] The model training module uses preprocessed data and the CeDiceLoss loss function to train the VM-UNet++ segmentation model and saves the optimal weights.

[0083] The segmentation prediction module loads the trained model and automatically segments the ocular muscle region in new ocular medical images.

[0084] The results display module overlays the segmentation results onto the original image for visualization, providing doctors with intuitive diagnostic assistance information.

[0085] In another embodiment of the present invention, a terminal device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for the operation of an ocular muscle segmentation method for MRI images.

[0086] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory). This computer-readable storage medium is a memory device in a terminal device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and extended storage media supported by the terminal device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device.

[0087] One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the ocular muscle segmentation method for an MRI image in the above embodiments; one or more instructions in the computer-readable storage medium are loaded and executed by a processor.

[0088] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0089] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0090] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0091] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0092] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the implementation methods of the present invention, and should be understood that the scope of protection of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of the present invention.

Claims

1. A method for MRI image eye muscle segmentation based on a VM_Unet++ model, characterized in that, The method comprises the following steps: S1, obtaining eye medical image data of a patient with thyroid-associated ophthalmopathy, in the form of three-dimensional medical images in NIfTI compressed file format; S2, preprocessing the medical image data, including normalizing the image data to the range [0, 1], processing multi-class label format, random data augmentation, and size adjustment; S3, constructing an improved VM-UNet++ network architecture, combining the long-distance dependency modeling capability of Vision Mamba and the multi-scale feature fusion mechanism of UNet++; S4, taking the preprocessed eye image data as the input of the VM-UNet++ model, taking the multi-class segmentation result of the eye muscle region as the output, and training the segmentation model using the CeDiceLoss loss function; S5, inputting the eye medical image of the patient to be diagnosed into the trained VM-UNet++ model to obtain accurate eye muscle segmentation results.

2. The eye muscle segmentation method of claim 1, wherein: The step S1 comprises the following sub-steps: S101, collecting three-dimensional eye medical image data of a patient with thyroid-associated ophthalmopathy, ensuring that the data format is NIfTI compressed file format; S102, performing quality inspection on the image data to remove images with severe artifacts or poor quality; S103, randomly dividing the image data into a training set, a validation set, and a test set in a ratio of 8:1:

1.

3. The eye muscle segmentation method of claim 1, wherein: The step S2 comprises the following sub-steps: S201, intensity standardization of three-dimensional medical images, normalizing pixel values to the range [0, 1]; S202, processing multi-class labels, converting floating-point labels to integer classes and limiting them to the valid range (0-22 classes) to achieve 23-class fine segmentation of eye muscles; S203, random data augmentation, including random rotation, flipping, scaling, and elastic deformation; S204, adjusting the image size to the fixed size required by the network input; S205, slice extraction, converting three-dimensional NIfTI format images to two-dimensional slices, preferentially selecting the middle slice or the first valid slice, and expanding the single-channel grayscale image to a three-channel RGB format to adapt to the network input.

4. The eye muscle segmentation method of claim 1, wherein: In step S3, the improved VM-UNet++ network architecture is constructed as follows: S301, constructing a VisionMamba encoder, modeling long-distance dependencies using a state space model; S302, designing a multi-scale dense skip connection structure to effectively fuse features at different levels; S303, introducing an intermediate module based on the Mamba architecture between the encoder and the decoder, combining low-level and high-level feature details; S304, using a deep supervision strategy to add auxiliary losses at multiple decoding layers.

5. The eye muscle segmentation method of claim 4, wherein: The VisionMamba encoder comprises: A plurality of VisionMamba blocks, each block containing a state space model layer, layer normalization, and residual connection; The state space model layer uses a selective scanning mechanism to effectively handle the two-dimensional nature of visual data; Four different scanning directions are used to process input features, enhancing the direction invariance of the model.

6. The eye muscle segmentation method of claim 1, wherein: The CeDiceLoss loss function is defined as follows: L total = a x L CE + b x L Dice wherein L CE is a multi-way cross-entropy loss, L Dice is a Dice loss, and a and b are weight factors.

7. The eye muscle segmentation method of claim 6, wherein: The Dice loss function is defined as follows: Wherein, P is a predicted value, G is a true value, and ε is a smoothing factor.

8. The eye muscle segmentation method of claim 1, wherein: The step S4 includes the following sub-steps: S401, initializing the VM-UNet++ network parameters, and loading the pre-trained Vision Mamba weight; S402, setting the training hyperparameters, including the learning rate, batch size, and training rounds; S403, using the Adam optimizer to train the model, and adopting the cosine annealing learning rate scheduling strategy; S404, performing model validation during the training process, and saving the model weight with the best performance on the validation set.

9. A VM-UNet++ based eye muscle segmentation system, characterized in that: The system can be used to implement the eye muscle segmentation method according to any one of claims 1-8, and specifically includes: a data acquisition module for collecting and managing three-dimensional eye medical image data of patients with thyroid-related eye diseases; a data preprocessing module for standardizing, label converting, data enhancing, and size adjusting the medical images; a VM-UNet++ network module for fusing the Vision Mamba and UNet++ architectures to realize high-precision eye muscle segmentation; a model training module for training the VM-UNet++ segmentation model using the preprocessed data; a segmentation prediction module for segmenting the eye muscle region of a new eye medical image; and a result display module for visualizing the segmentation result.

Images