Multi-level MRI segmentation methods for diagnosing tumors

Abstract

This application provides a multi-level MRI segmentation method for diagnosing tumors. In this application, the tumor type identification process is divided into three stages, from coarse to fine, using different models based on the U-Net framework. For 3D MRI scans of tumor patients, the first stage identifies the first human organ / tissue region using a first segmentation model; the second stage uses a binary segmentation model to identify the tumor region within the first human organ / tissue region; and the third stage uses a multi-class segmentation model to classify different tumors. This three-stage multi-level segmentation improves the accuracy of tumor classification, thereby contributing to the advancement of tumor treatment.

Description

Technical Field

[0001] This application relates to the field of image segmentation technology, and in particular to a multi-level MRI segmentation method for diagnosing tumors. Background Technology

[0002] Precise morphological quantification of tumors, including their size, location, and type, holds promise for enhancing monitoring protocols and optimizing treatment strategies. However, in clinical practice, identifying the various subtypes of tumors (such as spinal cord tumors) remains a significant challenge, sometimes leading to misdiagnosis and delays in subsequent treatment interventions. Summary of the Invention

[0003] This application provides a multi-level magnetic resonance imaging (MRI) segmentation method for diagnosing tumors.

[0004] In a first aspect, embodiments of this application provide a multi-level MRI segmentation method for diagnosing tumors, the method comprising:

[0005] The three-dimensional MRI of each cancer patient was preprocessed to obtain a standardized tumor dataset. The three-dimensional MRI included gadolinium-enhanced longitudinal relaxation time-weighted sagittal images (T1SC) and transverse relaxation time-weighted sagittal images (T2S).

[0006] The processed 3D MRI is used as input to the first segmentation model for the first stage of segmentation. The first segmentation model is used to segment the first human organ and tissue region in the 3D MRI.

[0007] The second-stage segmentation is performed based on T1SC, T2S, and the second segmentation model. The second segmentation model is a two-class tumor segmentation network and is used to segment the tumor region in the first human organ and tissue region.

[0008] The third-stage segmentation is performed based on T1SC, T2S, the first human organ and tissue region, the tumor region, and the third segmentation model. The third segmentation model is a multi-class tumor segmentation network. The third segmentation model is used to segment the tumor type in the tumor region and assign category labels to the tumor region.

[0009] Among them, the first segmentation model, the second segmentation model, and the third segmentation model are models based on the U-Net framework.

[0010] Optionally, the first human organ is the spinal cord, and the first, second, and third segmentation models are trained and optimized based on the spinal cord tumor dataset.

[0011] Optionally, the loss functions of the first segmentation model, the second segmentation model, and the third segmentation model include one or more of the following:

[0012] Dice Loss is used to quantify the similarity between predicted segmentation results and actual segmentation results.

[0013] Cross-entropy loss function is used to quantify the difference between correct and incorrect classifications at the voxel level of tumor regions;

[0014] The category consistency loss function is used to ensure that the tumor type in each tumor region is always classified as a single type.

[0015] Optional,

[0016] The dice loss function is:

[0017]

[0018] in Represents the dice loss function. Indicates the predicted segmentation result. p represents the true segmentation. i,k p is the output of the network's flexible maximum transfer function, representing the probability that the i-th voxel is classified into class k; g is a one-bit effective code corresponding to the ground truth segmentation map, and both p and g have dimensions N×K; i∈N represents the number of voxels in a given patch or batch; k∈K represents the total number of classes in the segmentation task; according to the ground truth segmentation, g is calculated only if the i-th voxel actually belongs to class k. i,k Only set it to 1; otherwise, g i,k Set to 0;

[0019] The cross-entropy loss function is:

[0020]

[0021] in Represents the cross-entropy loss function;

[0022] The category consistency loss function is:

[0023]

[0024] in The category consistency loss function represents the foreground. The category consistency loss function represents the background. This represents the category consistency loss function.

[0025] Optionally, the loss functions of the first and second segmentation models include the dice loss function and the cross-entropy loss function, while the loss function of the third segmentation model includes the dice loss function, the cross-entropy loss function, and the class consistency loss function.

[0026] Optionally, before performing the second-stage segmentation based on T1SC, T2S, and the second segmentation model, the method further includes:

[0027] Using the geometric center of the spinal cord region segmented by the first segmentation model as the center point of the key region, a first-size three-dimensional MRI is extracted from the T1SC, T2S and spinal cord regions as input for subsequent stages.

[0028] Optionally, the method further includes:

[0029] Based on the tumor types of the tumor regions segmented by the third segmentation model, the tumor types of the tumor regions are reassigned to the type containing the most voxels.

[0030] Optionally, preprocessing may include one or more of the following:

[0031] Resampling is used to modify the resolution of the 3D MRI to the first resolution;

[0032] Voxel clipping is used to cut the dimensions of a 3D MRI to a second dimension.

[0033] Scan intensity normalization is used to subtract the average intensity value from each voxel in a 3D MRI and divide the result by the standard deviation of the image intensity of the 3D MRI.

[0034] Image registration was performed to align T2S to T1SC using the spinal cord toolbox.

[0035] Optionally, the tumor regions from 3D MRI in the spinal cord tumor dataset are manually annotated with medical data.

[0036] Optionally, the spinal cord tumor dataset is based on three-dimensional MRI acquisitions of the vertebrae of multiple spinal cord tumor patients.

[0037] Secondly, embodiments of this application provide a tumor diagnostic device, comprising:

[0038] The preprocessing module is used to preprocess the three-dimensional MRI of each tumor patient to obtain a standardized tumor dataset. The three-dimensional MRI includes gadolinium-enhanced T1SC and T2S.

[0039] The first segmentation module is used to perform first-stage segmentation by taking the processed 3D MRI as input to the first segmentation model. The first segmentation model is used to segment the first human organ and tissue region in the 3D MRI.

[0040] The second segmentation module is used to perform second-stage segmentation based on T1SC, T2S and the second segmentation model. The second segmentation model is a two-class tumor segmentation network and is used to segment the tumor region in the first human organ and tissue region.

[0041] The third segmentation module is used to perform third-stage segmentation based on T1SC, T2S, the first human organ tissue region and the tumor region, as well as the third segmentation model. The third segmentation model is a multi-class tumor segmentation network. The third segmentation model is used to segment the tumor type in the tumor region and assign category labels to the tumor region.

[0042] Among them, the first segmentation model, the second segmentation model, and the third segmentation model are models based on the U-Net framework.

[0043] Optionally, the first human organ is the spinal cord, and the first, second, and third segmentation models are trained and optimized based on the spinal cord tumor dataset.

[0044] Optionally, the loss functions of the first segmentation model, the second segmentation model, and the third segmentation model include one or more of the following:

[0045] The dice loss function is used to quantify the similarity between the predicted segmentation results and the actual segmentation results;

[0046] Cross-entropy loss function is used to quantify the difference between correct and incorrect classifications at the voxel level of tumor regions;

[0047] The category consistency loss function is used to ensure that the tumor type in each tumor region is always classified as a single type.

[0048] Optionally, the dice loss function is:

[0049]

[0050] in Represents the dice loss function. Indicates the predicted segmentation result. p represents the true segmentation. i,k p is the output of the network's flexible maximum transfer function, representing the probability that the i-th voxel is classified into class k; g is a one-bit effective code corresponding to the ground truth segmentation map, and both p and g have dimensions N×K; i∈N represents the number of voxels in a given patch or batch; k∈K represents the total number of classes in the segmentation task; according to the ground truth segmentation, g is calculated only if the i-th voxel actually belongs to class k. i,k Only set it to 1; otherwise, g i,j Set to 0;

[0051] The cross-entropy loss function is:

[0052]

[0053] in Represents the cross-entropy loss function;

[0054] The category consistency loss function is:

[0055]

[0056] in The category consistency loss function represents the foreground. The category consistency loss function represents the background. This represents the category consistency loss function.

[0057] Optionally, the loss functions of the first and second segmentation models include the dice loss function and the cross-entropy loss function, while the loss function of the third segmentation model includes the dice loss function, the cross-entropy loss function, and the class consistency loss function.

[0058] Optionally, the tumor diagnostic device may also include:

[0059] The extraction module is used to extract a first-size three-dimensional MRI from the T1SC, T2S and spinal cord regions as input for subsequent stages before the second-stage segmentation is performed by the second segmentation module based on T1SC, T2S and the second segmentation model. This is done by using the geometric center of the spinal cord region segmented by the first segmentation model as the center point of the key region.

[0060] Optionally, the tumor diagnostic device may also include:

[0061] The reassignment module is used to reassign tumor types to tumor regions segmented based on the third segmentation model, with the tumor type containing the most voxels.

[0062] Optionally, preprocessing may include one or more of the following:

[0063] Resampling is used to modify the resolution of the 3D MRI to the first resolution;

[0064] Voxel clipping is used to cut the dimensions of a 3D MRI to a second dimension.

[0065] Scan intensity normalization is used to subtract the average intensity value from each voxel in a 3D MRI and divide the result by the standard deviation of the image intensity of the 3D MRI.

[0066] Image registration was performed to align T2S to T1SC using the spinal cord toolbox.

[0067] Optionally, the tumor regions from 3D MRI in the spinal cord tumor dataset are manually annotated with medical data.

[0068] Optionally, the spinal cord tumor dataset is based on three-dimensional MRI acquisitions of the vertebrae of multiple spinal cord tumor patients.

[0069] Thirdly, embodiments of this application provide an electronic device, including a memory, at least one processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method as described in any one of the first aspects above.

[0070] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method as described in any one of the first aspects above.

[0071] Fifthly, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to perform the method described in any one of the first aspects above.

[0072] In this application, the tumor type identification process is divided into three stages, from coarse to fine, using different models based on the U-Net framework. For 3D MRI scans of tumor patients, the first stage uses a first segmentation model to identify the first human organ / tissue region; the second stage uses a binary segmentation model to identify the tumor region within the first human organ / tissue region; and the third stage uses a multi-class segmentation model to classify different tumors. This multi-level segmentation based on three stages improves the accuracy of tumor classification, thereby contributing to the advancement of tumor treatment. Attached Figure Description

[0073] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0074] Figure 1A-1D This is an example diagram of the tumor region and tissue region provided in the embodiments of this application.

[0075] Figure 2 This is a flowchart illustrating a multi-level MRI segmentation method for diagnosing tumors provided in an embodiment of this application.

[0076] Figures 3A-3D These are three-dimensional MRI example images of four types of spinal cord tumors provided in the embodiments of this application.

[0077] Figures 4A-4C These are example images of the original T1SC MRI, T2S MRI, and T2S aligned to T1SC provided in the embodiments of this application.

[0078] Figures 5A-5C This is an example diagram of a multi-level MRI segmentation method for diagnosing tumors provided in an embodiment of this application.

[0079] Figures 6A-6BThis is a qualitative analysis result of the multi-level MRI segmentation method for diagnosing tumors provided in the embodiments of this application.

[0080] Figure 7 This is a schematic diagram of the structure of the tumor diagnostic device provided in the embodiments of this application.

[0081] Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0082] The technical solutions of the embodiments of the present invention will now be described with reference to the accompanying drawings.

[0083] In modern medicine, cancer is a major threat to life, especially spinal cord tumors. The spinal cord is a key component of the central nervous system, playing a multifaceted role in coordinating somatosensory and motor functions. Spinal cord tumors represent a unique category in the field of oncological diseases, significantly different from intracranial tumors and other malignancies such as lung or liver cancer. Spinal cord tumors significantly increase neurological morbidity and mortality. Precise morphological quantification of tumors, including their size, location, and type, holds promise for enhancing monitoring protocols and optimizing treatment strategies. This quantification requires segmenting intramedullary tumors and identifying them as their corresponding disease types. In clinical practice, identifying the various subtypes of spinal cord tumors and differentiating them from other pathologies remains a formidable challenge, sometimes leading to misdiagnosis and delays in subsequent treatment intervention.

[0084] Traditional manual segmentation of 3D structures remains time-consuming and labor-intensive, easily affected by intra- and inter-evaluator variability. Furthermore, the scientific community primarily focuses on brain tumor segmentation, such as the widely recognized Brain Tumor Segmentation Challenge (BraTS Challenge). In addition, convolutional neural networks (CNNs) have become powerful tools, achieving significant performance in brain tumor segmentation tasks. However, due to the inherent anatomical differences between the spinal cord and the brain, seamless application of these models in spinal cord tumor segmentation remains challenging. A key difference lies in the different fields of view of these neural structures. Notably, the average diameter of the spinal cord is approximately 13.3 ± 2.2 mm, while its length ranges from 420 to 450 mm. Figure 1A-1B (The spinal cord is represented by a black area surrounded by a white curve, and a spinal cord tumor is represented by a white area surrounded by a black curve.) In stark contrast, the brains of individuals exhibit a relatively consistent size, such as... Figure 1C-1D (Brain tumors are represented by white areas surrounded by black lines). Therefore, these anatomical variations need to be taken into account when using existing segmentation methods.

[0085] With the development of medical image segmentation technology, recent advances in neural networks have significantly improved the performance of medical image segmentation tasks, and the use of architectures similar to U-Net has become a popular trend in image segmentation. Combining attention mechanisms at different stages of the U-Net architecture can improve feature extraction and obtain more accurate image segmentation results. Transformer-based models have also made significant progress in medical image analysis. For example, the large-scale model SAM, pre-trained on 1 billion masks from 11 million natural images, has shown impressive zero-shot performance. Therefore, a large number of evaluation studies have rapidly emerged in the field of medical image segmentation.

[0086] Currently, research on spinal cord tumors mainly focuses on the segmentation of intraspinal tumors. One related technique proposes a dual-cascade 3D CNN for chordoma segmentation. Although both spinal tumors and intraspinal tumors are located within the spinal region, they exhibit different intensities and sizes and are juxtaposed with different tissue types (particularly bone tissue versus neuronal tissue). Therefore, due to these inherent differences, segmentation models designed for the spine are not applicable to the spinal cord. Another related technique attempts to use a two-level cascade architecture consisting of two U-Net models to separately locate the spinal cord and segment the tumor, but it has limitations in identifying the specific tumor type, and the dataset for this technique only includes three spinal cord tumor types from 343 patients.

[0087] Based on this Figure 2 The illustration shows an embodiment of this application that provides a multi-level magnetic resonance imaging (MRI) segmentation method for diagnosing tumors.

[0088] like Figure 2 As shown, the method includes steps S210-S240.

[0089] In step S210, the three-dimensional MRI of each cancer patient is preprocessed to obtain a standardized tumor dataset. The three-dimensional MRI includes gadolinium-enhanced T1SC and T2S. The first human organ tissue can be the spinal cord, brain, liver, lung, breast, etc., and the three-dimensional MRI of the cancer patient can be a three-dimensional MRI without prior treatment intervention.

[0090] In step S220, the processed 3D MRI is used as input to the first segmentation model for the first stage of segmentation. The first segmentation model is used to segment the first human organ tissue region in the MRI.

[0091] In step S230, a second-stage segmentation is performed based on T1SC, T2S, and a second segmentation model. The second segmentation model is a two-class tumor segmentation network used to segment the tumor region within the first human organ tissue region. T1SC and T2S can be used as inputs to the second segmentation model to segment the tumor region within the first human organ tissue region.

[0092] In step S240, a third-stage segmentation is performed based on T1SC, T2S, the first human organ tissue region, the tumor region, and a third segmentation model. The third segmentation model is a multi-class tumor segmentation network used to segment the tumor type within the tumor region and assign category labels to the tumor region. T1SC, T2S, the first human organ tissue region, and the tumor region can be used as inputs to the third segmentation model to segment the tumor type.

[0093] Among them, the first segmentation model, the second segmentation model, and the third segmentation model are models based on the U-Net framework, and there is no specific limitation. The U-Net framework is nnU-Net as an example.

[0094] In this application, the tumor type identification process is divided into three stages, from coarse to fine, using different models based on the U-Net framework. For MRI scans of tumor patients, the first stage uses a first segmentation model to identify the first human organ / tissue region; the second stage uses a binary segmentation model to identify the tumor region within the first human organ / tissue region; and the third stage uses a multi-class segmentation model to classify different tumors. This three-stage, multi-level segmentation improves the accuracy of tumor classification, thereby contributing to the advancement of tumor treatment.

[0095] Optionally, the first human organ tissue is the spinal cord, and the first, second, and third segmentation models can be trained and optimized based on a spinal cord tumor dataset. That is, multi-level segmentation can be performed using the first, second, and third segmentation models trained and optimized based on the spinal cord tumor dataset to diagnose the type of spinal cord tumor. This helps improve the accuracy of spinal cord tumor diagnosis.

[0096] Optionally, the loss functions of the first segmentation model, the second segmentation model, and the third segmentation model may include one or more of the following:

[0097] The Dice Loss function is used to quantify the similarity between predicted segmentation results and actual segmentation results;

[0098] Cross-entropy loss function is used to quantify the difference between correct and incorrect classifications at the voxel level of tumor regions;

[0099] The category consistency loss function is used to ensure that the tumor type in each tumor region is always classified as a single type.

[0100] Based on the above loss function, the training efficiency and model accuracy of the first segmentation model, the second segmentation model, and the third segmentation model can be improved.

[0101] Optionally, the Dice Loss function can be:

[0102]

[0103] in This represents the Dice Loss function. Indicates the predicted segmentation result. p represents the true segmentation. i,k p is the output of the network's flexible maximum transfer function, representing the probability that the i-th voxel is classified into class k; g is a one-bit effective code corresponding to the ground truth segmentation map, and both p and g have dimensions N×K; i∈N represents the number of voxels in a given patch or batch; k∈K represents the total number of classes in the segmentation task; according to the ground truth segmentation, g is calculated only if the i-th voxel actually belongs to class k. i,k Only set it to 1; otherwise, g i,k Set to 0;

[0104] The cross-entropy loss function can be:

[0105]

[0106] in Represents the cross-entropy loss function;

[0107] The class consistency loss function can be:

[0108]

[0109] in The category consistency loss function represents the foreground. The category consistency loss function represents the background. This represents the category consistency loss function.

[0110] Optionally, the loss functions of the first and second segmentation models may include the Dice Loss function and the cross-entropy loss function, while the loss function of the third segmentation model may include the Dice Loss function, the cross-entropy loss function, and the class consistency loss function.

[0111] Optionally, before performing the second-stage segmentation based on T1SC, T2S, and the second segmentation model, Figure 2 The method shown may also include:

[0112] The geometric center of the spinal cord region segmented by the first segmentation model is used as the center point of the key region. A first-size 3D MRI image is extracted from the T1SC, T2S, and spinal cord regions as input for subsequent stages. The first size can be determined according to actual needs and is not limited to any specific value. Using the cropped key region can alleviate the class imbalance problem and accelerate training and inference in subsequent stages.

[0113] Optional, Figure 2 The method shown may also include:

[0114] Based on the tumor type segmented by the third segmentation model, the tumor type of the tumor region is reassigned to the type containing the most voxels. This can enhance the segmentation output.

[0115] Optionally, the preprocessing of tumor data may include one or more of the following:

[0116] Resampling is used to modify the resolution of the 3D MRI to a primary resolution. The primary resolution can be determined based on actual needs and is not limited to a specific value.

[0117] Voxel clipping is used to cut the dimensions of a 3D MRI scan to a second dimension. The second dimension can be determined according to actual needs and is not limited to any specific size.

[0118] Scan intensity normalization is used to subtract the average intensity value from each voxel in a 3D MRI and divide the result by the standard deviation of the image intensity in the 3D MRI.

[0119] Image registration is used to align T2S images to T1SC images using the spinal toolbox. For example, T2S images can be aligned to T1SC images using affine transformations to ensure accurate registration.

[0120] Preprocessing can standardize tumor images, which is beneficial for training models.

[0121] Optionally, the tumor regions from 3D MRI in the spinal cord tumor dataset are manually annotated with medical data, and these manually annotated tumors can be used as actual values ​​for training and optimizing the second segmentation model.

[0122] Optionally, the spinal cord tumor dataset is based on three-dimensional MRI acquisitions of the vertebrae of multiple spinal cord tumor patients.

[0123] The following is combined Figures 3A-3D , Figures 4A-4C as well as Figures 5A-5C Taking spinal cord tumors as an example, the multi-level MRI segmentation method for diagnosing tumors according to embodiments of this application will be described in detail.

[0124] 1. Data collection and labeling

[0125] The dataset used for model training and optimization comprises 3D MRIs of 653 patients collected from October 2017 to September 2023, specifically designed for preoperative evaluation of spinal cord tumors without prior treatment intervention. This dataset is notable for its comprehensive vertebral presentation (including cervical, thoracic, and lumbar regions) and includes four major spinal cord tumor variants: meningioma (247 patients), astrocytoma (101 patients), ependymoma (203 patients), and hemangioblastoma (102 patients). 3D MRIs of the four tumor types are shown below. Figures 3A-3D As shown.

[0126] As shown in Table 1, the spinal cord tumor dataset was divided into two distinct subsets: a training set containing 80% of the subjects (521 patients) and a test set containing the remaining 20% ​​of the subjects (132 patients). Each patient's 3D MRI consisted of gadolinium-enhanced T1SC and T2S scans, with professionally hand-annotated tumors. Two independent neuroradiologists manually segmented the real tumor regions in the gadolinium-enhanced T1SC. If there was controversy regarding the segmentation of tumor regions, the final decision was made by a senior neuroradiologist to ensure the highest fidelity and accuracy of the data annotation. The raw resolution of the sagittal MRI scans in this dataset varied, ranging from 0.34 to 1.06 mm in in-plane resolution and from 1.5 to 8 mm in slice thickness. Each MRI image contained a varying number of slices scanned in the sagittal plane, ranging from a minimum of 9 to a maximum of 36 slices.

[0127] Table 1. Detailed information on the spinal cord tumor dataset.

[0128]

[0129] 2. Data Preprocessing

[0130] To standardize the dataset, the sagittal images were resampled to 0.47 mm in the anteroposterior and vertical directions, and 3.3 mm in the horizontal direction. The resampled images were voxel-cropped to form a bounding box measuring 512 mm × 256 mm × 64 mm, centered in the field of view for focus analysis. The intensity of each scan was first normalized by subtracting the average intensity value from each voxel and then dividing the result by the standard deviation of image intensity. Subsequently, image registration was performed using the spinal cord toolbox (SCT) from related techniques. Figures 4A-4C As shown, the process involves aligning the T2S image to the T1SC image via an affine transformation to ensure accurate registration.

[0131] 3. Multi-level MRI segmentation

[0132] 3.1 Spinal cord segmentation (stage one, also known as stage I)

[0133] like Figure 5A As shown, T2S(T2) and T1SC(T1) are registered to align anatomical structures. The first stage uses a network based on the U-Net framework (taking the nnU-Net framework as an example). With T1SC(T1) and the registered Spinal cord segmentation as input Then, using the geometric center of the predicted spinal cord mask (i.e., the spinal cord region) as the center point of the key region or region of interest (ROI), from T1, and An MRI scan measuring 512 mm × 256 mm × 64 mm was extracted and designated as follows: and Using cropped Regions of Interest (ROIs) can alleviate class imbalance and accelerate training and inference in subsequent stages.

[0134] 3.2 Secondary Division of Spinal Cord Tumors (Stage II)

[0135] Decomposing the multi-class segmentation task into a binary segmentation stage (Stage II) and a classification stage (Stage III) transforms a complex problem into two simpler tasks, allowing each network to focus on its corresponding function, thereby improving overall performance.

[0136] Figure 5B This illustrates the second phase of the framework, in which the cropped MRI... and Used as input. This stage trains a class II tumor segmentation network. (Based on the U-Net framework (taking the nnU-Net framework as an example)), it focuses solely on segmenting the tumor region as the foreground. However, it does not classify tumors into corresponding types. This approach simplifies multi-class segmentation by focusing on the location of tumors rather than their category.

[0137] 3.3 Segmentation of Multiple Types of Spinal Cord Tumors (Stage III)

[0138] In obtaining a mask for predicted spinal cord tumors The third phase aims to differentiate between different tumor types and accordingly... Assign category labels. For example... Figure 5C As shown, except and In addition, it also includes and The inputs are masks for the spinal cord and tumor regions, respectively. Then, a five-class segmentation network is trained. This process distinguishes between four tumor types and backgrounds, ultimately assigning category labels to tumor regions. Furthermore, to ensure the homogeneity of prediction results for individual patients, a novel category consistency loss is introduced in the third stage to prevent inconsistent labels from appearing in the predictions for a single patient.

[0139] 3.4 Training Loss Function

[0140] (1) Dice Loss function

[0141] The Dice Loss function was used in all three stages of segmentation, which quantifies the predicted segmentation. and true segmentation The similarity between them is represented as The Dice Loss function used in this paper is a variety of variant extensions introduced in related technologies, and can be implemented as follows:

[0142]

[0143] in This represents the Dice Loss function. Indicates the predicted segmentation result. p represents the true segmentation. i,k This is the softmax output of the network, representing the probability that the i-th voxel is classified into class k. `g` corresponds to the one-hot encoding of the ground truth segmentation map. Both `p` and `g` have dimensions N×K; `i∈N` represents the number of voxels in a given patch or batch; `k∈K` represents the total number of classes in the segmentation task; according to the ground truth segmentation, `g` is calculated only if the i-th voxel actually belongs to class k. i,k Only set it to 1; otherwise, g i,k Set to 0.

[0144] (2) Cross-entropy loss function

[0145] Furthermore, cross-entropy (CE) loss was used in all three stages, denoted as... It quantifies the difference between correct and incorrect classification at the voxel level.

[0146] The cross-entropy loss function is:

[0147]

[0148] (2) Category Consistency (CC) Loss Function

[0149] To ensure each segmented tumor region It is always classified into a single category, and a new category consistency loss function is proposed. Failure to comply with this standard will be subject to penalties.

[0150] The category consistency loss function is:

[0151]

[0152] in The category consistency loss function represents the foreground. The category consistency loss function represents the background. This represents the category consistency loss function.

[0153] Overall, the training loss function for the three stages is:

[0154]

[0155] It is worth noting that all training of the above models was performed on a single NVIDIA 3090 graphics processing unit (GPU). The training process was designed to run for 1000 epochs at a learning rate of 0.01. In post-processing, for Within the foreground, we reassign its predicted category to the category containing the most voxels, thereby enhancing the segmentation output.

[0156] The following is combined Figures 6A-6B Tables 2-3 and 3 analyze the performance of the segmentation methods in the embodiments of this application.

[0157] (1) Benchmark Models and Indicators

[0158] The segmentation method in this application is compared with state-of-the-art models in medical image segmentation, including the first model nnFormer, the second model 3D UX-Net, the third model Swin UNETR, and the fourth model nnUNet. Performance is evaluated using the evaluation metrics Dice Coefficient and Hausdorff Distance (HD) from the relevant paper M&Ms Challenge.

[0159] (2) Results

[0160] The quantitative analysis results are shown in Table 2. Based on Table 2, the segmentation method of this embodiment improves performance by 8.4% compared to the previously best-performing model nnUNet. When applied to different types of tumors, the DiceCoefficient of the method of this embodiment shows improvements of 0.8%, 7.8%, 13.6%, and 11.3%, respectively. Regarding Hausdorff Distance, as shown in Table 2, the method of this embodiment outperforms nnUNet by 49.2 mm. Figures 6A-6B The qualitative analysis results show that the method of the present application embodiments achieves high accuracy in predicting tumor type and region.

[0161] Table 2 Results of Quantitative Analysis of Parameters

[0162]

[0163] (3) Ablation test

[0164] Based on ablation experiments, the effectiveness of the method in this application embodiment was evaluated by changing the inputs and strategies used. The results of the ablation experiments are shown in Table 3, where bold indicates the best performance. As can be seen from Table 3:

[0165] 1) Model A uses unprocessed T1SC as input without any pruning, resulting in the worst performance.

[0166] 2) Model B enhances the method of this application embodiment by cropping T1SC MRI around the spinal cord region, thereby improving performance (4.5%).

[0167] 3) Model C combines registered T2S with predicted two-class segmentation masks, which helps the model focus on tumor location, thereby further refining segmentation accuracy (2.5%).

[0168] 4) Model D further adopts the category consistency loss of the embodiments of this application, which can guarantee the homogeneity of the segmentation results and achieve the best performance.

[0169] Table 3. Dice coefficients (%) of ablation experiments under different inputs and strategies.

[0170]

[0171] The methods provided in the embodiments of this application have been described in detail above. The apparatus provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0172] See Figure 7 This is a schematic diagram of the structure of a tumor diagnostic device provided in an embodiment of this application. Figure 7 The tumor diagnostic device 700 shown includes:

[0173] The preprocessing module 710 is used to preprocess the three-dimensional MRI of each tumor patient to obtain a standardized tumor dataset. The three-dimensional MRI includes gadolinium-enhanced T1SC and T2S.

[0174] The first segmentation module 720 is used to perform first-stage segmentation by taking the processed three-dimensional MRI as input to the first segmentation model. The first segmentation model is used to segment the first human organ tissue region in the MRI.

[0175] The second segmentation module 730 is used to perform second-stage segmentation based on T1SC, T2S and the second segmentation model. The second segmentation model is a two-class tumor segmentation network. The second segmentation model is used to segment the tumor region in the first human organ and tissue region.

[0176] The third segmentation module 740 is used to perform third-stage segmentation based on T1SC, T2S, the first human organ tissue region and the tumor region, as well as the third segmentation model. The third segmentation model is a multi-class tumor segmentation network. The third segmentation model is used to segment the tumor type in the tumor region and assign category labels to the tumor region.

[0177] Among them, the first segmentation model, the second segmentation model, and the third segmentation model are models based on the U-Net framework.

[0178] Optionally, the first human organ is the spinal cord, and the first, second, and third segmentation models are trained and optimized based on the spinal cord tumor dataset.

[0179] Optionally, the loss functions of the first segmentation model, the second segmentation model, and the third segmentation model include one or more of the following:

[0180] The Dice Loss function is used to quantify the similarity between predicted segmentation results and actual segmentation results;

[0181] Cross-entropy loss function is used to quantify the difference between correct and incorrect classifications at the voxel level of tumor regions;

[0182] The category consistency loss function is used to ensure that the tumor type in each tumor region is always classified as a single type.

[0183] Optionally, the Dice Loss function is:

[0184]

[0185] in This represents the Dice Loss function. Indicates the predicted segmentation result. p represents the true segmentation. i,kp is the output of the network's flexible maximum transfer function, representing the probability that the i-th voxel is classified into class k; g is a one-bit effective code corresponding to the ground truth segmentation map, and both p and g have dimensions N×K; i∈N represents the number of voxels in a given patch or batch; k∈K represents the total number of classes in the segmentation task; according to the ground truth segmentation, g is calculated only if the i-th voxel actually belongs to class k. i,k Only set it to 1; otherwise, g i,k Set to 0;

[0186] The cross-entropy loss function is:

[0187]

[0188] in Represents the cross-entropy loss function;

[0189] The category consistency loss function is:

[0190]

[0191] in The category consistency loss function represents the foreground. The category consistency loss function represents the background. This represents the category consistency loss function.

[0192] Optionally, the loss functions for the first and second segmentation models include the Dice Loss function and the cross-entropy loss function, while the loss function for the third segmentation model includes the Dice Loss function, the cross-entropy loss function, and the class consistency loss function.

[0193] Optionally, the tumor diagnostic device 700 also includes:

[0194] The extraction module 750 is used to extract a first-size three-dimensional MRI from the T1SC, T2S and spinal cord regions as input for subsequent stages before the second segmentation module 730 performs the second-stage segmentation based on T1SC, T2S and the second segmentation model, using the geometric center of the spinal cord region segmented by the first segmentation model as the center point of the key region.

[0195] Optionally, the tumor diagnostic device 700 also includes:

[0196] The reassignment module 760 is used to reassign tumor types to tumor regions segmented based on the third segmentation model, with the tumor type containing the most voxels.

[0197] Optionally, preprocessing may include one or more of the following:

[0198] Resampling is used to modify the resolution of the 3D MRI to the first resolution;

[0199] Voxel clipping is used to cut the dimensions of a 3D MRI to a second dimension.

[0200] Scan intensity normalization is used to subtract the average intensity value from each voxel of the MRI and divide the result by the standard deviation of the image intensity of the three-dimensional MRI.

[0201] Image registration was performed to align T2S to T1SC using the spinal cord toolbox.

[0202] Optionally, the tumor regions from 3D MRI in the spinal cord tumor dataset are manually annotated with medical data.

[0203] Optionally, the spinal cord tumor dataset is based on three-dimensional MRI acquisitions of the vertebrae of multiple spinal cord tumor patients.

[0204] This application also provides an electronic device. For example... Figure 8 As shown, the electronic device 800 includes: at least one processor 801, a memory 802, and a computer program 803 stored in the memory 802 and executable on the at least one processor 801. When the processor 801 executes the computer program 803, it implements the channel selection method provided in this application.

[0205] For example, computer program 803 may be divided into one or more modules / units, one or more of which are stored in memory 802 and executed by processor 801 to complete this application. One or more modules / units may be a series of computer program instruction segments capable of performing corresponding functions, which describe the execution process of the computer program in electronic device 800.

[0206] Those skilled in the art will understand that Figure 8 This is merely an example of electronic device 800 and does not constitute a limitation on electronic device. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device 800 may also include input / output devices, network access devices, buses, etc.

[0207] The processor 801 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0208] The memory 802 can be an internal storage unit of the electronic device 800, such as a hard disk or RAM of the electronic device 800. The memory 802 can also be an external storage device of the electronic device 800, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 800. Furthermore, the memory 802 can include both internal and external storage units of the electronic device 800. The memory 802 is used to store computer programs and other programs and data required by the electronic device 800. The memory 802 can also be used to temporarily store data that has been output or will be output.

[0209] The electronic device 800 provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0210] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the various method embodiments described above.

[0211] This application provides a computer program product that, when run on an electronic device, enables the electronic device to perform the steps described in the various method embodiments above.

[0212] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above method embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate form. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a photographic device / electronic device, a recording medium, a computer memory, ROM (read-only memory), RAM (random access memory), CD-ROM (compact disc read-only memory), magnetic tape, floppy disk, and optical data storage devices. The computer-readable storage medium mentioned in this application can be a non-volatile storage medium; in other words, it can be a non-transient storage medium.

[0213] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0214] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the corresponding application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each corresponding application, but such implementation should not be considered beyond the scope of this application.

[0215] In the embodiments provided in this application, it should be understood that the disclosed apparatus / device and method can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0216] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0217] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0218] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0219] Furthermore, in the description of this application and the appended claims, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0220] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A multi-level magnetic resonance imaging (MRI) segmentation method for diagnosing tumors, characterized in that, The method comprises: preprocessing three-dimensional MRI of each tumor patient to obtain a standardized tumor dataset, wherein the three-dimensional MRI comprises gadolinium-enhanced longitudinal relaxation time weighted sagittal image T1SC and transverse relaxation time weighted sagittal image T2S; performing first stage segmentation by taking the processed three-dimensional MRI as input of a first segmentation model, wherein the first segmentation model is used to segment a first human organ tissue region in the three-dimensional MRI; performing second stage segmentation based on the T1SC, T2S and a second segmentation model, wherein the second segmentation model is a two-class tumor segmentation network, and the second segmentation model is used to segment a tumor region in the first human organ tissue region; performing third stage segmentation based on the T1SC, T2S, the first human organ tissue region, the tumor region and a third segmentation model, wherein the third segmentation model is a multi-class tumor segmentation network, and the third segmentation model is used to segment a tumor type of the tumor region and assign a class label to the tumor region; wherein the first segmentation model, the second segmentation model and the third segmentation model are models based on a U-shaped network U-Net framework, the loss function of the first segmentation model and the second segmentation model comprises a dice loss function and a cross-entropy loss function, and the loss function of the third segmentation model comprises the dice loss function, the cross-entropy loss function and a class consistency loss function; the dice loss function is: wherein said denotes the dice loss function, said denotes the predicted segmentation result, said denotes the true segmentation, said is the flexible max-pooling function output of the network, denoting the probability that the -th voxel is classified as class; said is a one-hot encoding corresponding to the true segmentation map, said and have both dimension ; said denotes the number of voxels in a given patch or batch; said denotes the total number of classes in the segmentation task; said is set to 1 only if the -th voxel does belong to class according to the true segmentation; otherwise said is set to 0; the cross-entropy loss function is: , wherein represents the cross-entropy loss function; the class consistency loss function is: ， ， ， wherein the a class consistency loss function representing foreground, the a class consistency loss function representing background, the the class consistency loss function.

2. The method of claim 1, wherein, the first human organ tissue is a spinal cord, and the first segmentation model, the second segmentation model and the third segmentation model are trained and optimized based on a spinal cord tumor dataset.

3. The method of claim 2, wherein, Before the second stage segmentation based on the T1SC, T2S and the second segmentation model, the method further comprises: taking the geometric center of the spinal cord region segmented by the first segmentation model as the center point of the key region, and extracting a three-dimensional MRI of a first size from the T1SC, T2S and the spinal cord region as input of a subsequent stage.

4. The method of claim 3, wherein, The method further comprises: reassigning a tumor type to the tumor region based on the tumor type of the tumor region segmented by the third segmentation model, so that the tumor type contains the largest number of voxels.

5. The method of claim 4, wherein, The preprocessing comprises one or more of the following: resampling, used to modify the resolution of the three-dimensional MRI to a first resolution; voxel cropping, used to crop the size of the three-dimensional MRI to a second size; scan intensity normalization, used to subtract the average intensity value of each voxel of the three-dimensional MRI and divide the result by the standard deviation of the image intensity of the three-dimensional MRI; image registration, used to align the T2S to the T1SC by using a spinal cord toolbox.

6. The method of claim 5, wherein, The tumor region of the three-dimensional MRI in the spinal cord tumor dataset is manually annotated by a doctor.

7. The method of claim 6, wherein, The spinal cord tumor dataset is obtained based on three-dimensional MRI of the entire vertebra of a plurality of spinal cord tumor patients.

Images