Multi-stage MRI segmentation method for diagnosing tumors

Abstract

The invention provides a multi-stage MRI segmentation method for diagnosing tumors. In the application, through different models based on the U-Net framework, the identification process of the tumor type is divided into three stages from coarse to fine. For three-dimensional MRI of a tumor patient, in the first stage, a first human organ tissue area is recognized through a first segmentation model, in the second stage, a tumor area in the first human organ tissue area is recognized through a second-class segmentation model, and in the third stage, different tumors are classified through a multi-class segmentation model. And on the basis of multi-stage segmentation of three stages, the accuracy of tumor classification is improved, so that tumor treatment is facilitated.

Description

Technical Field

[0001] The present application relates to the technical field of image segmentation, and in particular to a multi-level MRI segmentation method for diagnosing tumors. Background Art

[0002] Accurate morphological quantification of tumors, including size, location, and type, is expected to enhance monitoring programs and optimize treatment planning strategies. However, in the clinical field, distinguishing the various subtypes of tumors (such as spinal cord tumors) remains a daunting challenge, sometimes leading to misdiagnosis and delays in subsequent therapeutic interventions. Summary of the invention

[0003] The embodiment of the present application provides a multi-level magnetic resonance imaging (MRI) segmentation method for diagnosing tumors.

[0004] In a first aspect, an embodiment of the present application provides a multi-level MRI segmentation method for diagnosing a tumor, the method comprising:

[0005] The three-dimensional MRI of each tumor patient was preprocessed to obtain a standardized tumor data set. The three-dimensional MRI included gadolinium-enhanced longitudinal relaxation time-weighted sagittal images (T1SC) and transverse relaxation time-weighted sagittal images (T2S).

[0006] The processed three-dimensional MRI is used as an input of a first segmentation model to perform a first stage segmentation, and the first segmentation model is used to segment a first human organ tissue region in the three-dimensional MRI;

[0007] Performing second-stage segmentation based on T1SC, T2S and a second segmentation model, where the second segmentation model is a two-class tumor segmentation network, and the second segmentation model is used to segment the tumor region in the first human organ tissue region;

[0008] The third stage segmentation is performed based on T1SC, T2S, the first human organ tissue region and the tumor region and the third segmentation model. The third segmentation model is a multi-class tumor segmentation network. The third segmentation model is used to segment the tumor type of the tumor region and assign a category label to the tumor region.

[0009] Among them, the first segmentation model, the second segmentation model, and the third segmentation model are models based on the U-net framework.

[0010] Optionally, the first human organ tissue is a spinal cord, and the first segmentation model, the second segmentation model, and the third segmentation model are trained and optimized based on a spinal cord tumor dataset.

[0011] Optionally, the loss functions of the first segmentation model, the second segmentation model, and the third segmentation model include one or more of the following:

[0012] Dice Loss, which is used to quantify the similarity between the predicted segmentation results and the true segmentation results;

[0013] The cross entropy loss function is used to quantify the difference between correct and misclassified tumor regions at the voxel level;

[0014] Class consistency loss function is used to ensure that the tumor type of each tumor region is always classified as a single type.

[0015] Optional,

[0016] The dice loss function is:

[0017]

[0018] in represents the dice loss function, Represents the predicted segmentation result, represents the true segmentation, p i,k is the flexible maximum transfer function output of the network, which indicates the probability that the i-th voxel is classified into class k; g is a single-bit effective code corresponding to the true segmentation map, and the dimensions of p and g are both N×K; i∈N, represents the number of voxels in a given patch or batch; k∈K, represents the total number of categories in the segmentation task; according to the true segmentation, g is only considered if the i-th voxel does belong to class k. i,k is set to 1; otherwise g i,k Set to 0;

[0019] The cross entropy loss function is:

[0020]

[0021] in represents the cross entropy loss function;

[0022] The category consistency loss function is:

[0023]

[0024] in represents the category consistency loss function of the foreground, Represents the category consistency loss function of the background, represents the category consistency loss function.

[0025] Optionally, the loss functions of the first segmentation model and the second segmentation model include a dice loss function and a cross entropy loss function, and the loss function of the third segmentation model includes a dice loss function, a cross entropy loss function and a category consistency loss function.

[0026] Optionally, before performing the second stage segmentation based on T1SC, T2S and the second segmentation model, the method further includes:

[0027] Using the geometric center of the spinal cord region segmented by the first segmentation model as the center point of the key region, the first-scale 3D MRI is extracted from the T1SC, T2S and spinal cord regions as the input for the subsequent stages.

[0028] Optionally, the method further includes:

[0029] Based on the tumor type of the tumor region segmented by the third segmentation model, the tumor type of the tumor region is reassigned as the type containing the largest number of voxels.

[0030] Optionally, the preprocessing includes one or more of the following:

[0031] Resampling, for modifying the resolution of the three-dimensional MRI to the first resolution;

[0032] Voxel cropping, used to crop the size of the three-dimensional MRI to a second size;

[0033] Scan intensity normalization is used to subtract the mean intensity value from each voxel of the three-dimensional MRI and divide the result by the standard deviation of the image intensity of the three-dimensional MRI;

[0034] Image registration for aligning T2S to T1SC using the Spinal Cord Toolbox.

[0035] Optionally, the tumor regions of the 3D MRI in the spinal cord tumor dataset are manually annotated by medical experts.

[0036] Optionally, the spinal cord tumor dataset is based on full vertebral 3D MRI acquisitions of multiple spinal cord tumor patients.

[0037] In a second aspect, an embodiment of the present application provides a tumor diagnosis device, comprising:

[0038] A preprocessing module is used to preprocess the three-dimensional MRI of each tumor patient to obtain a standardized tumor data set, where the three-dimensional MRI includes gadolinium-enhanced T1SC and T2S;

[0039] A first segmentation module, used to perform first stage segmentation using the processed three-dimensional MRI as input of a first segmentation model, wherein the first segmentation model is used to segment a first human organ tissue region in the three-dimensional MRI;

[0040] A second segmentation module is used to perform second-stage segmentation based on T1SC, T2S and a second segmentation model, the second segmentation model is a two-type tumor segmentation network, and the second segmentation model is used to segment the tumor region in the first human organ tissue region;

[0041] A third segmentation module is used to perform a third stage segmentation based on T1SC, T2S, the first human organ tissue region and the tumor region and a third segmentation model, wherein the third segmentation model is a multi-class tumor segmentation network, and the third segmentation model is used to segment the tumor type of the tumor region and assign a category label to the tumor region;

[0042] Among them, the first segmentation model, the second segmentation model, and the third segmentation model are models based on the U-Net framework.

[0043] Optionally, the first human organ tissue is a spinal cord, and the first segmentation model, the second segmentation model, and the third segmentation model are trained and optimized based on a spinal cord tumor dataset.

[0044] Optionally, the loss functions of the first segmentation model, the second segmentation model, and the third segmentation model include one or more of the following:

[0045] Dice loss function, which is used to quantify the similarity between the predicted segmentation results and the true segmentation results;

[0046] The cross entropy loss function is used to quantify the difference between correct and misclassified tumor regions at the voxel level;

[0047] Class consistency loss function is used to ensure that the tumor type of each tumor region is always classified as a single type.

[0048] Optionally, the dice loss function is:

[0049]

[0050] in represents the dice loss function, Represents the predicted segmentation result, represents the true segmentation, p i,k is the flexible maximum transfer function output of the network, which indicates the probability that the i-th voxel is classified into class k; g is a single-bit effective code corresponding to the true segmentation map, and the dimensions of p and g are both N×K; i∈N, represents the number of voxels in a given patch or batch; k∈K, represents the total number of categories in the segmentation task; according to the true segmentation, g is only considered if the i-th voxel does belong to class k. i,k is set to 1; otherwise g i,j Set to 0;

[0051] The cross entropy loss function is:

[0052]

[0053] in represents the cross entropy loss function;

[0054] The category consistency loss function is:

[0055]

[0056] in represents the category consistency loss function of the foreground, Represents the category consistency loss function of the background, represents the category consistency loss function.

[0057] Optionally, the loss functions of the first segmentation model and the second segmentation model include a dice loss function and a cross entropy loss function, and the loss function of the third segmentation model includes a dice loss function, a cross entropy loss function and a category consistency loss function.

[0058] Optionally, the tumor diagnostic device further includes:

[0059] The extraction module is used to extract the first-size three-dimensional MRI from T1SC, T2S and the spinal cord area as the input of the subsequent stage by using the geometric center of the spinal cord area segmented by the first segmentation model as the center point of the key area before the second segmentation module performs the second stage segmentation based on T1SC, T2S and the second segmentation model.

[0060] Optionally, the tumor diagnostic device further includes:

[0061] The reallocation module is used to reallocate the tumor type of the tumor region to a type containing the largest number of voxels based on the tumor type of the tumor region segmented by the third segmentation model.

[0062] Optionally, the preprocessing includes one or more of the following:

[0063] Resampling, for modifying the resolution of the three-dimensional MRI to the first resolution;

[0064] Voxel cropping, used to crop the size of the three-dimensional MRI to a second size;

[0065] Scan intensity normalization is used to subtract the mean intensity value from each voxel of the three-dimensional MRI and divide the result by the standard deviation of the image intensity of the three-dimensional MRI;

[0066] Image registration for aligning T2S to T1SC using the Spinal Cord Toolbox.

[0067] Optionally, the tumor regions of the 3D MRI in the spinal cord tumor dataset are manually annotated by medical experts.

[0068] Optionally, the spinal cord tumor dataset is based on full vertebral 3D MRI acquisitions of multiple spinal cord tumor patients.

[0069] In a third aspect, an embodiment of the present application provides an electronic device comprising a memory, at least one processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a method as described in any one of the above-mentioned first aspects is implemented.

[0070] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method as described in any one of the above-mentioned first aspects is implemented.

[0071] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes any one of the methods described in the first aspect.

[0072] In this application, the tumor type recognition process is divided into three stages from coarse to fine through different models based on the U-Net framework. For the three-dimensional MRI of tumor patients, the first stage uses the first segmentation model to identify the first human organ tissue region, the second stage uses the two-class segmentation model to identify the tumor region in the first human organ tissue region, and the third stage uses the multi-class segmentation model to classify different tumors. The multi-level segmentation based on the three stages improves the accuracy of tumor classification, which helps to promote tumor treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for describing the embodiments are briefly introduced below.

[0074] Figures 1A-1D This is an example diagram of a tumor area and a tissue area provided in an embodiment of the present application.

[0075] Figure 2 It is a flowchart of a multi-level MRI segmentation method for diagnosing tumors provided in an embodiment of the present application.

[0076] Figures 3A-3D These are three-dimensional MRI example images of four types of spinal cord tumors provided in the embodiments of the present application.

[0077] Figures 4A-4C 1 is an example diagram of the original T1SC MRI, T2S MRI, and T2S aligned to T1SC provided in the embodiments of the present application.

[0078] Figures 5A-5C This is an example diagram of a multi-level MRI segmentation method for diagnosing tumors provided in an embodiment of the present application.

[0079] Figure 6A-6BThis is a qualitative analysis result of the multi-level MRI segmentation method for diagnosing tumors provided in an embodiment of the present application.

[0080] Figure 7 It is a schematic diagram of the structure of the tumor diagnostic device provided in the embodiment of the present application.

[0081] Figure 8 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0082] The technical solution of the embodiment of the present invention will be described below in conjunction with the accompanying drawings in the embodiment of the present invention.

[0083] In modern medicine, tumors are a major threat to life, especially spinal cord tumors. The spinal cord is a key component of the central nervous system and plays a multifaceted role in coordinating somatosensory perception and motor function. Spinal cord tumors represent a unique category in the field of neoplastic diseases, which is significantly different from intracranial tumors and other malignancies (such as lung cancer or liver cancer). Spinal cord tumors significantly increase neurological morbidity and mortality. Accurate morphological quantification of tumors, including tumor size, location, and type, is expected to enhance monitoring programs and optimize treatment planning strategies. Such quantification requires segmentation and identification of intramedullary tumors as the corresponding disease type. In the clinical field, it remains a daunting challenge to distinguish the various subtypes of spinal cord tumors and distinguish them from other pathologies, sometimes leading to misdiagnosis and delays in subsequent therapeutic interventions.

[0084] Traditional manual segmentation of 3D structures remains time-consuming and laborious, susceptible to intra- and inter-rater variability. And the scientific community has mainly focused on brain tumor segmentation, such as the widely recognized brain tumor segmentation challenge (BraTS Challenge). In addition, convolutional neural networks (CNNs) have become powerful tools and have achieved remarkable performance in brain tumor segmentation tasks. However, due to the inherent anatomical differences between the spinal cord and the brain, the seamless application of these models in spinal cord tumor segmentation remains elusive. A key difference lies in the different fields of view of these neural structures. Notably, the average diameter of the spinal cord is approximately 13.3±2.2 mm, while its length ranges from 420 to 450 mm, as shown in Figure 2. Figure 1A-1B (The spinal cord is the black area surrounded by the white curve, and the spinal cord tumor is the white area surrounded by the black curve.) In stark contrast, the brain shows relatively consistent size between individuals, such as Figure 1C-1D (Brain tumors are white areas surrounded by black lines.) Therefore, these anatomical changes need to be taken into account using existing segmentation methods.

[0085] With the development of medical image segmentation technology, the latest progress of neural networks has significantly improved the performance of medical image segmentation tasks, and the use of U-Net-like architectures has become a popular trend in image segmentation. Incorporating attention mechanisms at different stages of the U-Net architecture can improve feature extraction and obtain more accurate image segmentation results. Transformer-based models have also made significant progress in medical image analysis. Take the large-scale model SAM as an example, which can be pre-trained on 1 billion masks from 11 million natural images and shows impressive zero-shot performance. As a result, a large number of evaluation studies have rapidly emerged in the field of medical image segmentation.

[0086] Currently, research on spinal cord tumors mainly focuses on the segmentation of intraspinal tumors. A related technology proposes a dual-cascade 3D (3D) CNN for chordoma segmentation. Although spinal tumors and intraspinal tumors are both located in the spinal region, they exhibit different intensities and sizes and are juxtaposed with different tissue types (especially bone tissue and neuronal tissue). Therefore, due to these inherent differences, the segmentation model designed for the spine is not suitable for the spinal cord. Another related technology attempts to use a two-level cascade architecture consisting of two U-Net models to locate the spinal cord and segment the tumor, respectively, but it is insufficient in determining the specific type of the tumor, and the data set of this technology only contains three types of spinal cord tumors from 343 patients.

[0087] Based on this, Figure 2 The embodiment of the present application provides a multi-level magnetic resonance imaging (MRI) segmentation method for diagnosing tumors.

[0088] like Figure 2 As shown, the method includes steps S210-S240.

[0089] In step S210, the three-dimensional MRI of each tumor patient is preprocessed to obtain a standardized tumor data set, wherein the three-dimensional MRI includes gadolinium-enhanced T1SC and T2S. The first human organ tissue may be a spinal cord, brain, liver, lung, breast, etc., and the three-dimensional MRI of the tumor patient may be a three-dimensional MRI without prior treatment intervention.

[0090] In step S220, the processed three-dimensional MRI is used as an input of a first segmentation model to perform a first stage of segmentation. The first segmentation model is used to segment a first human organ tissue region in the MRI.

[0091] In step S230, a second stage segmentation is performed based on T1SC, T2S and a second segmentation model, the second segmentation model is a two-type tumor segmentation network, and the second segmentation model is used to segment the tumor region in the first human organ tissue region. T1SC and T2S can be used as inputs of the second segmentation model to further segment the tumor region in the first human organ tissue region.

[0092] In step S240, the third stage segmentation is performed based on T1SC, T2S, the first human organ tissue region and the tumor region and the third segmentation model. The third segmentation model is a multi-class tumor segmentation network. The third segmentation model is used to segment the tumor type of the tumor region and assign a category label to the tumor region. T1SC, T2S, the first human organ tissue region and the tumor region can be used as inputs of the third segmentation model to segment the tumor type.

[0093] Among them, the first segmentation model, the second segmentation model, and the third segmentation model are models based on the U-Net framework, and are not specifically limited. The U-Net framework takes nnU-Net as an example.

[0094] In this application, the tumor type recognition process is divided into three stages from coarse to fine through different models based on the U-Net framework. For the MRI of tumor patients, the first stage uses the first segmentation model to identify the first human organ tissue region, the second stage uses the two-class segmentation model to identify the tumor region in the first human organ tissue region, and the third stage uses the multi-class segmentation model to classify different tumors. The multi-level segmentation based on the three stages improves the accuracy of tumor classification, which helps to promote tumor treatment.

[0095] Optionally, the first human organ tissue is a spinal cord, and the first segmentation model, the second segmentation model, and the third segmentation model can be trained and optimized based on a spinal cord tumor dataset. That is, multi-level segmentation can be performed to diagnose the type of spinal cord tumor by using the first segmentation model, the second segmentation model, and the third segmentation model trained and optimized based on the spinal cord tumor dataset. Based on this, it is helpful to improve the accuracy of spinal cord tumor diagnosis.

[0096] Optionally, the loss functions of the first segmentation model, the second segmentation model, and the third segmentation model may include one or more of the following:

[0097] Dice Loss function, used to quantify the similarity between the predicted segmentation results and the true segmentation results;

[0098] The cross entropy loss function is used to quantify the difference between correct and misclassified tumor regions at the voxel level;

[0099] Class consistency loss function is used to ensure that the tumor type of each tumor region is always classified as a single type.

[0100] Based on the above loss function, the training efficiency and model accuracy of the first segmentation model, the second segmentation model, and the third segmentation model can be improved.

[0101] Optionally, the Dice Loss function can be:

[0102]

[0103] in represents the Dice Loss function, Represents the predicted segmentation result, represents the true segmentation, p i,k is the flexible maximum transfer function output of the network, which indicates the probability that the i-th voxel is classified into class k; g is a single-bit effective code corresponding to the true segmentation map, and the dimensions of p and g are both N×K; i∈N, represents the number of voxels in a given patch or batch; k∈K, represents the total number of categories in the segmentation task; according to the true segmentation, g is only considered if the i-th voxel does belong to class k. i,k is set to 1; otherwise g i,k Set to 0;

[0104] The cross entropy loss function can be:

[0105]

[0106] in represents the cross entropy loss function;

[0107] The category consistency loss function can be:

[0108]

[0109] in represents the category consistency loss function of the foreground, Represents the category consistency loss function of the background, represents the category consistency loss function.

[0110] Optionally, the loss functions of the first segmentation model and the second segmentation model may include a Dice Loss function and a cross entropy loss function, and the loss function of the third segmentation model may include a Dice Loss function, a cross entropy loss function, and a category consistency loss function.

[0111] Optionally, before performing the second stage segmentation based on T1SC, T2S and the second segmentation model, Figure 2 The method shown may also include:

[0112] The geometric center of the spinal cord region segmented by the first segmentation model is used as the center point of the key region, and the first-size 3D MRI is extracted from the T1SC, T2S and spinal cord regions as the input for the subsequent stages. The first size can be determined according to actual needs and is not limited. Using the cropped key region can alleviate the problem of class imbalance and accelerate the training and reasoning in the subsequent stages.

[0113] Optional, Figure 2 The method shown may also include:

[0114] Based on the tumor type of the tumor region segmented by the third segmentation model, the tumor region is reassigned a tumor type as the type containing the largest number of voxels. Based on this, the segmentation output can be enhanced.

[0115] Optionally, the preprocessing of tumor data may include one or more of the following:

[0116] Resampling is used to modify the resolution of the three-dimensional MRI to a first resolution. The first resolution can be determined according to actual needs and is not specifically limited.

[0117] Voxel cropping is used to crop the size of the three-dimensional MRI to a second size. The second size can be determined according to actual needs and is not specifically limited.

[0118] Scan intensity normalization is used to subtract the mean intensity value from each voxel of the 3D MRI and divide the result by the standard deviation of the image intensity of the 3D MRI.

[0119] Image registration for aligning T2S to T1SC using the Spinal Cord Toolbox. For example, a T2S image can be aligned to a T1SC image using an affine transformation to ensure accurate registration.

[0120] Based on preprocessing, tumor images can be standardized, which is beneficial for training models.

[0121] Optionally, the tumor region of the three-dimensional MRI in the spinal cord tumor dataset is manually annotated by medical methods, and the manual tumor annotations can be used as actual values ​​to train and optimize the second segmentation model.

[0122] Optionally, the spinal cord tumor dataset is based on full vertebral 3D MRI acquisitions of multiple spinal cord tumor patients.

[0123] Combine the following Figures 3A-3D , Figures 4A-4C as well as Figures 5A-5C Taking spinal cord tumor as an example, the multi-level MRI segmentation method for diagnosing tumors in an embodiment of the present application is introduced by way of example.

[0124] 1. Data collection and annotation

[0125] The dataset used for model training and optimization contains 3D MRI of 653 patients collected from October 2017 to September 2023, specifically for the preoperative evaluation of spinal cord tumors without prior therapeutic intervention. This dataset is notable for its comprehensive vertebral representation (including cervical, thoracic, and lumbar regions), including four major spinal cord tumor variants: meningioma (247 patients), astrocytoma (101 patients), ependymoma (203 patients) and hemangioblastoma (102 patients), and the 3D MRI of the four tumor types is as follows: Figures 3A-3D shown.

[0126] As shown in Table 1, the spinal cord tumor dataset was divided into two different subsets: a training set containing 80% of the subjects (521 patients), and a test set containing the remaining 20% ​​of the subjects (132 patients). Each patient's 3D MRI consists of gadolinium-enhanced T1SC and T2S with professional manual tumor annotations. Two independent neuroradiologists manually segmented the true tumor region in the gadolinium-enhanced T1SC. If there was a dispute about the delineation of the tumor region, the final decision would be made by a senior neuroradiologist to ensure the highest fidelity and accuracy of the data annotation. The original resolution of the sagittal MRI scans in this dataset varied, with in-plane resolution ranging from 0.34 to 1.06 mm and slice thickness ranging from 1.5 to 8 mm. Each MRI image contained a different number of slices scanned in the sagittal plane, ranging from a minimum of 9 to a maximum of 36 slices.

[0127] Table 1 Detailed information of the spinal cord tumor dataset

[0128]

[0129] 2. Data Preprocessing

[0130] To standardize the dataset, the resolution of the sagittal images was resampled to 0.47 mm in the anteroposterior and superior-inferior directions and 3.3 mm in the left-right direction. The resampled images were voxel-cropped to form a bounding box of size 512 mm × 256 mm × 64 mm, which was located in the center of the field of view for focused analysis. The intensity of each scan was first normalized by subtracting the mean intensity value from each voxel and then dividing the result by the standard deviation of the image intensity. Subsequently, the images were aligned using the spinal cord toolbox (SCT) in the related art. Figures 4A-4C As shown, the process involves aligning the T2S image to the T1SC image via an affine transformation to ensure accurate registration.

[0131] 3. Multi-level MRI segmentation

[0132] 3.1 Spinal cord segmentation (first stage, also known as stage I)

[0133] like Figure 5A As shown, T2S (T2) is registered with T1SC (T1) to align the anatomical structure. The first stage uses a network based on the U-Net framework (taking the nnU-Net framework as an example) T1SC(T1) and registered As input for segmenting the spinal cord Then, the geometric center of the predicted spinal cord mask (i.e., the spinal cord region) is used as the center point of the key region or region of interest (ROI), and the center point of the spinal cord mask is calculated from T1, and The MRI scans with dimensions of 512 mm × 256 mm × 64 mm were extracted and designated as and Using cropped critical regions (ROIs) can alleviate the problem of class imbalance and accelerate training and inference in subsequent stages.

[0134] 3.2 Second-class segmentation of spinal cord tumors (second stage, also known as stage II)

[0135] Decomposing the multi-class segmentation task into a two-class segmentation stage (Stage II) and a classification stage (Stage III) transforms a complex problem into two simpler tasks, allowing each network to focus on its corresponding function and thus improve the overall performance.

[0136] Figure 5B The second stage of the framework is illustrated, where the cropped MRI and This stage trains a two-class tumor segmentation network (Based on the U-Net framework (taking the nnU-Net framework as an example)), it only focuses on segmenting the tumor area as the foreground But it does not classify the tumors into their corresponding types. This method simplifies multi-class segmentation by focusing on the location of tumors rather than their categories.

[0137] 3.3 Multi-class spinal cord tumor segmentation (Phase III)

[0138] Obtaining the predicted spinal cord tumor mask Finally, the third phase aims to distinguish different tumor types and treat them accordingly. Assign a category label. Figure 5C As shown, except and In addition, it also includes and As input, the masks of the spinal cord and tumor regions are respectively. Then, a five-class segmentation network is trained to distinguish the four tumor types and the background, and finally assign a class label to the tumor region. In addition, in order to ensure the homogeneity of the prediction results of a single patient, a new class consistency loss is introduced in the third stage to prevent different labels from appearing in the prediction results of a single patient.

[0139] 3.4 Training Loss Function

[0140] (1) Dice Loss Function

[0141] The Dice Loss function is used in all three stages of segmentation, which quantizes the predicted segmentation and true segmentation The similarity between The Dice Loss function used in this paper is a multi-class variant extension introduced in the relevant technology and can be implemented as follows:

[0142]

[0143] in represents the Dice Loss function, Represents the predicted segmentation result, represents the true segmentation, p i,k is the softmax output of the network, indicating the probability that the i-th voxel is classified as class k. g corresponds to the one-hot encoding of the true segmentation map, and the dimensions of p and g are both N×K; i∈N, indicating the number of voxels in a given patch or batch; k∈K, indicating the total number of categories in the segmentation task; according to the true segmentation, g is only considered if the i-th voxel does belong to class k. i,k is set to 1; otherwise g i,k Set to 0.

[0144] (2) Cross Entropy Loss Function

[0145] In addition, cross-entropy (CE) loss is used in all three stages, expressed as It quantifies the difference between correct and incorrect classifications at the voxel level.

[0146] The cross entropy loss function is:

[0147]

[0148] (2) Category consistent (CC) loss function

[0149] To ensure that each segmented tumor area is always classified as a single category, and a new category consistency loss function is proposed Failure to comply with this standard will result in penalties.

[0150] The category consistency loss function is:

[0151]

[0152] in represents the category consistency loss function of the foreground, Represents the category consistency loss function of the background, represents the category consistency loss function.

[0153] In general, the training loss function for the three stages is:

[0154]

[0155] It is worth noting that all training of the above models was performed on a single graphics processing unit (GPU) of type NVIDIA 3090. The training process was designed to run for 1000 epochs with a learning rate of 0.01. Within the foreground, we reassign its predicted class to the class containing the largest number of voxels, thereby enhancing the segmentation output.

[0156] Combine the following Figure 6A-6B and Tables 2-3, analyze the performance of the segmentation method in the embodiments of the present application.

[0157] (1) Benchmark Model and Indicators

[0158] The segmentation method in the embodiment of the present application is compared with the most advanced models established in medical image segmentation, including the first model nnFormer, the second model 3D UX-Net, the third model Swin UNETR and the fourth model nnUNet. The performance is evaluated using the evaluation indicators Dice Coefficient and Hausdorff Distance (HD) in the related paper M&Ms Challenge.

[0159] (2) Results

[0160] The results of the quantitative analysis are shown in Table 2. Based on Table 2, it can be seen that the segmentation method of the embodiment of the present application is 8.4% better than the previous best performing model nnUNet. When applied to different types of tumors, the DiceCoefficient of the method of the embodiment of the present application showed improvements of 0.8%, 7.8%, 13.6% and 11.3%, respectively. Regarding Hausdorff Distance, as shown in Table 2, the method of the embodiment of the present application is 49.2 mm better than nnUNet. Figure 6A-6B The qualitative analysis results are shown, indicating that the method of the embodiment of the present application has achieved high accuracy in predicting tumor type and region.

[0161] Table 2 Parameter quantitative analysis results

[0162]

[0163] (3) Ablation experiment

[0164] Based on the ablation experiment, the effectiveness of the method of the embodiment of the present application is evaluated by changing the input and strategy adopted. The results of the ablation experiment are shown in Table 3, where the bold indicates the best performance. It can be seen from Table 3 that:

[0165] 1) Model A utilizes unprocessed T1SC as input without any cropping, resulting in the worst performance.

[0166] 2) Model B enhances the method of the embodiment of the present application by cropping the T1SC MRI around the spinal cord region, thereby improving the performance (4.5%).

[0167] 3) Model C combines the registered T2S and the predicted two-class segmentation mask, which helps the model focus on the tumor location and further refines the segmentation accuracy (2.5%).

[0168] 4) Model D further adopts the category consistency loss of the embodiment of the present application, which can ensure the homogeneity of the segmentation results and achieve the best performance.

[0169] Table 3 Dice coefficients of ablation experiments under different inputs and strategies (%)

[0170]

[0171] The method provided in the embodiment of the present application is described in detail above. The device provided in the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0172] See also Figure 7 , is a schematic diagram of the structure of a tumor diagnosis device provided in an embodiment of the present application. Figure 7 The tumor diagnostic device 700 shown includes:

[0173] A preprocessing module 710 is used to preprocess the three-dimensional MRI of each tumor patient to obtain a standardized tumor data set, where the three-dimensional MRI includes gadolinium-enhanced T1SC and T2S;

[0174] A first segmentation module 720, for performing first stage segmentation using the processed three-dimensional MRI as an input of a first segmentation model, the first segmentation model being used to segment a first human organ tissue region in the MRI;

[0175] A second segmentation module 730, for performing second-stage segmentation based on T1SC, T2S and a second segmentation model, the second segmentation model is a second-class tumor segmentation network, and the second segmentation model is used to segment the tumor region in the first human organ tissue region;

[0176] A third segmentation module 740 is used to perform a third stage segmentation based on T1SC, T2S, the first human organ tissue region and the tumor region and a third segmentation model, wherein the third segmentation model is a multi-class tumor segmentation network, and the third segmentation model is used to segment the tumor type of the tumor region and assign a class label to the tumor region;

[0177] Among them, the first segmentation model, the second segmentation model, and the third segmentation model are models based on the U-Net framework.

[0178] Optionally, the first human organ tissue is a spinal cord, and the first segmentation model, the second segmentation model, and the third segmentation model are trained and optimized based on a spinal cord tumor dataset.

[0179] Optionally, the loss functions of the first segmentation model, the second segmentation model, and the third segmentation model include one or more of the following:

[0180] Dice Loss function, used to quantify the similarity between the predicted segmentation results and the true segmentation results;

[0181] The cross entropy loss function is used to quantify the difference between correct and misclassified tumor regions at the voxel level;

[0182] Class consistency loss function is used to ensure that the tumor type of each tumor region is always classified as a single type.

[0183] Optionally, the Dice Loss function is:

[0184]

[0185] in represents the Dice Loss function, Represents the predicted segmentation result, represents the true segmentation, p i,kis the flexible maximum transfer function output of the network, which indicates the probability that the i-th voxel is classified into class k; g is a single-bit effective code corresponding to the true segmentation map, and the dimensions of p and g are both N×K; i∈N, represents the number of voxels in a given patch or batch; k∈K, represents the total number of categories in the segmentation task; according to the true segmentation, g is only considered if the i-th voxel does belong to class k. i,k is set to 1; otherwise g i,k Set to 0;

[0186] The cross entropy loss function is:

[0187]

[0188] in represents the cross entropy loss function;

[0189] The category consistency loss function is:

[0190]

[0191] in represents the category consistency loss function of the foreground, Represents the category consistency loss function of the background, represents the category consistency loss function.

[0192] Optionally, the loss functions of the first segmentation model and the second segmentation model include a Dice Loss function and a cross entropy loss function, and the loss function of the third segmentation model includes a Dice Loss function, a cross entropy loss function, and a category consistency loss function.

[0193] Optionally, the tumor diagnostic device 700 further includes:

[0194] The extraction module 750 is used to extract the three-dimensional MRI of the first size from T1SC, T2S and the spinal cord area as the input of the subsequent stage, using the geometric center of the spinal cord area segmented by the first segmentation model as the center point of the key area before the second segmentation module 730 performs the second stage segmentation based on T1SC, T2S and the second segmentation model.

[0195] Optionally, the tumor diagnostic device 700 further includes:

[0196] The reallocation module 760 is configured to reallocate the tumor type of the tumor region to a type containing the largest number of voxels based on the tumor type of the tumor region segmented by the third segmentation model.

[0197] Optionally, the preprocessing includes one or more of the following:

[0198] Resampling, for modifying the resolution of the three-dimensional MRI to the first resolution;

[0199] Voxel cropping, used to crop the size of the three-dimensional MRI to a second size;

[0200] Scan intensity normalization is used to subtract the mean intensity value from each voxel of the MRI and divide the result by the standard deviation of the image intensity of the three-dimensional MRI;

[0201] Image registration for aligning T2S to T1SC using the Spinal Cord Toolbox.

[0202] Optionally, the tumor regions of the 3D MRI in the spinal cord tumor dataset are manually annotated by medical experts.

[0203] Optionally, the spinal cord tumor dataset is based on full vertebral 3D MRI acquisitions of multiple spinal cord tumor patients.

[0204] The present application also provides an electronic device. Figure 8 As shown, the electronic device 800 includes: at least one processor 801, a memory 802, and a computer program 803 stored in the memory 802 and executable on the at least one processor 801. When the processor 801 executes the computer program 803, the channel selection method provided in the present application is implemented.

[0205] Exemplarily, the computer program 803 may be divided into one or more modules / units, one or more modules / units are stored in the memory 802, and are executed by the processor 801 to complete the present application. One or more modules / units may be a series of computer program instruction segments capable of completing corresponding functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device 800.

[0206] Those skilled in the art will understand that Figure 8 It is only an example of the electronic device 800 and does not constitute a limitation of the electronic device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 800 may also include input and output devices, network access devices, buses, etc.

[0207] The processor 801 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0208] The memory 802 may be an internal storage unit of the electronic device 800, such as a hard disk or memory of the electronic device 800. The memory 802 may also be an external storage device of the electronic device 800, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 800. Further, the memory 802 may also include both an internal storage unit of the electronic device 800 and an external storage device. The memory 802 is used to store computer programs and other programs and data required by the electronic device 800. The memory 802 may also be used to temporarily store data that has been output or is to be output.

[0209] The electronic device 800 provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0210] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0211] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0212] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above method embodiments, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / electronic device, recording medium, computer memory, ROM (read-only memory), RAM (random access memory), CD-ROM (compact disc read-only memory), magnetic tape, floppy disk and optical data storage device. The computer-readable storage medium mentioned in the present application can be a non-volatile storage medium, in other words, a non-transient storage medium.

[0213] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0214] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the corresponding application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each corresponding application, but such implementation should not be considered to be beyond the scope of this application.

[0215] In the embodiments provided in the present application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0216] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0217] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0218] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0219] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0220] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A multi-level magnetic resonance imaging (MRI) segmentation method for diagnosing tumors, characterized in that: include: The three-dimensional MRI of each tumor patient is preprocessed to obtain a standardized tumor data set, wherein the three-dimensional MRI includes a gadolinium-enhanced longitudinal relaxation time-weighted sagittal image T1SC and a transverse relaxation time-weighted sagittal image T2S; Using the processed three-dimensional MRI as an input of a first segmentation model to perform a first stage of segmentation, wherein the first segmentation model is used to segment a first human organ tissue region in the three-dimensional MRI; Performing second-stage segmentation based on the T1SC, T2S and a second segmentation model, where the second segmentation model is a two-type tumor segmentation network, and the second segmentation model is used to segment the tumor region in the first human organ tissue region; Performing a third stage of segmentation based on the T1SC, T2S, the first human organ tissue region, the tumor region, and a third segmentation model, wherein the third segmentation model is a multi-class tumor segmentation network, and the third segmentation model is used to segment the tumor type of the tumor region and assign a category label to the tumor region; Among them, the first segmentation model, the second segmentation model, and the third segmentation model are models based on the U-net framework.

2. The method according to claim 1, characterized in that The first human organ tissue is the spinal cord, and the first segmentation model, the second segmentation model, and the third segmentation model are trained and optimized based on a spinal cord tumor dataset.

3. The method according to claim 2, characterized in that The loss functions of the first segmentation model, the second segmentation model, and the third segmentation model include one or more of the following: Dice loss function, which is used to quantify the similarity between the predicted segmentation results and the true segmentation results; A cross entropy loss function for quantifying the difference between correct classification and misclassification at the voxel level of the tumor region; A class consistency loss function is used to ensure that the tumor type of each tumor region is always classified as a single type.

4. The method according to claim 3, characterized in that: The dice loss function is: It is stated represents the dice loss function, the Represents the predicted segmentation result, represents the true segmentation, the p i,k is the flexible maximum transfer function output of the network, indicating the probability that the i-th voxel is classified into class k; g is a one-bit effective code corresponding to the true segmentation map, and the dimensions of p and g are both N×K; i∈N represents the number of voxels in a given patch or batch; k∈K represents the total number of categories in the segmentation task; according to the true segmentation, g is considered to be the correct value only if the i-th voxel does belong to class k. i,k is set to 1; otherwise, the g i,k Set to 0; The cross entropy loss function is: in represents the cross entropy loss function; The category consistency loss function is: It is stated Represents the category consistency loss function of the foreground, Represents the category consistency loss function of the background, Denotes the category consistency loss function.

5. The method according to claim 4, characterized in that The loss functions of the first segmentation model and the second segmentation model include the dice loss function and the cross entropy loss function, and the loss function of the third segmentation model includes the dice loss function, the cross entropy loss function and the category consistency loss function.

6. The method according to claim 5, characterized in that Before performing the second stage segmentation based on the T1SC, T2S and the second segmentation model, the method further includes: The geometric center of the spinal cord region segmented by the first segmentation model is used as the center point of the key region, and a first-size three-dimensional MRI is extracted from the T1SC, T2S and the spinal cord region as an input for a subsequent stage.

7. The method according to claim 6, characterized in that The method further comprises: Based on the tumor type of the tumor region segmented by the third segmentation model, the tumor type of the tumor region is reassigned as a type containing the largest number of voxels.

8. The method according to claim 7, characterized in that The pretreatment includes one or more of the following: Resampling, for modifying the resolution of the three-dimensional MRI to a first resolution; Voxel cropping, used to crop the size of the three-dimensional MRI to a second size; Scan intensity normalization, for subtracting a mean intensity value from each voxel of the three-dimensional MRI and dividing the result by a standard deviation of the image intensity of the three-dimensional MRI; Image registration for aligning the T2S to the T1SC using the Spinal Cord Toolbox.

9. The method according to claim 8, characterized in that The tumor region of the three-dimensional MRI in the spinal cord tumor dataset is manually annotated by medical experts.

10. The method according to claim 9, characterized in that The spinal cord tumor dataset is based on full-body vertebral 3D MRI acquisitions of multiple spinal cord tumor patients.

Images