Brain glioma classification method and device based on multi-modal MRI

Abstract

The invention discloses a brain glioma classification method and device based on multi-mode MRI, and relates to the technical field of image processing. According to the method, multiple tumor sub-regions of the brain glioma are acquired based on a to-be-processed multi-modal MR image, multiple local ROIs are extracted based on the multiple tumor sub-regions, and then the sub-type of the brain glioma is determined based on the multiple local ROIs and a global ROI. The subtype can be determined through the multi-modal MR image, so that on one hand, non-invasive identification of the subtype is realized, and bleeding and infection risks caused by identification of the subtype are avoided; and on the other hand, the classification accuracy can be improved by utilizing complementary information between different modal images and a boundary form between different tumor sub-regions. Besides, through the fine structure features of each tumor sub-region reserved from a plurality of visual angles by the plurality of local ROIs, and in combination with the spatial distribution context information of the whole brain glioma provided by the global ROI, a better balance between the processing precision and the calculation efficiency can be realized.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and apparatus for classifying gliomas based on multimodal MRI. Background Technology

[0002] Gliomas are the most common tumors of the central nervous system, accounting for 80% of primary malignant intracranial tumors, and seriously endangering patients' lives and health. Accurate identification of glioma subtypes is crucial for their treatment. Subtypes refer to more refined classifications obtained by combining molecular and histological characteristics with the traditional pure histological grading (grades 1-4).

[0003] Currently, the determination of glioma subtypes mainly relies on pathological diagnosis following biopsy or surgery. However, biopsy and postoperative pathological diagnosis are invasive procedures, which may carry risks such as bleeding and infection. Therefore, there is an urgent need to develop a non-invasive technique for glioma subtype identification. Summary of the Invention

[0004] This application provides a method and device for classifying gliomas based on multimodal MRI, which can non-invasively and reliably identify subtypes of gliomas. The technical solution is as follows: On the one hand, a method for classifying gliomas based on multimodal MRI is provided, the method comprising: Based on the multimodal MR images to be processed, multiple tumor sub-regions of glioma are obtained, including: necrotic sub-regions, enhancement sub-regions, and edema sub-regions; Based on the multiple tumor sub-regions, multiple regions of interest (ROIs) are extracted. Based on the multiple local ROIs and the global ROI of the glioma, the subtype of the glioma is determined; Each of the local ROIs includes a portion of the glioma and reflects the state of the infiltration boundary between at least two tumor sub-regions; the global ROI includes the entire glioma, and the size of the global ROI is the same as the size of each of the local ROIs.

[0005] Optionally, based on the multiple tumor sub-regions, multiple regions of interest (ROIs) are extracted, including: A local region of interest (ROI) is extracted centered on the centroid of each of the tumor sub-regions, resulting in multiple local ROIs.

[0006] Optionally, the tumor percentage of the portion of the glioma included in each local ROI is greater than a ratio threshold, where the tumor percentage is the proportion of the portion of the glioma included in the local ROI to the entire glioma.

[0007] Optionally, based on the multiple local ROIs and the global ROI of the glioma, the subtype of the glioma is determined, including: The multiple local ROIs and the global ROI are all input into the target model to obtain the subtype of the glioma output by the target model.

[0008] Optionally, the target model can also output the first segmentation result for each of the tumor sub-regions.

[0009] Optionally, before inputting the plurality of local ROIs and the global ROI into the target model, the method further includes: A pre-trained model is obtained, which is trained using an open-source multimodal MR image set; Based on the model parameters of the pre-trained model, transfer learning is performed to obtain the target segmentation model; The target model is obtained by performing transfer learning based on the model parameters of the target segmentation model.

[0010] Optionally, based on the multimodal MR images to be processed, multiple tumor sub-regions of the glioma are acquired, including: Based on the multimodal MR image to be processed, second segmentation results of multiple tumor sub-regions of glioma are obtained, and the second segmentation results of each tumor sub-region are used to characterize the tumor sub-region in the multimodal MR image.

[0011] Optionally, based on the multimodal MR images to be processed, multiple tumor sub-regions of the glioma are acquired, including: Based on the multimodal MR images to be processed, if it is determined that the multimodal MR images include a brain tumor, then it is determined whether the brain tumor is a glioma. If the brain tumor is determined to be a glioma, then multiple tumor sub-regions of the glioma are obtained.

[0012] On the other hand, a controller is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the multimodal MRI-based glioma classification method as described above.

[0013] In another aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the multimodal MRI-based glioma classification method as described above.

[0014] In another aspect, a computer program product is provided, the computer program product comprising a computer program or computer instructions, which, when executed by a processor, implement the multimodal MRI-based glioma classification method as described above.

[0015] In another aspect, an electronic device is provided, the electronic device comprising: a controller as described in the above aspects.

[0016] The beneficial effects of the technical solution provided in this application include at least the following: This application provides a method and apparatus for classifying gliomas based on multimodal MRI. The method can acquire multiple tumor sub-regions of a glioma based on the multimodal MR images to be processed, and extract multiple local areas of interest (ROIs) based on these sub-regions. Then, based on these local ROIs and the global ROI, the subtype of the glioma is determined. Since the subtype can be determined using multimodal MR images, on the one hand, compared to pathological diagnosis after biopsy or surgery, non-invasive identification of the subtype is achieved, avoiding the risks of bleeding and infection associated with subtype identification; on the other hand, it can effectively utilize complementary information between different modal images and the boundary morphology between different tumor sub-regions, improving classification accuracy. Furthermore, each local ROI includes a portion of the glioma and reflects the state of the intrusive boundary between at least two tumor sub-regions. The global ROI includes the entire glioma, and the size of the global ROI is the same as the size of each local ROI. Therefore, by preserving the subtle structural features of each tumor sub-region from multiple perspectives through multiple local ROIs, and combining this with the spatial distribution context information of the entire glioma provided by the global ROI, it is possible to effectively avoid the reduction in processing accuracy while saving computational overhead, thus achieving a better balance between processing accuracy and computational efficiency.

[0017] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0018] Figure 1 This is a flowchart of a glioma classification method based on multimodal MRI provided in an embodiment of this application; Figure 2 This is a flowchart of another glioma classification method based on multimodal MRI provided in the embodiments of this application; Figure 3This is a schematic diagram of the structure of an initial model provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an initial segmentation model provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a multi-task model provided in an embodiment of this application; Figure 6 This is a schematic diagram of a local region of interest (ROI) extracted with the centroid of the edema sub-region as the center, provided in an embodiment of this application. Figure 7 This is a schematic diagram of a local ROI extracted with the centroid of the necrotic sub-region as the center, provided in an embodiment of this application. Figure 8 This is a schematic diagram of a global ROI provided in an embodiment of this application; Figure 9 This is a schematic diagram of the first segmentation result of each tumor sub-region output by a target model provided in an embodiment of this application; Figure 10 This is a schematic diagram of a segmented gold standard image provided in an embodiment of this application; Figure 11 This is a schematic diagram of the structure of a controller provided in an embodiment of this application. Detailed Implementation

[0019] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0020] This application provides a method for classifying gliomas based on multimodal magnetic resonance imaging (MRI). This method is applied to electronic devices, such as the controller of an electronic device. Optionally, the electronic device can be a scanning device, a mobile terminal, a fixed terminal, or a server. The scanning device can be an MRI device. The mobile terminal can be a mobile phone, a laptop computer, or a tablet computer, etc. The fixed terminal can be a desktop computer or a digital TV, etc. The server can be a single server, a server cluster consisting of several servers, or a cloud computing service center. See also... Figure 1 The method includes: Step 101: Based on the multimodal MR images to be processed, obtain multiple tumor sub-regions of the glioma.

[0021] The tumor comprises multiple sub-regions, including necrotic sub-regions, enhancer sub-regions, and edema sub-regions. These sub-regions collectively constitute the entire glioma.

[0022] Step 102: Based on multiple tumor sub-regions, multiple local ROIs are extracted.

[0023] Each region of interest (ROI) includes a portion of a glioma and reflects the invasive boundary state between at least two tumor subregions (also known as the infiltrative boundary state). This invasive boundary state can refer to the complex interactive process of mutual penetration and invasion between two or more tumor regions at the boundary, reflecting the heterogeneity and invasiveness of the tumor. Any two ROIs have the same size. This size can be the total number of voxel points included in the ROI.

[0024] It is understandable that any two local regions of interest (ROIs) are located in different places. Therefore, the subtle structural features of each tumor subregion can be provided from different perspectives.

[0025] Step 103: Based on multiple local ROIs and the global ROI of glioma, determine the subtype of glioma.

[0026] The global region of interest (ROI) encompasses the entire glioma, meaning it provides spatial distribution contextual information about the glioma. The size of the global ROI is the same as that of each local ROI. Therefore, the resolution of the global ROI is lower than that of the local ROIs.

[0027] In one alternative implementation, the electronic device can input both a global ROI and multiple local ROIs into the target model to obtain the subtype of glioma output by the target model.

[0028] In another alternative implementation, the electronic device can pre-store global reference ROIs and multiple local reference ROIs for various subtypes of gliomas. These multiple local reference ROIs correspond one-to-one with the multiple local ROIs. The electronic device can determine the similarity between each local ROI and its corresponding local reference ROI, as well as the similarity between the global ROI and the global reference ROI. It can then identify the subtype of glioma in the multimodal MR image to be processed as the global reference ROI with the highest sum of similarities and the subtypes of the multiple local reference ROIs to which it belongs.

[0029] In summary, this application provides a method for classifying gliomas based on multimodal MRI. This method can acquire multiple tumor sub-regions of a glioma based on the multimodal MR images to be processed, and extract multiple local areas of interest (ROIs) based on these multiple tumor sub-regions. Then, based on these multiple local ROIs and the global ROI, the subtype of the glioma is determined. Since the subtype can be determined using multimodal MR images, on the one hand, compared with pathological diagnosis after biopsy or surgery, this method achieves non-invasive identification of the subtype, avoiding the risks of bleeding and infection associated with subtype identification; on the other hand, it can effectively utilize the complementary information between different modal images and the boundary morphology between different tumor sub-regions, improving classification accuracy. Furthermore, each local ROI includes a portion of the glioma and reflects the state of the intrusion boundary between at least two tumor sub-regions. The global ROI includes the entire glioma, and the size of the global ROI is the same as the size of each local ROI. Therefore, by preserving the subtle structural features of each tumor sub-region from multiple perspectives through multiple local ROIs, and combining this with the spatial distribution context information of the entire glioma provided by the global ROI, it is possible to effectively avoid the reduction in processing accuracy while saving computational overhead, thus achieving a better balance between processing accuracy and computational efficiency.

[0030] This application uses the determination of glioma subtypes through target models as an example to illustrate the glioma classification method based on multimodal MRI provided in this application. This method can be applied to electronic devices; see [link to relevant documentation]. Figure 2 The method may include: Step 201: Obtain the pre-trained model.

[0031] Electronic devices can use open-source multimodal MR image sets to train an initial model (i.e., an untrained model) to obtain a pre-trained model. This pre-trained model needs to have a deep understanding of the content of the input multimodal MR images and strong representation capabilities to effectively capture the structural and tissue features of the brain, and to correctly identify and reconstruct both tumor regions and normal brain tissue regions.

[0032] This multimodal MR image set includes: multimodal MR images of the brain with gliomas, and multimodal MR images of the healthy brain (i.e., without gliomas). The multimodal MR images include MR images from multiple imaging modalities. These multiple imaging modalities include, but are not limited to, at least two of the following imaging models: T1-weighted image (T1WI), T1-weighted imaging with contrast enhancement (T1CE), T2-weighted image (T2WI), and fluid attenuated inversion recovery (FLAIR) sequences. Accordingly, the multimodal MR images include, but are not limited to, T1WI images, T1CE images, T2WI images, and FLAIR images. Each imaging modality's MR image can be a 3D (dimension D) image.

[0033] MR images from different imaging modalities can reflect different tissue features of brain lesions. For example, T1CE images can more clearly show the necrotic areas of the tumor, while T2WI or FLAIR images are more conducive to observing the edema areas around the tumor. Based on the complementarity of multiple imaging modalities, using open-source 3D multimodal MR images as input to the initial model to fully integrate the structures provided by each modality of MR images can effectively improve the ability of the subsequent target model to distinguish gliomas and their subtypes.

[0034] Considering the difficulties in collecting and annotating multimodal MR images of gliomas, and the abundance of open-source multimodal MR images available (e.g., at least 2000 cases of multimodal MR images of gliomas, and some open-source multimodal MR images of healthy brains are also available), this application fully utilizes open-source multimodal MR image sets and employs large-scale model pre-training to obtain a pre-trained model, effectively alleviating data pressure.

[0035] Optionally, the electronic device can employ unsupervised learning to train the model, resulting in a pre-trained model. Furthermore, the initial model can utilize a vision transformer (ViT) autoencoder architecture. See also Figure 3 The architecture includes: a ViT encoder 01, an intermediate layer 02, and a ViT decoder 03 connected in sequence. The intermediate layer 02 connects the ViT encoder 01 and the ViT decoder 03. See also... Figure 3 The training process of a pre-trained model can include: First, open-source multimodal MR images are cascaded and fused (i.e., stitched) along the channel dimension to obtain a cascaded fused image. Then, a random masking strategy is used to massively occlude the cascaded fused image (e.g., occluding 40%-60% of the region) to enhance the model's ability to extract features at different scales. Next, the occluded cascaded fused image is input into the initial model, and the original cascaded fused image (i.e., the unoccluded cascaded fused image) is used as the reconstruction target. By minimizing the reconstruction loss, the initial model learns discriminative features, resulting in a pre-trained model. This self-supervised pre-training method enables the model to deeply understand the anatomical structural features of brain MR images, providing high-quality initialization parameters for subsequent segmentation and classification tasks. The reconstruction loss can be either mean squared error (MSE) loss or L1 loss.

[0036] In this embodiment, the electronic device can use the `concatenate()` function to process open-source multimodal MR images, thereby cascading and fusing the open-source multimodal MR images along the channel dimension. Furthermore, the electronic device can perform random masking on the cascaded images to achieve occlusion. The size of the mask patch can be flexibly configured, ranging from a small scale such as 4×4×4 to a large scale such as 16×16×16.

[0037] Understandably, before training an initial model using an open-source multimodal MR image dataset, electronic devices can preprocess the individual multimodal MR images included in that dataset. This preprocessing may include image registration and skull removal. Image registration enables precise spatial alignment of the multimodal MR images, while skull removal effectively eliminates interference from non-brain tissue (i.e., the skull) in subsequent analysis, improving the focus and accuracy of the pre-trained model.

[0038] The process of image registration can include: using a T1CE image as a reference, registering all other imaging modal MR images with the T1CE image using a registration algorithm to ensure that the anatomical structures of MR images of different imaging modalities maintain spatial consistency.

[0039] The process of skull removal can include: extracting brain tissue regions based on T1CE images using a skull removal algorithm to generate an accurate intracranial mask; then, applying the intracranial mask to registered MR images of other imaging modalities to achieve skull removal in full-modality MR images.

[0040] Step 202: Perform transfer learning based on the model parameters of the pre-trained model to obtain the target model.

[0041] This target model is used to perform segmentation and classification tasks for multiple tumor subregions within gliomas. These multiple tumor subregions include: necrotic subregions, enhancement subregions, and edema subregions. Glioma subtypes may include, but are not limited to, one of the following: astrocytoma, oligodendroglioma, glioblastoma, diffuse pediatric glioma, localized astrocytoma, neuronal tumors, and ependymoma.

[0042] Due to the complexity and diverse morphologies of gliomas, data collection and annotation are extremely difficult, resulting in limited training data. However, in this embodiment, a pre-trained model capable of accurately interpreting multimodal MR images is first trained using large-scale open-source data through unsupervised learning. Then, transfer learning is performed based on the parameters of the pre-trained model to obtain a target model for multimodal MR images of the brain. This achieves efficient learning and accurate classification with limited training data, significantly reducing the modeling difficulty of the target model. In other words, the method provided in this embodiment supports building a target model through few-shot learning.

[0043] In some embodiments, the process of an electronic device performing step 202 may include: Step S1: Perform transfer learning based on the model parameters of the pre-trained model to obtain the target segmentation model.

[0044] In this embodiment of the application, the process of the electronic device performing step S1 may include: Step A1: Perform transfer learning based on the model parameters of the pre-trained model to obtain the initial segmentation model.

[0045] See Figure 4 The electronic device can add a lightweight segmenthead 04 to the initial model to obtain the first model to be trained. The input of the segmenthead 04 is connected to the output of the ViT decoder 03. Then, the electronic device can initialize the first model based on the model parameters of the pre-trained model to achieve transfer learning of the model parameters of the pre-trained model, thereby obtaining the initial segmentation model.

[0046] That is, the initial model parameters of the initial segmentation model are the same as those of the pre-trained model. This allows the first model to utilize the representational capabilities of the pre-trained model for multimodal MR images, thereby improving the segmentation performance and ultimately enhancing the reliability of the trained target segmentation model.

[0047] Optionally, the segmentation head can be a shallow convolutional neural network (CNN).

[0048] Step A2: Train the initial segmentation model using multiple first training data to obtain the target segmentation model.

[0049] This target segmentation model is used to segment gliomas, specifically segmenting necrotic sub-regions, enhanced sub-regions, and edema sub-regions. Each first training dataset includes: a multimodal MR image sample containing a glioma, and segmentation labels for each tumor sub-region corresponding to that multimodal MR image sample.

[0050] In the embodiments of this application, when training the initial segmentation model, the electronic device can fine-tune the model parameters of the initial segmentation model with a low learning rate, freeze some shallow layers to retain pre-trained features, and combine Dice loss and cross-entropy loss to optimize the segmentation performance of multi-class imbalanced data. This can improve the generalization ability and segmentation accuracy of the model while reducing training costs.

[0051] Step S2: Perform transfer learning based on the model parameters of the target segmentation model to obtain the target model.

[0052] In this embodiment of the application, the process of the electronic device performing step S2 may include: Step B1: Perform transfer learning based on the model parameters of the target segmentation model to obtain a multi-task model.

[0053] See Figure 5 The electronic device can add a classification head 05 to the first model to obtain a second model to be trained. The input of the classification head 05 is connected to the output of the intermediate layer 02. Therefore, the second model includes: a ViT encoder 01 for extracting deep features from multimodal MR images, an intermediate layer 02, a ViT decoder 03 for image feature reconstruction and enhancement, a segmentation head 04 for tumor region segmentation prediction, and a classification head 05 for glioma subtype discrimination. Then, the electronic device can initialize the second model based on the model parameters of the target segmentation model to achieve transfer learning of the model parameters of the target segmentation model, thereby obtaining a multi-task model.

[0054] Even if the initial model parameters of the multi-task model are the same as those of the target segmentation model, the multi-task model can utilize the segmentation ability of the target segmentation model to segment gliomas in multimodal MR images.

[0055] Step B2: Train the multi-task model using multiple second training data sets to obtain the target model.

[0056] Each second training dataset may include: a global region of interest (ROI) sample and multiple local ROI samples obtained from multimodal MR image samples, as well as segmentation labels for each tumor sub-region and glioma classification labels corresponding to the multimodal MR image samples. The classification labels include subtypes of gliomas.

[0057] In this system, multiple local ROI samples correspond one-to-one with multiple tumor sub-regions in the multimodal MR image samples, and each local ROI sample is extracted from the multimodal MR image samples based on a segmentation label of a corresponding tumor sub-region. For example, each local ROI sample can be extracted with the centroid of the corresponding tumor sub-region as the center.

[0058] The global ROI sample includes the entire glioma, while each local ROI sample includes a portion of the glioma. Each local ROI can reflect the state of the intrusion boundary between at least two tumor subregions. The size of each local ROI sample is the same as the size of the global ROI sample; that is, the resolution of each local ROI sample is higher than the resolution of the global ROI sample.

[0059] Global ROI samples can reflect the spatial distribution context of gliomas, while multiple local ROI samples can reflect the fine structural features of brain tumor sub-regions from different angles with higher resolution. Therefore, the target model trained by multiple local ROI samples and global ROI samples can accurately achieve fine-grained discrimination of glioma subtypes.

[0060] In the embodiments of this application, when the electronic device trains the multi-task model using the second training data, it can fine-tune the parameters of the segmentation head and the classification head with a low learning rate to obtain the target model.

[0061] It is understandable that, such as Figure 5 As shown, the architecture of the multi-task model is that the segmentation head and the classification head share an encoder, meaning that the segmentation task and the classification task share the same underlying feature representation. In this way, during the training process of the multi-task model, the segmentation task and the classification task can mutually constrain each other, achieving collaborative supervision.

[0062] Segmentation and classification tasks can complement each other through attention mechanisms. For example, in the design of the loss function, higher (e.g., 3-5 times) weight coefficients can be applied to the boundary pixels of each tumor sub-region to constrain the model to prioritize the boundary regions of each tumor sub-region when making classification decisions. That is, during the model training process, edge awareness can be added to emphasize the morphological features of necrotic, enhanced, and edematous sub-regions of gliomas.

[0063] In this embodiment, the electronic device can accurately extract the boundary regions of tumor sub-regions by using tumor sub-region segmentation tags combined with morphological operations. These morphological operations may include operations such as expansion and erosion, for example, expansion and erosion of a 3×3×3 nucleus.

[0064] Step 203: Obtain the multimodal MR image to be processed.

[0065] In this embodiment, the multimodal MR image to be processed may be pre-stored by an electronic device. Alternatively, if the electronic device is an MRI device, the multimodal MR image to be processed may be obtained by MRI scanning the brain of the imaging subject.

[0066] It is understandable that after acquiring the multimodal MR image, the electronic device can preprocess the image before proceeding with subsequent processes to obtain a more refined first segmentation result for the subtypes of gliomas and each tumor sub-region. The preprocessing process of the multimodal MR image can refer to relevant implementations of preprocessing open-source multimodal MR images, and will not be elaborated further in this application.

[0067] Step 204: Based on the multimodal MR image to be processed, if it is determined that the multimodal MR image includes a glioma, then obtain each tumor sub-region of the glioma based on the multimodal MR image.

[0068] Given the diverse types of gliomas and the common characteristics they share, this application proposes a grading strategy that simulates a clinical diagnostic process to determine the subtype of gliomas. Specifically, the electronic device can first determine whether a brain tumor exists in the multimodal MR image to be processed. If the electronic device determines that a brain tumor is present in the multimodal MR image, it can further determine whether the brain tumor is a glioma. If the electronic device determines that the brain tumor is a glioma, it can acquire multiple tumor sub-regions of the glioma so that the subtype of the glioma can be determined subsequently based on these multiple tumor sub-regions.

[0069] In other words, the method provided in this application can determine whether a brain tumor exists in a multimodal MR image at the first level, i.e., perform preliminary brain tumor screening; further distinguish whether it is a glioma at the second level, i.e., distinguish between gliomas and other brain tumors; and achieve fine classification of glioma subtypes at the third level, i.e., perform fine-grained identification of specific subtypes.

[0070] By employing a hierarchical judgment strategy—a diagnostic reasoning mechanism that progresses from coarse to fine and layer by layer—the difficulty of recognition can be reduced on the one hand, and the classification accuracy can be improved on the other, thus balancing recognition efficiency and accuracy.

[0071] Optionally, the electronic device can pre-train a first classification segmentation model and a second classification segmentation model. After obtaining the multimodal MR image to be processed, the electronic device can first input the multimodal MR image into the first classification segmentation model to obtain a first result output by the first classification segmentation model. This first result is used to indicate whether a brain tumor exists. If the first result indicates the presence of a brain tumor, the electronic device can input the multimodal MR image into the second classification segmentation model to obtain a second result output by the second classification segmentation model. This second result is used to indicate whether the brain tumor is a glioma.

[0072] In this embodiment, the network structure design of the first classification segmentation model, the second classification segmentation model, and the target model can be the same. The differences include the training data and the number of neurons in the last layer of the "classification head" network. This number of neurons corresponds to the final number of classifications.

[0073] Correspondingly, the process of training the electronic device to obtain the first classification segmentation model and the second classification segmentation model can refer to the relevant implementation process of training the target model on the electronic device, and will not be repeated here in the embodiments of this application.

[0074] Optionally, the electronic device can pre-store a pre-trained segmentation network, the segmentation accuracy of which can be lower than that of the segmentation head in the target model. When the electronic device determines that the brain tumor in the multimodal MR image is a glioma, it can input the multimodal MR image into the pre-trained segmentation network to obtain the second segmentation results for each tumor sub-region output by the pre-trained segmentation network.

[0075] The second segmentation result for each tumor sub-region can characterize that tumor sub-region in the multimodal MR image. The pre-trained segmentation network can be trained using multiple first training datasets.

[0076] Step 205: Based on multiple tumor sub-regions, multiple local ROIs are extracted.

[0077] Each local ROI includes a portion of a glioma and reflects the state of the infiltration boundary between at least two tumor subregions.

[0078] For example, the tumor proportion of the glioma included in each local ROI is greater than a ratio threshold. This tumor proportion is the percentage of the glioma included in the local ROI relative to the entire glioma. In other words, each local ROI can cover the vast majority of the glioma area. Therefore, each local ROI can reflect the infiltration boundary between at least two tumor sub-regions.

[0079] In this embodiment, the electronic device can employ a parallel ROI extraction strategy to extract multiple local ROIs based on multiple tumor sub-regions. For example, the electronic device can first locate the spatial distribution of the corresponding tumor sub-regions based on the second segmentation results of each tumor sub-region output by a pre-trained segmentation network. Then, the electronic device can extract a local ROI centered on the centroid of each tumor sub-region, thereby obtaining multiple local ROIs.

[0080] The dimensions of these multiple local areas of interest (ROIs) can all be target dimensions. These target dimensions can be pre-stored by the electronic device and can be statistically designed to cover the majority of the glioma's area. For example, the size could be 80×100×100.

[0081] Because the morphology of each tumor subregion is irregular and heterogeneous, and their positions differ in multimodal MR images, the portions of gliomas included in any two local areas of interest (ROIs) extracted based on the centroids of each tumor subregion will vary. Therefore, it is possible to perform multi-view tumor region characterization using the centroids of different tumor subregions as reference points. Furthermore, since multiple local ROIs typically represent different emphases, the complementary information between these ROIs allows for a comprehensive analysis of gliomas, thereby improving the accuracy of glioma segmentation and classification.

[0082] Step 206: Obtain the global ROI of the glioma.

[0083] The global ROI includes the entire glioma, and the size of the global ROI is the same as the size of each local ROI, i.e., they are all the target size.

[0084] In this embodiment of the application, the electronic device can extract the initial ROI of the glioma based on the entire boundary of the glioma, and scale the size of the initial ROI to the target size to obtain the global ROI of the glioma.

[0085] Compared to the global Region of Interest (ROI), local ROIs offer higher resolution, preserving the fine structural features of each tumor subregion, while the global ROI provides spatial distribution contextual information about gliomas. This dual representation approach of "local refinement + global context" avoids the computational burden of directly processing the entire multimodal MRI image, and effectively prevents feature loss caused by the single scaling method of directly scaling the multimodal MRI image to reduce computational burden. Thus, a better balance can be achieved between processing accuracy and computational efficiency.

[0086] For example, using multimodal MR images including T1WI, T1CE, T2WI, and FLAIR images, Figure 6 A schematic diagram of a local ROI extracted with the centroid of the edema subregion as the center is shown. Figure 7 A schematic diagram of a local ROI extracted with the centroid of the necrotic sub-region as the center is shown. Figure 8 A schematic diagram of a global ROI is shown.

[0087] Figures 6 to 8 The first row in the image shows the multimodal MR image, and the second row shows the local or global region of interest (ROI). The local and global ROIs can be displayed using pseudo-color images, with different colors used for each tumor sub-region.

[0088] Step 207: Input multiple local ROIs and global ROIs into the target model to obtain the subtypes of gliomas output by the target model, as well as the segmentation results of each tumor sub-region.

[0089] Understandably, electronic devices can stitch together multiple local and global ROIs along the channel dimension and input the stitched result into the target model.

[0090] Taking multimodal MR images, including T1WI, T1CE, T2WI, and FLAIR images, as an example, see [link to relevant documentation]. Figure 9 , Figure 9 The first segmentation results of each tumor sub-region output by the target model are shown. Figure 10 This is a segmentation of the gold standard image. (Comparison) Figure 9 and Figure 10 As can be seen, in the first segmentation result obtained using the method provided in this application embodiment, each tumor sub-region is accurately identified and the boundaries are clearly segmented. Therefore, it can effectively assist the classification prediction of the classification head in the target model and has good practicality.

[0091] It is understood that the application of the method provided in this application is not limited to T1WI images, T1CE images, T2WI images and FLAIR images, but is applicable to MR images of any imaging modality of glioma and combinations thereof.

[0092] Table 1 shows the quantitative classification metrics of the first classification segmentation model, the second classification segmentation model, and the target model obtained through experiments in the embodiments of this application. The area under the receiver operating characteristic curve (AUC) is one of the core metrics for evaluating the predictive performance of binary classification models, and the higher the value, the more accurately the model can identify the positive and negative classes.

[0093] As shown in Table 1, the classification accuracy (Acc) of the first classification segmentation model is 0.9, and the area under the receiver operating characteristic (ROC) curve is 0.86. The classification accuracy of the second classification segmentation model is 0.85, and the ROC curve area is 0.912. The classification accuracy of the third classification segmentation model is 0.69, and the ROC curve area is 0.87. Therefore, it is evident that the reliability of the first classification segmentation model, the second classification segmentation model, and the target model provided in this application embodiment is high.

[0094] Table 1

[0095] Gliomas are the most common tumors of the central nervous system, accounting for 80% of primary malignant intracranial tumors. Diffuse gliomas, the most common type, are characterized by their difficulty in complete eradication, high recurrence rate, and low survival rate, seriously endangering patients' lives and health. According to the 2021 updated World Health Organization (WHO) classification of tumors of the central nervous system, the classification of gliomas has shifted from the traditional purely histological grading (grades 1-4) to a comprehensive classification combining molecular and histological characteristics. This molecular subtyping-based classification method is crucial for developing individualized treatment strategies, predicting patient prognosis, and guiding clinical decision-making. The molecular subtyping-based classification method further emphasizes that the diagnosis of brain tumors requires the integration of histological features, genetic and molecular information, which is more conducive to the development of individualized clinical treatment plans for patients.

[0096] In clinical practice, accurate diagnosis of gliomas is fraught with challenges. Low-grade astrocytomas have a mild mass effect and are easily misdiagnosed as inflammation, delaying surgical intervention. Meanwhile, non-neoplastic lesions such as tumor-like demyelination may also have a mass effect, easily misdiagnosed as tumors, leading to unnecessary surgery. Furthermore, high-grade gliomas are often difficult to differentiate from other intracranial tumors such as solitary brain metastases and lymphomas using only conventional MRI.

[0097] Currently, the classification, grading, and molecular characterization of gliomas primarily rely on pathological diagnosis following biopsy or surgery. These invasive procedures not only carry risks such as bleeding and infection but may also fail to fully reflect the spatial heterogeneity of the tumor due to limited sampling. For some deep-seated or functionally located gliomas, biopsy itself carries high risks or presents significant technical challenges. Therefore, there is an urgent need to develop a reliable and non-invasive technique for identifying glioma molecular subtypes.

[0098] Multiparametric magnetic resonance imaging (mpMRI) provides a wealth of information for non-invasive assessment of gliomas, including not only tumor morphology and location, but also reflecting the biological and functional characteristics of the tissue through different imaging modalities. However, despite the rich tissue feature information provided by mpMRI, the inherent heterogeneity within gliomas and the overlap of imaging features between different molecular subtypes mean that accurately predicting glioma molecular subtypes by relying solely on traditional single-modality imaging interpretation remains a significant challenge.

[0099] Currently, deep learning-based glioma classification algorithms, such as those based on convolutional neural networks (CNNs), can be used to non-invasively determine the subtype or grade of gliomas. The process involves preprocessing the input MR image, including cropping the foreground region and scaling the image to a smaller resolution to control computational resource usage. The preprocessed image is then used as input to a classification model (such as ResNet50), and the network's forward propagation calculates the predicted probability for each category.

[0100] However, deep learning-based glioma classification algorithms have the following drawbacks: Training a deep learning model from scratch typically requires a large amount of labeled data to ensure the model can learn sufficient feature representations. Specifically, in the task of glioma classification, the data distribution is even more complex due to the high heterogeneity and diversity of gliomas themselves. This means that to ensure the performance of the classification model, it is necessary to collect and label more training samples for each category, which is a huge challenge for clinical practice. Insufficient training data makes it difficult to guarantee the reliability of the model.

[0101] Second, the classification model has poor generalization ability. Specifically, considering the complexity of gliomas and the diversity of their internal structures, classification networks trained using classic CNNs are unlikely to have good generalization ability.

[0102] Third, classification tasks require more consideration of global information, thus necessitating the use of the entire 3D volume as network input. Considering computational overhead and resource consumption, the input image needs to be scaled down to a smaller resolution. However, this process leads to a loss of spatial resolution, making it impossible to capture subtle structural changes within the tumor, thereby reducing classification accuracy.

[0103] Fourth, classification models suffer from poor interpretability. Specifically, models that rely on a single classification network for predictions are significantly lacking in the ability to explain their decision-making process. The lack of explicit constraints guiding the network on how to use input data for category prediction makes understanding why the model makes a particular prediction extremely difficult. This reduces the model's acceptance and application value in real-world clinical settings.

[0104] The method provided in this application, firstly, employs unsupervised learning by utilizing a large set of open-source multimodal MR images to pre-train a powerful brain MR image analysis model, i.e., a pre-trained model. This model effectively captures brain structural and tissue features, possessing the ability to deeply understand and represent the content of input multimodal MR images. Based on this, a specific glioma classification task is used as a downstream task. Fine-tuning is performed based on the parameters of the pre-trained model, achieving efficient learning and accurate classification with limited labeled samples, significantly reducing the modeling difficulty of the target model. In other words, the target model is constructed through transfer learning based on the model parameters of the pre-trained model.

[0105] Second, compared to the single-modal imaging of mpMRI, multimodal 3D MR images can reflect different tissue features of brain lesions through different imaging modalities. Therefore, using multimodal 3D MR images as input to the target model for classification can leverage the complementarity between different modalities of 3D MRI images, fully integrate the structural and pathological information provided by each modality, and improve the model's ability to distinguish gliomas and their subtypes.

[0106] Third, based on the strong correlation between segmentation and classification tasks, a multi-task learning architecture design based on edge perception is provided. Specifically, the embodiments of this application fully consider the invasive boundary morphology between different tumor regions and the key role of the boundary features of each tumor region in glioma subtype discrimination. To this end, a multi-task learning architecture with a clear objective, well-defined structure, and integration of clinical experience guidance is proposed. This architecture introduces the morphological changes of different tumor sub-regions and the features of the infiltration boundaries between different tumor sub-regions into the classification head of the target model as an attention mechanism. This enables the simultaneous completion of glioma segmentation and classification tasks while strengthening the utilization of boundary information between different tumor regions, enhancing the model's ability to model tumor heterogeneity and subregional structural relationships, thereby effectively improving the accuracy and interpretability of glioma classification.

[0107] Fourth, addressing the issues of high resource consumption and loss of detail in full-image processing of multimodal 3D MR images due to direct scaling, the method provided in this application extracts local ROIs of target size at the original resolution, centered on the centroid of each tumor sub-region, to achieve multi-scale, multi-view tumor region representation. The three types of tumor sub-regions are then merged to generate a minimum bounding box containing the entire glioma, which is then scaled to the target size to obtain the global ROI. Finally, the local and global ROIs are input into the target model in parallel for processing. Thus, through the complementary information from parallel ROIs (i.e., the global ROI and each local ROI), comprehensive analysis can be achieved, thereby preserving lesion details while reducing computational overhead, balancing efficiency and accuracy.

[0108] Furthermore, considering the diverse types of gliomas and their shared characteristics, this application proposes a hierarchical classification strategy that simulates clinical diagnosis to improve classification accuracy and reduce identification difficulty. Specifically, it consists of three levels: the first level determines whether the input image includes a brain tumor; the second level, if a brain tumor is included, further distinguishes whether the brain tumor is a glioma; and the third level, if the brain tumor is a glioma, achieves a fine classification of the specific subtypes of the glioma, thereby constructing a coarse-to-fine, progressively layered reasoning mechanism to achieve a balance between diagnostic efficiency and accuracy.

[0109] The method provided in this application can achieve more granular classification of glioma subtypes, and can accurately distinguish six major types of gliomas: astrocytoma, oligodendroglioma, glioblastoma, juvenile diffuse glioma, localized astrocytoma and neuronal tumor, and ependymoma.

[0110] It is understood that the order of steps in the multimodal MRI-based glioma classification method provided in this application embodiment can be appropriately adjusted, and steps can be added or removed as needed. For example, steps 201 and 202 can be deleted if the target model is pre-trained. Any variations that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application, and therefore will not be elaborated further.

[0111] In summary, this application provides a method for classifying gliomas based on multimodal MRI. This method can acquire multiple tumor sub-regions of a glioma based on the multimodal MR images to be processed, and extract multiple local areas of interest (ROIs) based on these multiple tumor sub-regions. Then, based on these multiple local ROIs and the global ROI, the subtype of the glioma is determined. Since the subtype can be determined using multimodal MR images, on the one hand, compared with pathological diagnosis after biopsy or surgery, this method achieves non-invasive identification of the subtype, avoiding the risks of bleeding and infection associated with subtype identification; on the other hand, it can effectively utilize the complementary information between different modal images and the boundary morphology between different tumor sub-regions, improving classification accuracy. Furthermore, each local ROI includes a portion of the glioma and reflects the state of the intrusion boundary between at least two tumor sub-regions. The global ROI includes the entire glioma, and the size of the global ROI is the same as the size of each local ROI. Therefore, by preserving the subtle structural features of each tumor sub-region from multiple perspectives through multiple local ROIs, and combining this with the spatial distribution context information of the entire glioma provided by the global ROI, it is possible to effectively avoid the reduction in processing accuracy while saving computational overhead, thus achieving a better balance between processing accuracy and computational efficiency.

[0112] This application provides a controller that can be used to execute the multimodal MRI glioma classification method provided in the above-described method embodiments. See also Figure 11 The controller 300 includes a processor 301, which is used for: Based on the multimodal MR images to be processed, multiple tumor sub-regions of glioma are obtained, including: necrosis sub-region, enhancement sub-region and edema sub-region; Based on multiple tumor sub-regions, multiple regions of interest (ROIs) were extracted. Based on multiple local ROIs and the global ROI of glioma, subtypes of glioma are identified. Each local ROI includes a portion of the glioma and reflects the state of the infiltration boundary between at least two tumor subregions; the global ROI includes the entire glioma, and the size of the global ROI is the same as that of each local ROI.

[0113] Optionally, the processor 301 can be used for: By taking the centroid of each tumor subregion as the center, a local ROI is extracted, resulting in multiple local ROIs.

[0114] Optionally, the tumor percentage of the partial glioma included in each local ROI is greater than the ratio threshold, where the tumor percentage is the proportion of the partial glioma included in the local ROI to the total number of gliomas.

[0115] Optionally, the processor 301 can be used for: Multiple local and global ROIs are input into the target model to obtain the subtypes of gliomas output by the target model.

[0116] Optionally, the target model can also output the first segmentation result for each tumor sub-region.

[0117] Optionally, the processor 301 can also be used for: Before inputting multiple local ROIs and global ROIs into the target model, a pre-trained model is obtained. The pre-trained model is trained using an open-source multimodal MR image set. A target segmentation model is obtained by transferring learning the model parameters of the pre-trained model. The target model is obtained by transfer learning based on the model parameters of the target segmentation model.

[0118] Optionally, the processor 301 can be used for: Based on the multimodal MR images to be processed, the second segmentation results of multiple tumor sub-regions of glioma are obtained. The second segmentation results of each tumor sub-region are used to characterize the tumor sub-regions in the multimodal MR images.

[0119] Optionally, the processor 301 can be used for: Based on the multimodal MR images to be processed, if it is determined that the multimodal MR images include a brain tumor, then it is determined whether the brain tumor is a glioma. If the brain tumor is determined to be a glioma, then multiple tumor subregions of the glioma are obtained.

[0120] In summary, this application provides a controller capable of acquiring multiple tumor sub-regions of a glioma based on multimodal MR images to be processed, extracting multiple local areas of interest (ROIs) based on these sub-regions, and then determining the subtype of the glioma based on these local ROIs and the global ROI. Since the subtype can be determined using multimodal MR images, on the one hand, compared to pathological diagnosis after biopsy or surgery, non-invasive identification of the subtype is achieved, avoiding the risks of bleeding and infection associated with subtype identification; on the other hand, it effectively utilizes complementary information between different modal images and the boundary morphology between different tumor sub-regions, improving classification accuracy. Furthermore, each local ROI includes a portion of the glioma and reflects the state of the intrusion boundary between at least two tumor sub-regions. The global ROI includes the entire glioma, and the size of the global ROI is the same as the size of each local ROI. Therefore, by preserving the subtle structural features of each tumor sub-region from multiple perspectives through multiple local ROIs, and combining this with the spatial distribution context information of the entire glioma provided by the global ROI, it is possible to effectively avoid the reduction in processing accuracy while saving computational overhead, thus achieving a better balance between processing accuracy and computational efficiency.

[0121] like Figure 11 As shown, the controller 300 may further include a memory 303. The processor 301 and the memory 303 are connected, for example, via a bus 302. Optionally, the controller 300 may also include a transceiver 304. It should be noted that in practical applications, the transceiver 304 is not limited to one, and the structure of the controller 300 does not constitute a limitation on the embodiments of this application.

[0122] Processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 301 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0123] Bus 302 may include a pathway for transmitting information between the aforementioned components. Bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 302 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 11 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0124] The memory 303 stores a computer program corresponding to the multimodal MRI glioma classification method provided in the above embodiments of this application. This computer program is executed under the control of the processor 301. The processor 301 executes the computer program stored in the memory 303 to implement the content shown in the aforementioned method embodiments.

[0125] This application also provides a multimodal MRI glioma classification device, which can be used to implement the multimodal MRI glioma classification method provided in the above-described method embodiments. The device may include: The acquisition module is used to acquire multiple tumor sub-regions of glioma based on the multimodal MR image to be processed. The multiple tumor sub-regions include: necrosis sub-region, enhancement sub-region and edema sub-region. The extraction module is used to extract multiple regions of interest (ROIs) based on multiple tumor sub-regions. The determination module is used to determine the subtype of glioma based on multiple local ROIs and the global ROI of the glioma; Each local ROI includes a portion of the glioma and reflects the state of the infiltration boundary between at least two tumor subregions; the global ROI includes the entire glioma, and the size of the global ROI is the same as that of each local ROI.

[0126] In summary, this application provides a glioma classification device based on multimodal MRI. This device can acquire multiple tumor sub-regions of a glioma based on the multimodal MR images to be processed, and extract multiple local areas of interest (ROIs) based on these sub-regions. Then, based on these local ROIs and the global ROI, the subtype of the glioma is determined. Since the subtype can be determined using multimodal MR images, on the one hand, compared to pathological diagnosis after biopsy or surgery, non-invasive identification of the subtype is achieved, avoiding the risks of bleeding and infection associated with subtype identification; on the other hand, it can effectively utilize the complementary information between different modal images and the boundary morphology between different tumor sub-regions, improving classification accuracy. Furthermore, each local ROI includes a portion of the glioma and reflects the state of the intrusive boundary between at least two tumor sub-regions. The global ROI includes the entire glioma, and the size of the global ROI is the same as the size of each local ROI. Therefore, by preserving the subtle structural features of each tumor sub-region from multiple perspectives through multiple local ROIs, and combining this with the spatial distribution context information of the entire glioma provided by the global ROI, it is possible to effectively avoid the reduction in processing accuracy while saving computational overhead, thus achieving a better balance between processing accuracy and computational efficiency.

[0127] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the multimodal MRI-based glioma classification method as described above. For example, Figure 1 or Figure 2 The method described.

[0128] This application provides a computer program product, which includes a computer program or computer instructions that, when executed by a processor, implement the multimodal MRI-based glioma classification method as described above. For example, Figure 1 or Figure 2 The method described.

[0129] This application provides an electronic device, which includes a controller as described in the above-described device embodiments. For example, Figure 11 The controller shown.

[0130] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0131] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0132] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0133] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0134] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0135] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for classifying gliomas based on multimodal MRI, characterized in that, The method includes: Based on the multimodal MR images to be processed, multiple tumor sub-regions of glioma are obtained, including: necrotic sub-regions, enhancement sub-regions, and edema sub-regions; Based on the multiple tumor sub-regions, multiple regions of interest (ROIs) were extracted. Based on the multiple local ROIs and the global ROI of the glioma, the subtype of the glioma is determined; Each of the local ROIs includes a portion of the glioma and reflects the state of the infiltration boundary between at least two tumor sub-regions; the global ROI includes the entire glioma, and the size of the global ROI is the same as the size of each of the local ROIs.

2. The method according to claim 1, characterized in that, Based on the multiple tumor sub-regions, multiple regions of interest (ROIs) are extracted, including: A local region of interest (ROI) is extracted centered on the centroid of each of the tumor sub-regions, resulting in multiple local ROIs.

3. The method according to claim 1, characterized in that, The tumor percentage of the portion of the glioma included in each of the local ROIs is greater than a ratio threshold, where the tumor percentage is the proportion of the portion of the glioma included in the local ROI to the entire glioma.

4. The method according to claim 1, characterized in that, Based on the multiple local ROIs and the global ROI of the glioma, the subtype of the glioma is determined, including: The multiple local ROIs and the global ROI are all input into the target model to obtain the subtype of the glioma output by the target model.

5. The method according to claim 4, characterized in that, The target model can also output the first segmentation result for each of the tumor sub-regions.

6. The method according to claim 5, characterized in that, Before inputting the multiple local ROIs and the global ROI into the target model, the method further includes: A pre-trained model is obtained, which is trained using an open-source multimodal MR image set; Based on the model parameters of the pre-trained model, transfer learning is performed to obtain the target segmentation model; The target model is obtained by performing transfer learning based on the model parameters of the target segmentation model.

7. The method according to any one of claims 1 to 6, characterized in that, Based on the multimodal MR images to be processed, multiple tumor sub-regions of glioma are obtained, including: Based on the multimodal MR image to be processed, second segmentation results of multiple tumor sub-regions of glioma are obtained, and the second segmentation results of each tumor sub-region are used to characterize the tumor sub-region in the multimodal MR image.

8. The method according to any one of claims 1 to 6, characterized in that, Based on the multimodal MR images to be processed, multiple tumor sub-regions of glioma are obtained, including: Based on the multimodal MR images to be processed, if it is determined that the multimodal MR images include a brain tumor, then it is determined whether the brain tumor is a glioma. If the brain tumor is determined to be a glioma, then multiple tumor sub-regions of the glioma are obtained.

9. A controller, characterized in that, The controller includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the method as described in any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-8.

11. An electronic device, characterized in that, The electronic device includes: the controller as described in claim 9.

Images