Brain glioma subtype segmentation identification method, electronic equipment and storage medium
By combining end-to-end models of brain glioma segmentation and imagingomics methods, the problem of independent tasks of brain glioma segmentation and subtype recognition in the prior art is solved, and better segmentation effect and subtype recognition are achieved. Accuracy.
Patent Information
- Application Number
- CN202411541148.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-31
- Publication Date
- 2025-02-14
AI Technical Summary
In the prior art, the division and subtype recognition tasks of brain glioma are independent of each other, and common knowledge cannot be effectively utilized, resulting in insufficient correctness of the results.
An end-to-end model combining brain glioma segmentation and imagingomics methods is proposed, and brain glioma is realized through convolutional neural network encoder, segmented image decoder, imagingomic feature attention module and subtype classification branch Identification and segmentation of tumor subtypes.
By simultaneously training the MRI image segmentation task and subtype classification recognition task of brain glioma, better segmentation effect and subtype recognition accuracy are achieved, which improves recognition accuracy by 30.51% and 13.33% compared with traditional methods.
Smart Images
Figure CN119444770A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of image processing technology, and relates to computer vision, medical image processing and deep learning technology. It proposes a combination of brain glioma segmentation and imaging genomics methods, and is a brain glioma subtype segmentation and identification method, electronic device and storage medium. Background Art
[0002] With the continuous development of computer technology and deep learning technology, artificial intelligence and deep learning technology have begun to be widely used in auxiliary diagnosis in the medical context. As part of computer-aided diagnosis, computer vision target segmentation tasks have received increasing attention in the analysis of image results in medical imaging. Medical auxiliary diagnosis technology based on deep learning target segmentation has also been applied to the diagnosis of brain gliomas.
[0003] Most of the current deep learning neural networks perform segmentation of glioma MR (Magnetic Resonance) images alone, or diagnose subtypes based only on glioma segmentation maps annotated by professional physicians, but lack an end-to-end process that directly predicts segmentation results and glioma subtypes from MR images. Common medical image segmentation models include U-Net, etc., and common glioma subtype identification models include models based on strongly supervised machine learning using radiomics features. In the past, the practice of deep learning-based diagnostic models was mostly to train models separately for medical image-based glioma segmentation and radiomics-based glioma subtype classification. The two models do not share inputs. The two tasks have a certain correlation. Completing the two tasks alone cannot effectively utilize the common knowledge between the two tasks, but the two models cannot directly share inputs due to different learning tasks.
[0004] Therefore, the present invention proposes an end-to-end model that combines glioma segmentation and radiomics methods for glioma subtype identification, while utilizing MRI images and radiomics knowledge to achieve better performance in both glioma segmentation and subtype classification. Summary of the invention
[0005] The purpose of the present invention is to provide a model construction method and application for glioma subtype classification combining glioma segmentation and imaging genomics methods, so as to solve the problem that the two existing tasks are independent of each other and cannot effectively utilize common knowledge, thereby effectively improving the correctness of the results of the two tasks.
[0006] The technical solution of the present invention is: a method for segmenting and identifying glioma subtypes, for multimodal MR images composed of four MR imaging modalities of glioma patients, a segmentation and recognition model is constructed to identify glioma subtypes, the segmentation and recognition model includes a convolutional neural network encoder, a segmentation image decoder, an imaging genomics feature attention module and a subtype classification branch, the convolutional neural network encoder encodes the MR image to extract MR image features, the segmentation image decoder performs lesion segmentation based on the MR image features, the imaging genomics feature attention module extracts glioma label features based on imaging genomics, the MR image features and the glioma label features are input into the subtype classification branch to identify glioma subtypes, and the magnetic resonance imaging segmentation task and the glioma subtype recognition task are completed at the same time, wherein when the segmentation and recognition model is trained, the segmentation task and the subtype recognition task are respectively calculated by the Dice loss function and the CrossEntropy loss function to obtain the corresponding loss L dc and L ce , and get the total loss of the model.
[0007] Furthermore, in the segmentation recognition model, a convolutional neural network encoder and a segmentation image decoder are constructed based on convolution blocks, maximum pooling blocks, transposed convolution blocks, output convolution blocks, fully connected blocks, output fully connected blocks, and global average pooling blocks; a radiomics feature attention module is constructed based on the fully connected block to encode high-dimensional radiomics features; a subtype classification branch is constructed based on the fully connected module to classify brain glioma subtypes by combining MR image features and radiomics features;
[0008] The convolutional neural network encoder accepts a magnetic resonance image as input, and implements multi-level downsampling and encoding of the input magnetic resonance image through continuous multi-layer convolution blocks and maximum pooling blocks; the segmented image decoder accepts the encoding output of each level output by the convolutional neural encoder, uses transposed convolution blocks and convolution blocks, and combines the encoding output of each level to implement hierarchical upsampling from the lowest sampling result, and uses the output convolution block to implement the output of the final result; the imaging genomics feature attention module accepts imaging genomics features, and automatically implements imaging genomics feature selection and extraction through deep learning; the subtype classification branch combines the multi-layer downsampling encoding results of the convolution encoder and the imaging genomics features provided by the imaging genomics feature attention module to realize the identification of brain glioma subtypes.
[0009] Furthermore, each of the four MR imaging modalities of glioma patients, T1, T2, Flair, and T1ce, was cropped to a size of 1×128×128×128, and the four modality data were spliced to form a 4×128×128×128 multimodal MR image, and omics features were extracted based on the multimodal glioma images and the corresponding glioma segmentation maps.
[0010] Furthermore, the subtype classification branch transmits the gradient to the convolutional neural network encoder through gradient backpropagation, thereby optimizing the image features extracted from the image.
[0011] Furthermore, during the training of the segmentation recognition model, the loss L dc and L ce Based on the weights w1 and w2, the total loss L is obtained. total =w1L dc +w2L ce .
[0013] The present invention also provides an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the segmentation and recognition model of the above-mentioned glioma subtype segmentation and recognition method, which is used for identifying glioma subtypes in multimodal MR images.
[0014] The present invention also provides a computer-readable storage medium, in which at least one instruction or at least one program is stored. When the at least one instruction or at least one program is executed, a segmentation and recognition model of the above-mentioned brain glioma subtype segmentation and recognition method is implemented.
[0015] The model proposed in the present invention combines glioma segmentation and imaging genomics methods for end-to-end glioma subtype discrimination, which simultaneously realizes glioma MRI image segmentation and subtyping, and provides more accurate results.
[0016] The beneficial effects of the present invention are:
[0017] The present invention simultaneously trains the glioma MRI image segmentation task and the subtype classification and recognition task, and simultaneously learns the glioma MR image and imaging genomics features, thereby achieving better results in both tasks. Compared with the existing commonly used glioma medical image segmentation network or glioma subtype recognition model, the present invention has the following advantages:
[0018] 1). The present invention proposes a method for identifying glioma subtypes by combining glioma segmentation and imaging omics methods. In the segmentation and recognition model, additional subtype classification information is passed into the classifier during back propagation to assist the segmentation model in segmentation. The segmentation effect is about 30.51% better than the classic segmentation network U-Net when tested in the non-TCIA part of the BraTS2018 training set according to the Hausdorff95 WT standard.
[0019] 2). The segmentation and recognition model constructed by the method of the present invention uses features extracted by semantic segmentation in radiomics. The recognition effect of the model is about 7% better than the subtype recognition method based only on radiomics and random forests in terms of recognition accuracy for high-grade gliomas and low-grade gliomas, and about 13.33% better than the subtype recognition method based only on convolutional neural networks.
[0020] 3) The present invention combines glioma segmentation and imaging genomics methods to construct a glioma subtype recognition model, which performs segmentation and recognition of multimodal MR images. The two tasks interact with each other and have better MR image glioma segmentation and glioma subtype recognition effects. It has extremely high application and promotion value in the field of deep learning and medical image processing technology. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 It is a schematic diagram of the structure of the brain glioma subtype identification model of the present invention. DETAILED DESCRIPTION
[0022] The present invention uses radiomics knowledge and deep learning methods in a comprehensive manner, integrates radiomics features into image features to give segmentation results based on nuclear magnetic resonance images, and combines radiomics features, nuclear magnetic resonance image features and glioma label features to give glioma subtype classification results, thereby completing glioma subtype identification and lesion segmentation end-to-end. The method of the present invention constructs a segmentation recognition model for multimodal MR images composed of four MR imaging modalities of glioma patients to identify glioma subtypes. The segmentation recognition model is composed of a convolutional neural network encoder, a segmentation image decoder, an radiomics feature attention module and a subtype classification branch; the output results of each module are connected using a splicing method, and the model construction method is as follows.
[0023] Data processing: Each of the four MR imaging modalities of glioma patients, T1, T2, Flair, and T1ce, was cut into a size of 1×128×128×128, and the four modality data were spliced to form a 4×128×128×128 multimodal MR image. Omic features were extracted based on the multimodal glioma images and the corresponding glioma segmentation maps, and a total of 13,488 omics features were extracted;
[0024] Network construction: A neural network encoder and a segmentation image decoder are constructed based on convolution blocks, maximum pooling blocks, transposed convolution blocks, output convolution blocks, fully connected blocks, output fully connected blocks, and global average pooling blocks, which are used to extract features from multimodal MR images and segment lesions based on MR image features, respectively. An imaging feature attention module is constructed based on the fully connected block to encode high-dimensional imaging features. A subtype classification branch is constructed based on the fully connected module to classify glioma subtypes by combining MR image features and imaging features. The convolutional neural network encoder accepts magnetic resonance images as input, and realizes multi-level downsampling and encoding of the input magnetic resonance images through continuous multi-layer convolution blocks and maximum pooling blocks. The segmentation image decoder accepts the encoding outputs of each level output by the convolutional neural encoder, uses transposed convolution blocks and convolution blocks, combines the encoding outputs of each level, realizes upsampling by level from the lowest sampling result, and uses the output convolution block to realize the output of the final result. The radiomics feature attention module receives radiomics features and automatically selects and extracts radiomics features through deep learning. The subtype classification branch combines the multi-level downsampling encoding results of the convolutional encoder and the radiomics features provided by the radiomics feature attention module to achieve brain glioma subtype identification;
[0025] Network training: The multimodal MR images and the corresponding radiomics features are input into the glioma subtype recognition model through the neural network encoder and the radiomics feature attention module respectively. The multimodal MR images are encoded by the neural network encoder to obtain the image features, and the radiomics features are encoded by the radiomics feature attention module to obtain the radiomics features. The image features are further input into the segmentation image decoder for decoding to obtain the glioma lesion segmentation map predicted by the model. At the same time, the radiomics features are also input into the subtype classification branch, spliced with the radiomics features, and decoded by the subtype classification branch to output the model's prediction of the glioma subtype. The segmentation map prediction and subtype prediction of the model are calculated using the Dice loss function and the CrossEntropy loss function respectively to obtain the corresponding loss L dc and L ce , and weighted calculation is performed based on weights w1 and w2, and the final loss is L total =w1L dc +w2L ce .
[0027] The present invention is further described in detail below through a specific embodiment. The present invention combines brain glioma segmentation and imaging genomics methods to perform brain glioma subtype classification model, which is composed of a convolutional neural network encoder, a segmentation image decoder, and a subtype classification branch, a total of 14 convolution blocks, 3 transposed convolution blocks, 3 maximum pooling blocks, 4 global average pooling blocks, 3 fully connected blocks, 1 output convolution block, and 1 output fully connected block.
[0028] like Figure 1 As shown, the present invention combines glioma segmentation and imaging omics methods to perform glioma subtype classification model, which consists of a convolutional neural network encoder, a segmentation image decoder, and a subtype classification branch, with a total of 14 convolution blocks: AC1, AC2, AC3, AC4, AC5, AC6, AC7, AC8, BC9, BC10, BC11, BC12, BC13, BC14, 3 transposed convolution blocks: BT1, BT2, BT3, 3 maximum pooling blocks: AM1, AM2, AM3, 4 global average pooling blocks: CP1, CP2, CP3, CP4, 3 fully connected blocks: DF1, DF2, DF3, 1 output convolution block: BO, 1 output fully connected block: CO, wherein:
[0029] Each convolution block consists of a convolution layer and a LeakyReLU (LReLU) activation function layer connected linearly; the convolution kernel size of the convolution layer is 3×3×3, and the stride of the convolution kernel is 1;
[0030] Each maximum pooling block consists of a maximum pooling function layer; the pooling kernel size of the maximum pooling function layer is 2×2×2, and the step size of the pooling kernel is 2;
[0031] Each transposed convolution block consists of a transposed convolution layer; the transposed convolution kernel size of the transposed convolution layer is 2×2×2, and the stride of the pooling kernel is 2;
[0032] Each output convolution block consists of a convolution layer and a Softmax function layer connected linearly; the convolution kernel size of the convolution layer is 1×1×1, and the step size of the pooling kernel is 1;
[0033] Each fully connected block consists of a fully connected layer, an LReLU activation function layer, and a Dropout layer linearly connected;
[0034] Each output fully connected block consists of a Dropout layer, a fully connected layer, and a Softmax activation function layer linearly connected;
[0035] Each global average pooling block consists of a global pooling layer.
[0036] The modules of the glioma subtype classification model combined with the glioma segmentation and imaging genomics method of the present invention are as follows: Figure 1 The structural connections shown.
[0037] The method for constructing a glioma subtype classification and identification model by combining glioma segmentation and imaging omics methods of the present invention specifically comprises the following steps:
[0038] Step 1: Build the basic model modules according to the above instructions, including convolution block, maximum pooling block, transposed convolution block, output convolution block, fully connected block, output fully connected block, and global average pooling block.
[0039] Step 2: Build the convolutional neural network encoder, segmentation image decoder, radiomics feature attention, and subtype classification branches of the deep learning model.
[0040] The convolutional neural network encoder includes a first convolution block AC1, a second convolution block AC2, a first maximum pooling block AM1, a third convolution block AC3, a fourth convolution block AC4, a second maximum pooling block AM2, a fifth convolution block AC5, a sixth convolution block AC6, a third maximum pooling block AM3, a seventh convolution block AC7, and an eighth convolution block AC8, which are connected in sequence.
[0041] The segmented image decoder includes a first transposed convolution block BT1, a ninth convolution block BC9, a tenth convolution block BC10, a second transposed convolution block BT2, an eleventh convolution block BC11, a twelfth convolution block BC12, a third transposed convolution block BT3, a thirteenth convolution block BC13, a fourteenth convolution block BC14, and an output convolution block BO, which are connected in sequence.
[0042] The subtype classification branch includes: the first global average pooling block CP1, the second global average pooling block CP2, the third global average pooling block CP3, the fourth global average pooling block CP4, the first fully connected block CF1, the second fully connected block CF2, and the output fully connected block CO; CP1, CP2, CP3, and CP4 receive the outputs of AC2, AC4, AC6, and AC8 respectively, and the output of the imaging feature attention module is spliced and fused with the output of CP1, CP2, CP3, and CP4 through CF1 and CF2, and the classification recognition result is output through CO.
[0043] The radiomics feature attention module includes: the third fully connected block DF3.
[0044] Step 3: Connect the modules. Specifically, let the constructed deep learning model be N. The deep learning model is a set of directed edges. Each directed edge represents a directed connection between two modules. For example, (X, Y) means that the output of module X is connected to the input of module Y. When a module has multiple inputs, the inputs are connected in sequence in a matrix splicing manner.
[0045] The modules of the deep learning-based segmentation and recognition model constructed by the present invention are specifically connected as follows: Figure 1 As shown, the details are as follows: the first convolution block AC1 is connected to the second convolution block AC2 through a directed edge (AC1, AC2), the second convolution block AC2 is connected to the first maximum pooling block AM1 through a directed edge (AC2, AM1), the first maximum pooling block AM1 is connected to the third convolution block AC3 through a directed edge (AM1, AC3), the third convolution block AC3 is connected to the fourth convolution block AC4 through a directed edge (AC3, AC4), the fourth convolution block AC4 is connected to the second maximum pooling block AM2 through a directed edge (AC4, AM1). 2) Connection, the second maximum pooling block AM2 is connected to the fifth convolution block AC5 through a directed edge (AM2, AC5), the fifth convolution block AC5 is connected to the sixth convolution block AC6 through a directed edge (AC5, AC6), the sixth convolution block AC6 is connected to the third maximum pooling block AM3 through a directed edge (AC6, AM3), the third maximum pooling block AM3 is connected to the seventh convolution block AC7 through a directed edge (AM3, AC7), and the seventh convolution block AC7 is connected to the eighth convolution block AC8 through a directed edge (AC7, AC8);
[0046] The sixth convolution block AC6 is connected to the ninth convolution block BC9 through a directed edge (AC6, BC9), the fourth convolution block AC4 is connected to the eleventh convolution block BC11 through a directed edge (AC4, BC11), the second convolution block AC2 is connected to the thirteenth convolution block BC13 through a directed edge (AC2, BC13), the eighth convolution block AC8 is connected to the first transposed convolution block BT1 through a directed edge (AC8, BT1), and the first transposed convolution block BT1 is connected to the ninth convolution block BC9 through a directed edge (BT1, BC9);
[0047] The ninth convolution block BC9 is connected to the tenth convolution block BC10 through a directed edge (BC9, BC10), the tenth convolution block BC10 is connected to the second transposed convolution block BT2 through a directed edge (BC10, BT2), the second transposed convolution block BT2 is connected to the eleventh convolution block BC11 through a directed edge (BT2, BC11), the eleventh convolution block BC11 is connected to the twelfth convolution block BC12 through a directed edge (BC11, BC12), the twelfth convolution block BC12 is connected to the third transposed convolution block BT3 through a directed edge (BC12, BT3), the third transposed convolution block BT3 is connected to the thirteenth convolution block BC13 through a directed edge (BT3, BC13), the thirteenth convolution block BC13 is connected to the fourteenth convolution block BC14 through a directed edge (BC13, BC14), and the fourteenth convolution block BC14 is connected to the output convolution block BO through a directed edge (BC14, BO);
[0048] The second convolution block AC2 is connected to the first global average pooling block CP1 through a directed edge (AC2, CP1), the fourth convolution block AC4 is connected to the second global average pooling block CP2 through a directed edge (AC4, CP2), the sixth convolution block AC6 is connected to the third global average pooling block CP3 through a directed edge (AC6, CP3), and the eighth convolution block AC8 is connected to the fourth global average pooling block CP4 through a directed edge (AC8, CP4);
[0049] The first fully connected block CF1 is connected to the second fully connected block CF2 through a directed edge (CF1, CF2), the second fully connected block CF2 is connected to the output fully connected block CO through a directed edge (CF2, CO), the first global average pooling block CP1 is connected to the output fully connected block CO through a directed edge CP1, CO, the second global average pooling block CP2 is connected to the output fully connected block CO through a directed edge (CP2, CO), the third global average pooling block CP3 is connected to the output fully connected block CO through a directed edge (CP3, CO), and the fourth global average pooling block CP4 is connected to the output fully connected block CO through a directed edge (CP4, CO);
[0050] The third fully connected block DF3 is connected to the first fully connected block CF1 through a directed edge (DF3, CF1).
[0051] The constructed deep learning model N is expressed by the following formula:
[0052] N={(AC1,AC2),(AC2,AM1),(AM1,AC3),(AC3,AC4),(AC4,AM2),(AM2,AC5),(AC5,AC6),(AC6,AM3),(AM3 ,AC7),(AC7,AC8),(AC6,BC9),(AC4,BC11),(AC2,BC13),(AC8,BT1),(BT1,BC9),(BC9,BC10),(BC10,BT2 ),(BT2,BC11),(BC11,BC12),(BC12,BT3),(BT3,BC13),(BC13,BC14),(BC14,BO),(AC2,CP1),(AC4,CP2 ),(AC6,CP3),(AC8,CP4),(DF3,CF1),(CF1,CF2),(CF2,CO),(CP1,CO),(CP2,CO),(CP3,CO),(CP4,CO)}.
[0053] Step 4: Set the loss function of the model. The loss function of the segmentation recognition model constructed by the present invention consists of two parts, which are respectively calculated by the Dice loss function and the cross entropy loss function. Let the loss function be L total, whose expression is L total =w1L dc +w2L ce , where w1 is the control L dc The weight of L ce The weight, L dc For Dice Loss, L ce is the cross entropy loss function.
[0054] The segmentation recognition model of the present invention uses the Nesterov's Accelerated Gradient (NAG) model optimizer when deployed, and sets the momentum parameter to 0.9, the initial learning rate to 0.01, and if the loss does not decrease after 30 rounds of training, the learning rate attenuation factor is set to 0.1.
[0055] The existing classification of glioma subtypes relies heavily on the lesion segmentation maps carefully annotated by doctors, and lacks an end-to-end model that can directly predict the segmentation results and glioma subtypes from MR images. In the past, the practice of deep learning-based diagnostic models was mostly to train models separately for glioma segmentation based on medical images and glioma subtype classification based on radiomics, and the two models did not share inputs. The method of the present invention splices multimodal MR images, extracts radiomics features based on multimodal glioma images and corresponding glioma segmentation maps, and couples the glioma segmentation network and the glioma subtype classification network based on image features. In the subtype classification branch, subtype prediction is performed by combining image features and radiomics features; the subtype classification branch transmits gradients to the neural network encoder through gradient backpropagation, thereby optimizing the image features extracted from the image.
[0056] An embodiment of the present invention also provides an electronic device, including a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the segmentation and recognition model of the glioma subtype segmentation and recognition method described in the above embodiment, which is used for identifying glioma subtypes in multimodal MR images.
[0057] The embodiment of the present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program implements the segmentation recognition model of the above-mentioned brain glioma subtype segmentation recognition method when executed by a processor of a computer. The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as an independent software package and partially on a remote machine, or entirely on a remote machine or server.
[0058] The present invention combines the results of MRI image segmentation with the features of radiomics, and simultaneously completes the two tasks of MRI image segmentation and glioma subtype classification, and has better performance in both tasks, and has extremely high application and promotion value in the fields of deep learning, medical image processing technology and computer-aided diagnosis. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A method for segmenting and identifying subtypes of glioma, characterized in that For multimodal MR images consisting of four MR imaging modalities of patients with glioma, a segmentation and recognition model is constructed to identify glioma subtypes. The segmentation and recognition model includes a convolutional neural network encoder, a segmentation image decoder, an imaging genomics feature attention module and a subtype classification branch. The convolutional neural network encoder encodes the MR image to extract MR image features. The segmentation image decoder performs lesion segmentation based on MR image features. The imaging genomics feature attention module extracts glioma label features based on imaging genomics. The MR image features and the glioma label features are input into the subtype classification branch to identify glioma subtypes, and the magnetic resonance imaging segmentation task and the glioma subtype recognition task are completed simultaneously. When the segmentation and recognition model is trained, the segmentation task and the subtype recognition task are calculated by the Dice loss function and the CrossEntropy loss function respectively to obtain the corresponding loss L dc and L ce , and get the total loss of the model.
2. The method for segmenting and identifying subtypes of glioma according to claim 1, characterized in that In the segmentation recognition model, a convolutional neural network encoder and a segmentation image decoder are constructed based on convolution blocks, maximum pooling blocks, transposed convolution blocks, output convolution blocks, fully connected blocks, output fully connected blocks, and global average pooling blocks; a radiomics feature attention module is constructed based on the fully connected block to encode high-dimensional radiomics features; a subtype classification branch is constructed based on the fully connected module to classify glioma subtypes by combining MR image features and radiomics features; The convolutional neural network encoder accepts a magnetic resonance image as input, and implements multi-level downsampling and encoding of the input magnetic resonance image through continuous multi-layer convolution blocks and maximum pooling blocks; the segmented image decoder accepts the encoding output of each level output by the convolutional neural encoder, uses transposed convolution blocks and convolution blocks, and combines the encoding output of each level to implement hierarchical upsampling from the lowest sampling result, and uses the output convolution block to implement the output of the final result; the imaging genomics feature attention module accepts imaging genomics features, and automatically implements imaging genomics feature selection and extraction through deep learning; the subtype classification branch combines the multi-layer downsampling encoding results of the convolution encoder and the imaging genomics features provided by the imaging genomics feature attention module to realize the identification of brain glioma subtypes.
3. The method for segmenting and identifying subtypes of glioma according to claim 1, characterized in that Each of the four MR imaging modalities of glioma patients, T1, T2, Flair and T1ce, was cropped to a size of 1×128×128×128, and the four modality data were spliced to form a 4×128×128×128 multimodal MR image. Omic features were extracted based on the multimodal glioma images and the corresponding glioma segmentation maps.
4. The method for segmenting and identifying subtypes of glioma according to claim 1, characterized in that The subtype classification branch transmits the gradient to the convolutional neural network encoder through gradient backpropagation, thereby optimizing the image features extracted from the image.
5. The method for segmenting and identifying subtypes of glioma according to claim 1, characterized in that The segmentation recognition model is trained with a loss L dc and L ce Based on the weights w1 and w2, a weighted calculation is performed to obtain the total loss L total =w1L dc +w2L ce .
6. An electronic device, characterized in that The electronic device includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the segmentation and recognition model of the glioma subtype segmentation and recognition method described in any one of claims 1 to 5, which is used for identifying glioma subtypes in multimodal MR images.
7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one instruction or at least one program. When the at least one instruction or the program is executed, the segmentation and recognition model of the method for segmenting and recognizing subtypes of glioma according to any one of claims 1 to 5 is implemented.