Medical image feature mining method and related device for central nervous system tumors

Abstract

This application discloses a method for mining medical image features for central nervous system tumors, comprising: obtaining a preoperative magnetic resonance imaging scan image and corresponding diagnostic text; performing a three-dimensional image feature extraction operation on the image image to obtain a first feature set; performing a radiomics feature extraction operation on the image image to obtain a second feature set; performing a semantic feature extraction operation on the diagnostic text to obtain a third feature set; and outputting the mining results after comprehensive processing of the first, second, and third feature sets. By combining radiomics feature extraction with semantic feature extraction of diagnostic text, the errors generated by image feature extraction based entirely on machine learning are overcome to a certain extent, and the accuracy of image feature annotation can be improved by combining medical background, thereby improving the reliability of the training model.

Description

Technical Field

[0001] The present application relates to the field of medical image analysis, and in particular to a method and related device for mining medical image features of central nervous system tumors. Background Art

[0002] Central nervous system tumors refer to tumors that arise in the brain and spinal cord. Primary brain tumors differ from tumors elsewhere in the body in the following ways: ① Even histologically benign brain tumors can still lead to death if they develop in unresectable locations (such as ependymomas at the floor of the fourth ventricle). ② Intraparenchymal brain tumors, particularly astrocytomas, generally develop infiltratively, with boundaries unclear macroscopically and histologically. Therefore, radical resection is virtually impossible. ③ Even highly malignant brain tumors rarely metastasize. Such metastasis is typically seen in glioblastomas or medulloblastomas. However, metastasis through the cerebrospinal fluid is common. ④ Certain brain tumors have a specific predilection site. For example, medulloblastomas are confined to the cerebellum. Furthermore, there is a predilection age for onset: medulloblastomas are most common in children under 10 years of age, while malignant astrocytomas and glioblastomas are more common in middle-aged or older individuals. Therefore, central nervous system tumors are easily detected through medical imaging but difficult to treat.

[0003] Taking glioblastoma (GBM) as an example, GBM is a type of astrocytoma in the central nervous system tumors. It is the most common invasive brain tumor, with 3 cases per 100,000 people each year, making it one of the most common invasive brain tumors in adults. In April 2021, the World Health Organization (WHO) divided GBM into four levels based on its malignant clinical, histological and molecular characteristics. With the development of second-generation sequencing technology and epigenetics, GBM has been further divided into different subtypes based on the presence of citrate dehydrogenase (IDH) mutations, 1p / 19q co-deletion, α-thalassemia / mental retardation syndrome X-linked (ATRX) mutations, or O6-methylguanine-DNA methyltransferase (MGMT) promoter methylation.

[0004] In recent years, histopathological diagnosis assisted by immunohistochemistry and second-generation sequencing have become standard glioma diagnostic procedures. However, these testing protocols also exhibit several limitations. First, it is difficult to obtain samples of advanced GBM and diffuse pontine gliomas due to their inaccessible tumor location and the high surgical risks associated with them. Second, molecular subtype reports based on tumor samples from a single lesion area do not represent the entire tumor. GBM exhibits a high degree of molecular heterogeneity, resulting in inappropriate treatment plans based on molecular diagnosis and inefficient analysis. Furthermore, current second-generation sequencing processes typically require 10–14 days from DNA extraction, cDNA library construction, sequencing, to final report analysis. This long processing time can be extremely detrimental to GBM patients, whose disease may rapidly worsen. Finally, the preventive costs of second-generation sequencing and epigenetic testing limit their widespread application. Summary of the Invention

[0005] The purpose of this application is to at least partially overcome the deficiencies of the existing technology and to provide an accurate, inexpensive and convenient method for mining medical imaging features of central nervous system tumors and related application equipment, so as to provide support for subsequent molecular subtype determination and prognosis prediction.

[0006] In order to achieve the above technical objectives, the technical solutions adopted in this application are as follows:

[0007] In a first aspect, a method for mining medical image features of central nervous system tumors is provided, comprising:

[0008] Obtain preoperative magnetic resonance imaging scan images and corresponding diagnostic text;

[0009] Performing a three-dimensional image feature extraction operation on the image to obtain a first feature set;

[0010] performing a radiomics feature extraction operation on the image to obtain a second feature set;

[0011] Performing a semantic feature extraction operation on the diagnostic text to obtain a third feature set;

[0012] The first feature set, the second feature set and the third feature set are processed comprehensively to output mining results.

[0013] Optionally, after obtaining the preoperative magnetic resonance imaging scan image, the image is subjected to standardization processing.

[0014] Further optionally, the standardization process includes at least one of the following:

[0015] Performing format conversion on the image;

[0016] Normalizing the reference coordinates of the image;

[0017] performing contrast adjustment on the image;

[0018] Extracting a target area from the image;

[0019] The image is uniformly sized.

[0020] Preferably, the format conversion of the image graph includes converting a plurality of associated two-dimensional image graphs into three-dimensional image graphs.

[0021] Furthermore, before performing the three-dimensional image feature extraction operation, a tumor segmentation model is constructed using training samples, including:

[0022] The original training samples are rotated at random angles and the rotated images are collected as expanded training samples;

[0023] After performing convolution calculations on all training samples, training features are extracted to construct the brain tumor segmentation model.

[0024] Furthermore, the three-dimensional image feature extraction operation and the construction of a tumor segmentation model using training samples include:

[0025] After performing convolution calculation on the training samples, the extracted training features are classified according to the epigenetic features of the corresponding tumors.

[0026] Furthermore, it is characterized in that, when performing the radiomics feature extraction operation, the extracted second feature set is associated with the epigenetic features of the corresponding tumor.

[0027] Furthermore, before performing the semantic feature extraction operation on the diagnostic text, a semantic analysis model is constructed using training samples, including:

[0028] The training samples include the original diagnosis report text and related text records, as well as the secondary manual analysis text of the original image images;

[0029] The training samples are segmented into words based on a medical dictionary and a specified term list to construct the semantic analysis model.

[0030] Furthermore, the step of comprehensively processing the first feature set, the second feature set, and the third feature set to output a mining result includes:

[0031] fusing the first feature set, the second feature set, and the third feature set;

[0032] Obtain mining results related to target features.

[0033] Optionally, the target features include epigenetic features and prognostic features of the corresponding tumor.

[0034] Optionally, the central nervous system tumor includes astrocytoma, ependymoma, medulloblastoma or meningioma.

[0035] In a second aspect, a medical imaging feature mining device for central nervous system tumors is provided, comprising:

[0036] An acquisition module, which acquires preoperative magnetic resonance imaging scan images and corresponding diagnostic texts;

[0037] A three-dimensional processing module performs a three-dimensional image feature extraction operation on the image to obtain a first feature set;

[0038] a radiomics image processing module, performing a radiomics feature extraction operation on the image to obtain a second feature set;

[0039] A semantic processing module performs a semantic feature extraction operation on the diagnostic text to obtain a third feature set;

[0040] The comprehensive processing module comprehensively processes the first feature set, the second feature set and the third feature set to output mining results.

[0041] In a third aspect, a computer-readable medium is provided, comprising one or more applications, wherein the one or more applications are stored in a memory and configured to be executed by one or more processors, and the one or more applications are configured to execute the medical imaging feature mining method for central nervous system tumors as described above.

[0042] Compared with the existing technology, this application has the following advantages:

[0043] (1) This application overcomes the errors caused by image feature extraction based entirely on machine learning to a certain extent by combining radiomics feature extraction with semantic feature extraction of diagnostic text. It can improve the accuracy of image feature annotation by combining medical background and improve the reliability of the training model.

[0044] (2) The present application can obtain the required feature set based on mature computer neural network technology, making it easier to construct the training model of the present application, thereby greatly improving the efficiency of interpreting MRI images and the accuracy of prediction;

[0045] (3) For glioblastoma, through statistical verification, the mining results output by this application are related to some characteristics of the molecular subtypes of the samples, and are not related to the general condition and clinical condition of the individual samples, which can provide a more reliable reference for the molecular subtype classification of glioblastoma. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Schematic diagram of the process of mining medical image features of central nervous system tumors in this application.

[0047] Figure 2 Schematic diagram of the computational flow of the convolutional neural network used in this application.

[0048] Figure 3 The statistical analysis results were performed after the application was adopted, in which irrelevant evaluation indicators were displayed.

[0049] Figure 4 The statistical analysis results after adopting this application are shown, in which the relevant evaluation indicators are displayed.

[0050] Figure 5 The ROC curves corresponding to different molecular subtypes were drawn after adopting this application.

[0051] Figure 6 This is a schematic diagram of the structure of the medical imaging feature mining device for central nervous system tumors in this application. DETAILED DESCRIPTION

[0052] The present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0053] refer to Figure 1 The present application provides a method for mining medical imaging features of central nervous system tumors, comprising:

[0054] Obtain preoperative magnetic resonance imaging scan images and corresponding diagnostic text;

[0055] Performing a three-dimensional image feature extraction operation on the image to obtain a first feature set;

[0056] performing a radiomics feature extraction operation on the image to obtain a second feature set;

[0057] Performing a semantic feature extraction operation on the diagnostic text to obtain a third feature set;

[0058] The first feature set, the second feature set and the third feature set are processed comprehensively to output mining results.

[0059] Specifically, taking glioblastoma as an example, in this embodiment, preoperative head magnetic resonance imaging (MRI) images and corresponding diagnostic texts are first used as training samples. These training samples come from 329 primary GBM patients (astrocytoma, grade 4). The training samples were collected from August 1, 2015 to August 31, 2021, and these training samples were divided into a training set and a validation set according to a random ratio. In one possible implementation, this random ratio is 6:1.

[0060] Due to differences in operator experience and system differences in equipment, the MRI images in the training samples need to be homogenized before feature extraction, that is, the images need to be preprocessed and segmented. Specifically, all original MRI images are anonymized and obtained in DICOM format, which is a two-dimensional medical image format. Subsequently, SimpleITK (sitk) - a python tool is used to convert the original MRI images into 3D format (.nitfi) to obtain the three-dimensional image to be processed. Since the original MRI images are acquired on different scanners, the image spacing of each sample is inconsistent. Sitk resamples the image into standardized spacing coordinates. One possible implementation method is to apply Z-score to standardize the coordinates. The specific calculation formula is as follows:

[0061]

[0062] In the formula, μ represents the mean of the population data, σ represents the standard deviation of the population data, x represents the individual observation value, Z represents the standard score, and N represents the number of population data.

[0063] The next step is to segment the processed MRI images. This involves separating the tumor region and surrounding edema from surrounding normal anatomical structures. The Multimodal Brain Tumor Segmentation Benchmark Toolkit (BraTS) is a tool designed for brain tumor segmentation that coordinates brain tumor segmentation algorithms and enables fully automated segmentation.

[0064] BraTS adjusts the contrast of MRI images to generate T1-added imaging data (T1WI) and T2-added imaging data (T2WI), which are then segmented separately. Images identified as normal tissue are discarded, and the target area containing tumor tissue is extracted. After segmentation, all images are reshaped to a uniform size of 120*120*60 to facilitate training using a convolutional neural network (CNN) in the next step. Furthermore, to compensate for the limited size of training samples and the imbalanced data distribution caused by the clinical characteristics of GBM itself, the MRI images are rotated at random angles, such as rotating the images in three-dimensional space or flipping the images vertically and horizontally. The rotated images are collected as expanded training samples, and balance is achieved through the increase of data to better train the model.

[0065] The aforementioned problem about sample size in this embodiment is mainly caused by the epidemiological distribution of IDH1 mutant and ATRX mutant types of GBM. The characteristics of the IDH1 mutant and ATRX mutant molecular subtypes themselves lead to uneven data distribution, and the number of subjects with IDH1 mutant and ATRX mutant types that can be collected is very small, which is difficult to meet the large number of training samples required by CNN. In order to solve this problem as much as possible and reduce the impact of a single sample on the effectiveness of model training and subsequent classification and prediction results, the aforementioned method is used for data expansion, that is, the MRI image is rotated at random angles in three-dimensional space. The image data generated can appropriately balance the imbalance between the data of each molecular subtype, thereby improving the effectiveness and universality of this application.

[0066] In this embodiment, 3D deep neural network (3D ResNet50) and C3D (Convolutional3D) are the most commonly used convolutional models suitable for image feature analysis and extraction. This embodiment selects these two CNN models and compares them. The specific architectures of 3DResNet50 and C3D are as follows: Figure 2 As shown in the figure, the main difference between the two models is the different convolution kernels, which leads to different output feature values and their numbers.

[0067] In this embodiment, the pre-processed MRI image is subjected to C3D processing, alternating convolution and pooling five times, ultimately outputting 4096 feature values, from which 1000 feature values associated with glioblastoma molecular subtypes are identified to form the first feature set. In another possible implementation, the processed MRI image is subjected to 3D ResNet50 processing, alternating convolution and pooling five times, ultimately outputting 2048 feature values, from which 1000 feature values associated with glioblastoma molecular subtypes are identified to form the first feature set.

[0068] The network was trained 50 times during the training phase, using stochastic gradient descent as the optimizer, with a learning rate of 0.001, a minimum batch size of 32, a momentum of 0.5, and a weight decay of 0.1. When the validation accuracy during the training phase stabilized at more than 10 epochs, training was stopped to facilitate subsequent evaluation of the prediction performance. Furthermore, four metrics were selected to evaluate the performance of the model: precision (P), recall (R), accuracy (A), and F1 score (F1). These four metrics are defined as follows:

[0069]

[0070] Here, a True Positive (TP) is when the model correctly predicts the positive class. Similarly, a True Negative (TN) is a result when the model correctly predicts the negative class. Conversely, a False Positive (FP) is when the model incorrectly predicts the positive class. A False Negative (FN) is when a model incorrectly predicts the negative class.

[0071] Furthermore, this example uses PyRadiomics (version 2.2.0) to extract radiomic features from preprocessed and segmented MRI images. PyRadiomics is an open source platform (https: / / pyradiomics.readthedocs.io / ) for flexible and repeatable radiomic feature calculations, which can automatically perform data processing, feature definition, and batch processing. In order to reduce noise and computational complexity, each MRI image was resampled into isotropic 1mm voxels, and the grayscale value was set with a fixed width of 0.1mm. All radiomic features comply with the Imaging Biomarker Standardization Initiative (ISBI, International Symposium on Biomedical Imaging). 428 features were extracted from each sample to form the second feature set. Furthermore, the second feature set includes 72 first-order statistical features, 56 shape features, 96 Gray Level Cooccurence Matrix (GLCM) features, 64 Gray Level Run Length Matrix (GLRLM) features, 64 Gray Level Size Zone Matrix (GLSZM) features, 56 Gray Level Dependence Matrix (GLDM) features, and 20 Neighbouring Gray Tone Difference Matrix (NGTDM) features. Furthermore, through statistical analysis, as shown in Figure 2, Figure 3 As shown, these 428 radiomic features were not statistically significant in the comparison of ATRX mutant / wild-type tumors (P ≥> 0.05), nor in the comparison of MGMT methylation / unmethylation groups (P ≥> 0.05). However, when comparing IDH1 mutant and wild-type tumors, 19 of the 428 features were statistically significant (P ≤ 0.05 and FDR ≤ 0.215). These 19 features were mainly associated with various radiomic features on contrasting T1WI. These 19 features are considered to be associated with the epigenetic characteristics of molecular subtypes of glioblastoma.

[0072] like Figure 4The heat map shown here displays the radiomic signatures of patients with IDH1 mutations, demonstrating the differences in signatures between IDH1 mutant and wild-type tumors. Radiomic signatures quantify tumor characteristics that cannot be observed with the naked eye, such as intensity distribution, tumor compactness, texture pattern, and the relationship of the tumor to surrounding tissue. The IDH1 mutant group had higher scores for GLSZM_HGLZE (Gray Level Size Zone Matrix_High Gray Level Run Emphasis) and GLSZM_SAHGLE (Gray Level Size Zone Matrix_Short Run High Gray Level Emphasis), indicating a greater proportion of high gray-level values and signal intensity within the tumor. Similarly, the IDH1 wild-type group had higher scores for GLRLM_HGLRE and GLRLM_LRHGLE (Gray Level Size Zone Matrix_Long Run High Gray Level Emphasis), indicating greater regional variability and signal intensity within the tumor. The higher gray-level values within the group were concentrated within the tumor, indicating that the gradient of gray-level distribution in the MRI image correlated clearly with the tumor region. In addition, higher values of Variance, GLSZM_GLV, GLRLM_GLV, GLDM_GLV, and NGTDM_Contrast showed heterogeneity in grayscale intensity and more variation in spatial intensity in the MIR images of patients with IDH1 mutations. However, when these 19 radiomic features were selected to predict IDH1 mutation status, the accuracy rate after machine learning using the support vector machine algorithm was only about 52%, which was not as ideal as expected.

[0073] Therefore, this application introduces semantic analysis to improve the reliability of the training model. The training samples used for semantic analysis include the original diagnostic report text corresponding to the MRI image and related text records, such as medical records or clinical medical records, as well as the secondary manual analysis text of the original MRI image. The original diagnostic report text and related text records may contain the subjective judgment of the examining doctor or the treating doctor, so the original MRI image must be subjected to a secondary manual analysis to balance the subjectivity of different doctors. In this embodiment, the secondary manual analysis is performed by two radiologists with 20 years of experience who independently perform Chinese analysis and cross-check confirmation. Furthermore, medical dictionaries and technical terms related to neurosurgery, neuro-oncology and neuroimaging constitute the basis of the semantic analysis model of this application. In one possible implementation, word2vec, a standard natural language processing model, can be used to extract words. Each sample will extract 200 dimensional feature values to form a third feature set for subsequent analysis.

[0074] The diagnostic performance of a multimodal model integrating CNN, radiomics, and semantics was demonstrated. All these features were expanded to 1648 dimensions via a fully connected layer. A performance comparison of the corresponding trained models showed that the model incorporating semantic analysis had superior performance indices, demonstrating the indispensable role of semantic features in the clinical diagnosis of GBM. In the IDH1 status prediction based on the C3D model, the training results after adding semantic analysis achieved accuracy, precision, recall, and F1 scores exceeding 5%. Semantic analysis achieved an average accuracy of almost 11.32% in predicting MGMT promoter methylation status. In contrast, the model incorporating semantics achieved an accuracy of 91.11% in IDH1 status prediction. For the prediction of ATRX and MGMT status, C3D-based feature extraction outperformed 3D ResNet50, as demonstrated by accuracy, precision, recall, and F1 scores.

[0075] In addition, further analysis showed that the multimodal model integrating C3D, radiomics and semantics has the potential to improve the classification accuracy of GBM, which was further demonstrated by the AUC (Area Under Curve) values of the IDH1, ATRX and MGMT groups (0.991976, 0.961953 and 0.974955, respectively).

[0076] The imaging workflow of the multimodal model based on C3D takes an average of 5.87 seconds per patient (5 seconds for detection and extraction, 0.013 seconds for classification). Furthermore, the present application can also be used for prognosis prediction of a multimodal model that integrates C3D, radiomics and semantics. Based on the prognostic information, 183 patients were grouped according to their 1-year survival status, and 48 patients were excluded due to insufficient follow-up. The performance indices of the 1-year prognosis group included accuracy, precision, recall and F1 scores, which reached 85.71%, 81.82%, 90.00% and 85.72% respectively (82.05%, 80%, 84.21% and 82.05% of external validation 7). Further, the multimodal model based on C3D performed better in 1-year prognosis, with an AUC value of 0.988.976.

[0077] In one possible embodiment, in order to perform statistical analysis, the patient information corresponding to each training sample should also provide the results of histopathological biopsy, and the molecular subtypes of each training sample should be determined by second-generation sequencing and immunohistochemistry: ATRX mutation status, IDH1 mutation status, and MGMT promoter methylation status. This embodiment uses the Social Science Statistical Program 21.0 (SPSS, IBM, West Grove, PA, USA). The statistical method is based on the distribution and type of data. The diagnostic and prognostic performance of the diagnosis and model is evaluated by the ROC curve (calculating the AUC value) using the mean area, and the Kaplan-Meir survival curve is drawn, such as Figure 5 The results show that the P values calculated for factors such as age, gender, IDH1 mutation status, ATRX mutation status, MGMT methylation status, follow-up time, recurrent tumor volume, tumor location, resection range, molecular subtype, median survival time, and remission time are far greater than 0.05, proving that the mining method of this application has no correlation with these factors.

[0078] The method of this application can be used to analyze other central nervous system tumors, including astrocytomas, ependymomas, medulloblastomas, or meningiomas. Those skilled in the art should adjust the convolution calculation, radiomic feature extraction, and semantic analysis strategies based on the characteristics of the specific type of tumor, or determine the mining results output after comprehensive processing of the feature set based on the corresponding molecular subtype or other classification criteria.

[0079] The method of the present application will also be embodied in a medical imaging feature mining device for central nervous system tumors, specifically, Figure 6 As shown, the device includes:

[0080] An acquisition module 61 acquires preoperative magnetic resonance imaging scan images and corresponding diagnostic texts;

[0081] A three-dimensional processing module 62 performs a three-dimensional image feature extraction operation on the image to obtain a first feature set;

[0082] a radiomics image processing module 63 for performing a radiomics feature extraction operation on the image to obtain a second feature set;

[0083] The semantic processing module 64 performs a semantic feature extraction operation on the diagnostic text to obtain a third feature set;

[0084] The comprehensive processing module 65 comprehensively processes the first feature set, the second feature set and the third feature set to output a mining result.

[0085] In another possible embodiment, the medical image feature mining device for central nervous system tumors includes a processing chip and a storage medium, one or more programs are stored in the storage medium and configured to be executed by the processing chip, and the one or more programs are used to drive the processing chip structure to execute the medical image feature mining method for central nervous system tumors as described above: obtaining a preoperative magnetic resonance imaging scan image and a corresponding diagnostic text; performing a three-dimensional image feature extraction operation on the image image to obtain a first feature set; performing a radiomics feature extraction operation on the image image to obtain a second feature set; performing a semantic feature extraction operation on the diagnostic text to obtain a third feature set; and outputting the mining results after comprehensive processing of the first feature set, the second feature set, and the third feature set.

[0086] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0087] Those skilled in the art will appreciate that the present invention includes devices for performing one or more of the operations described herein. These devices may be specially designed and manufactured for the desired purpose, or they may include known devices found in general-purpose computers. These devices have computer programs stored therein, which are selectively activated or reconfigured. Such computer programs may be stored on a device (e.g., a computer) readable medium or on any type of medium suitable for storing electronic instructions and coupled to a bus, including but not limited to any type of disk (including floppy disks, hard disks, optical disks, CD-ROMs, and magneto-optical disks), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic cards, or optical cards. In other words, a readable medium includes any medium that can be used by a device (e.g., a computer) to store or transmit information in a form that can be read.

[0088] Those skilled in the art will appreciate that each block in these structural diagrams and / or block diagrams and / or flow charts, as well as combinations of blocks in these structural diagrams and / or block diagrams and / or flow charts, can be implemented using computer program instructions. Those skilled in the art will appreciate that these computer program instructions can be provided to a general-purpose computer, a specialized computer, or a processor of other programmable data processing methods for implementation, thereby executing the schemes specified in the blocks or multiple blocks in the structural diagrams and / or block diagrams and / or flow charts disclosed in the present invention through the processor of the computer or other programmable data processing method.

[0089] Those skilled in the art will appreciate that the steps, measures, and schemes in the various operations, methods, and processes discussed in the present invention may be interchanged, modified, combined, or deleted. Furthermore, other steps, measures, and schemes in the various operations, methods, and processes discussed in the present invention may also be interchanged, modified, rearranged, decomposed, combined, or deleted. Furthermore, steps, measures, and schemes in the prior art that are similar to those disclosed in the present invention may also be interchanged, modified, rearranged, decomposed, combined, or deleted.

[0090] In summary, the medical image feature mining method for central nervous system tumors of the present application includes: obtaining a preoperative magnetic resonance imaging scan image and the corresponding diagnostic text; performing a three-dimensional image feature extraction operation on the image to obtain a first feature set; performing a radiomics feature extraction operation on the image to obtain a second feature set; performing a semantic feature extraction operation on the diagnostic text to obtain a third feature set; and outputting the mining results after comprehensive processing of the first feature set, the second feature set, and the third feature set. By combining radiomics feature extraction and semantic feature extraction of diagnostic text, the errors caused by image feature extraction based entirely on machine learning are overcome to a certain extent, and the accuracy of image feature annotation can be improved in combination with the medical background, thereby improving the reliability of the training model.

[0091] The above embodiments are preferred implementation methods of the present application, but are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present application should be considered as equivalent replacement methods and are included in the scope of protection of the present application.

Claims

1. A method for mining medical image features of central nervous system tumors, characterized in that: include: Obtain preoperative magnetic resonance imaging scan images and corresponding diagnostic text; performing a three-dimensional image feature extraction operation on the image, constructing a tumor segmentation model using training samples, performing a convolution calculation on the training samples, and classifying the extracted training features according to the epigenetic features of the corresponding tumor to obtain a first feature set; performing a radiomics feature extraction operation on the image to obtain a second feature set; and correlating the extracted second feature set with an epigenetic feature of the corresponding tumor; Performing a semantic feature extraction operation on the diagnostic text to obtain a third feature set; The first feature set, the second feature set, and the third feature set are comprehensively processed to output mining results; the first feature set, the second feature set, and the third feature set are fused and expanded through a fully connected layer to obtain mining results related to target features; the target features include epigenetic features and prognostic features of the corresponding tumor.

2. The method according to claim 1, wherein After obtaining the preoperative magnetic resonance imaging scan image, the image is standardized.

3. The method according to claim 2, wherein The standardization process includes at least one of the following: Performing format conversion on the image; Normalizing the reference coordinates of the image; performing contrast adjustment on the image; Extracting a target area from the image; The image is uniformly sized.

4. The method according to claim 3, wherein The format conversion of the image graph includes converting a plurality of associated two-dimensional image graphs into three-dimensional image graphs.

5. The method according to claim 1, wherein: Before performing the three-dimensional image feature extraction operation, a tumor segmentation model is constructed using training samples, including: The original training samples are rotated at random angles and the rotated images are collected as expanded training samples; After performing convolution calculations on all training samples, training features are extracted to construct the tumor segmentation model.

6. The method according to claim 1, wherein Before performing the semantic feature extraction operation on the diagnostic text, a semantic analysis model is constructed using training samples, including: The training samples include the original diagnosis report text and related text records, as well as the secondary manual analysis text of the original image images; The training samples are segmented into words based on a medical dictionary and a specified term list to construct the semantic analysis model.

7. The method according to claim 1, wherein The central nervous system tumors include astrocytoma, ependymoma, medulloblastoma or meningioma.

8. A medical image feature mining device for central nervous system tumors, characterized in that: include: An acquisition module, used to acquire preoperative magnetic resonance imaging scan images and corresponding diagnostic texts; a three-dimensional processing module, configured to perform a three-dimensional image feature extraction operation on the image, construct a tumor segmentation model using training samples, perform a convolution calculation on the training samples, and classify the extracted training features according to the epigenetic features of the corresponding tumor to obtain a first feature set; a radiomics feature extraction module for the radiomics image to obtain a second feature set; and the extracted second feature set is associated with the epigenetic features of the corresponding tumor. A semantic processing module, configured to perform a semantic feature extraction operation on the diagnostic text to obtain a third feature set; A comprehensive processing module is used to output mining results after comprehensive processing of the first feature set, the second feature set and the third feature set; after fusing and expanding the first feature set, the second feature set and the third feature set through a fully connected layer, a mining result related to the target feature is obtained; the target feature includes the epigenetic features and prognostic features of the corresponding tumor.

9. A computer-readable medium, characterized in that The one or more applications are stored in a memory and configured to be executed by one or more processors, and the one or more applications are configured to execute the medical imaging feature mining method for central nervous system tumors as described in any one of claims 1 to 7.

Images