Neurological function prediction method and device based on radiomics, electronic equipment and storage medium

Abstract

The invention discloses a neurological function prediction method and device based on radiomics, electronic equipment and a storage medium. The method comprises the following steps: acquiring magnetic resonance imaging; constructing a first neural network model by using the two-dimensional convolution architecture, the three-dimensional convolution architecture, the encoder and the decoder, and segmenting the magnetic resonance imaging by using the first neural network model to obtain an image of the region of interest; performing feature extraction on the image of the region of interest to obtain feature data of magnetic resonance imaging; performing feature screening on the feature data by combining a filtering method and an embedding method; and constructing a second neural network model by using a one-dimensional convolution architecture, a gating circulation module and a full-connection architecture, and performing neural function prediction according to the screened feature data by using the second neural network model. The method is a brand-new neurological function prediction method based on radiomics, and the function of predicting the postoperative neurological function retention condition before an operation is achieved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence research, and more specifically, to a method, device, electronic device and storage medium for predicting neural function based on imaging omics. Background Art

[0002] In the treatment of some central nervous system tumors, surgery is still the preferred treatment method for most doctors. However, it is difficult to determine whether the corresponding central nervous system function can be preserved when the tumor is removed during surgery. If the preservation of facial nerve function after surgery can be predicted before surgery, an individualized treatment plan can be formulated in advance to avoid irreversible neurological damage caused by excessive surgery.

[0003] At present, deep learning and machine learning methods are increasingly being used in postoperative prognosis research. However, existing deep learning and machine learning methods have shortcomings in processing related images, including the large workload of manual image segmentation and the easy occurrence of subjective errors, the low accuracy of segmentation methods, the difficulty of feature extraction and screening, and the easy overfitting and poor classification accuracy of postoperative prognosis prediction models under small sample data.

[0004] Therefore, how to improve the existing deep learning and machine learning methods to make them more suitable for postoperative prognosis prediction and improve prediction efficiency and accuracy has become an urgent problem to be solved in this field. Summary of the invention

[0005] In view of this, in the first aspect, the present application proposes a neurological function prediction method based on imaging omics, comprising:

[0006] Obtain magnetic resonance imaging;

[0007] Using a two-dimensional convolutional architecture, a three-dimensional convolutional architecture, an encoder and a decoder to construct a first neural network model, and using the first neural network model to segment the magnetic resonance imaging to obtain an image of a region of interest;

[0008] Extracting features from the image of the region of interest to obtain feature data of magnetic resonance imaging;

[0009] The feature data is subjected to feature screening by combining filtering method and embedding method; wherein the filtering method includes: one or more of Spearman coefficient test, independent sample t test and Levene test; the embedding method includes: Lasso regression test;

[0010] A second neural network model is constructed using a one-dimensional convolutional architecture, a gated recurrent module, and a fully connected architecture, and the second neural network model is used to predict neural function based on the screened feature data.

[0011] Preferably, segmenting the magnetic resonance imaging using the first neural network model to obtain an image of a region of interest includes:

[0012] sequentially using two two-dimensional convolutional layers and two three-dimensional convolutional layers to perform convolution operations on the magnetic resonance imaging to obtain a feature map;

[0013] Performing linear mapping on the feature map to obtain a vector sequence of the feature map;

[0014] Using the encoder, performing global semantic extraction on the vector sequence to obtain a global semantic encoding;

[0015] The decoder is used to perform a deconvolution operation on the global semantic code to obtain an image of the region of interest.

[0016] Further preferred:

[0017] The two two-dimensional convolutional layers are two two-dimensional convolutional layers with residual connections, so that the in-plane resolution and inter-plane resolution of the feature map are consistent.

[0018] Further preferred:

[0019] The encoder is a Transformer encoder;

[0020] The decoder is a mirrored deconvolution layer of the two two-dimensional convolutional layers and the two three-dimensional convolutional layers.

[0021] Preferably, the Levene test comprises:

[0022] Calculate the W value of the Levene test of the characteristic data;

[0023] Searching the F distribution table of the W value of the characteristic data to obtain the p value corresponding to the W value of the characteristic data;

[0024] If the p value corresponding to the W value of the characteristic data is not greater than a first preset value, the characteristic data is excluded.

[0025] Preferably, the independent sample t-test includes:

[0026] Calculate the t value of the independent sample t test of the characteristic data;

[0027] Comparing the t value of the characteristic data with the theoretical t value distribution to obtain the p value corresponding to the t value of the characteristic data;

[0028] If the p value corresponding to the t value of the characteristic data is smaller than a second preset value, the characteristic data is excluded.

[0029] Preferably, the Spearman coefficient test includes:

[0030] Calculate the Spearman coefficient ρ of the characteristic data;

[0031] If the Spearman coefficient ρ of the feature data is not greater than a third preset value, the feature data is excluded.

[0032] Preferably, the combining filtering method and embedding method to perform feature screening on the feature graph includes:

[0033] The characteristic data are sequentially subjected to Levene test, independent sample t test, Spearman coefficient test and Lasso regression test.

[0034] Preferably, using the second neural network model to predict neural function according to the screened feature data includes:

[0035] Using three one-dimensional convolutional layers, extracting spatial features of the filtered feature data;

[0036] Using the gated recurrent module, learning the context information of the spatial features to obtain sequence features of feature data;

[0037] Using three fully connected layers, the sequence features are mapped to probability intervals to obtain classification probability values;

[0038] A neural function prediction is performed according to the classification probability value.

[0039] In a second aspect, an embodiment of the present invention further provides a neurological function prediction device based on radiomics, the device comprising:

[0040] an acquisition module configured to acquire a magnetic resonance image;

[0041] A first network module is configured to construct a first neural network model using a two-dimensional convolutional architecture, a three-dimensional convolutional architecture, an encoder, and a decoder, and segment the magnetic resonance imaging using the first neural network model to obtain an image of a region of interest;

[0042] A feature extraction module is configured to extract features from the image of the region of interest to obtain feature data of magnetic resonance imaging;

[0043] A feature screening module is configured to perform feature screening on the feature data by combining a filtering method and an embedding method; wherein the filtering method includes: one or more of a Spearman coefficient test, an independent sample t test, and a Levene test; and the embedding method includes: a Lasso regression test;

[0044] The second network module is configured to construct a second neural network model using a one-dimensional convolutional architecture, a gated recurrent module and a fully connected architecture, and use the second neural network model to predict neural function based on the screened feature data.

[0045] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the method disclosed in the first aspect above.

[0046] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the method disclosed in the first aspect above.

[0047] The neural function prediction method provided in the present application uses a two-dimensional convolutional architecture, a three-dimensional convolutional architecture, an encoder and a decoder to construct a first neural network model to segment the region of interest in magnetic resonance imaging, and then extracts and screens features of the image of the region of interest. Finally, a one-dimensional convolutional architecture, a gated loop module and a fully connected architecture are used to construct a second neural network model for neural function prediction. By respectively improving the three steps of region of interest labeling, feature extraction and feature screening, and prediction model establishment, the function of predicting the postoperative neural function retention before surgery is realized, which can assist doctors in formulating individualized treatment plans in advance to avoid irreversible neural function damage during the treatment process, thereby better implementing individualized diagnosis and treatment of patients.

[0048] Other features and advantages of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings constituting a part of the present application are used to provide a further understanding of the present application, and the schematic implementation modes and descriptions of the present application are used to explain the present application. In the accompanying drawings:

[0050] Figure 1 A flowchart of a neurological function prediction method based on imaging omics according to a preferred embodiment of the application;

[0051] Figure 2 A schematic diagram of the structure of a first neural network model of a preferred implementation scheme of the application;

[0052] Figure 3 A schematic diagram of the structure of an encoder according to a preferred embodiment of the present invention;

[0053] Figure 4 A training Loss curve of the first neural network model of the preferred embodiment of the application;

[0054] Figure 5 Mean Dice diagram of the first neural network model of the preferred embodiment of the application;

[0055] Figure 6 A distribution histogram of the Dice coefficient of the first neural network model of the preferred embodiment of the application;

[0056] Figure 7 A schematic diagram of segmentation results using the first neural network model according to a preferred embodiment of the application;

[0057] Figure 8 A schematic diagram of the structure of a second neural network model of a preferred embodiment of the application;

[0058] Fig. 9 An iterative curve diagram of the second neural network model in the training process of the preferred embodiment of the application;

[0059] Fig.10 Confusion matrix of the second neural network model and other deep learning models on the test set for the preferred implementation scheme of the application;

[0060] Fig.11 A comparison chart of the ROC curve results of the second neural network model and other models in the preferred implementation mode of the application;

[0061] Fig.12 A schematic diagram of a neurological function prediction device based on imaging omics according to a preferred embodiment of the application;

[0062] Fig.13 A schematic diagram of the structure of an electronic device according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0063] The technical solution of the present application will be described in detail below with reference to the accompanying drawings and in combination with the implementation modes.

[0064] First, this application proposes a neurological function prediction method based on radiomics, such as Figure 1 As shown, the steps include: 110-150:

[0065] Step 110, obtaining magnetic resonance imaging;

[0066] Specifically, Magnetic Resonance Imaging (MRI) is a technique that utilizes the magnetic resonance phenomenon produced when hydrogen atoms in the human body are excited by pulses in a strong magnetic field. Through spatial encoding technology, the electromagnetic waves emitted during the magnetic resonance process and various information parameters related thereto are processed by computer to finally form an image.

[0067] Step 120, constructing a first neural network model using a two-dimensional convolutional architecture, a three-dimensional convolutional architecture, an encoder, and a decoder, and using the first neural network model to segment the magnetic resonance imaging to obtain an image of the region of interest;

[0068] Specifically, in view of the problems that manual segmentation of magnetic resonance imaging has a large workload and segmentation standards vary from doctor to doctor, the first neural network model of the present invention automatically segments each group of magnetic resonance imaging, and uses the segmentation label as the region of interest (ROI) during feature extraction in the following steps, that is, obtaining an image of the region of interest. Among them, in the field of machine vision and image processing, the region of interest can be understood as an image area selected from an image, which is the focus of subsequent image analysis, that is, the focus of feature extraction. Therefore, the present application needs to first segment the image of the region of interest from the magnetic resonance imaging, and then perform feature extraction in the segmented image of the region of interest, so as to reduce processing time and increase processing accuracy.

[0069] The first neural network model in the present application is a self-designed neural network model with deep learning capabilities, which is constructed using a two-dimensional convolutional architecture, a three-dimensional convolutional architecture, an encoder, and a decoder. Among them, the two-dimensional convolutional architecture specifically includes two two-dimensional convolutional layers, and the three-dimensional convolutional architecture includes two three-dimensional convolutional layers. These four convolutional layers can form a 2.5D UNet (U-shaped Network) architecture. 2.5D combines the processing methods of two-dimensional (2D) and three-dimensional (3D) information, and UNet is a "U"-shaped deep learning network model. The encoder in the present application preferably uses the Transformer encoder in the Transformer model. The Transformer model is a deep learning model for processing sequence data, and has an excellent ability to capture global semantic features. Therefore, the present application integrates the Transformer encoder module into the network design to make the first neural network model a new 2.5D Trans-UNet model.

[0070] More specifically, Figure 2 The 2.5D Trans-UNet model structure diagram provided in this application is also the first neural network model architecture diagram. The left side of the figure is the magnetic resonance imaging, and the right side is the output image of the region of interest. Figure 2 As shown, the magnetic resonance imaging is segmented using the first neural network model to obtain an image of the region of interest, including steps 121-124:

[0071] Step 121: using two two-dimensional convolutional layers and two three-dimensional convolutional layers in sequence to perform convolution operations on the magnetic resonance imaging to obtain a feature map;

[0072] The first and second layers of the first neural network model in the present application are convolutional layers, and two-dimensional convolutional layers with residual connections are used. The reason is that the resolution of magnetic resonance imaging in a plane is about four times the resolution between planes. Therefore, two two-dimensional convolutions with a step size of 2 and a convolution kernel size of 3 can be used to downsample and reduce the resolution of the feature map in a plane, so that the resolution of the feature map in a plane and between planes can be consistent, so that the three-dimensional convolutional layer can perform convolution operations and extract features in the plane at the same time. The resolution in a plane, also called lateral resolution or radial resolution, refers to the minimum distance or minimum interval at which an imaging system can clearly distinguish two adjacent objects in an imaging plane; the resolution between planes, also called longitudinal resolution or axial resolution, refers to the ability of an imaging system to clearly distinguish two adjacent objects between imaging planes (i.e., in the depth direction).

[0073] The third and fourth layers of the first neural network model are still convolutional layers, but three-dimensional convolution is used to perform further convolution operations on the feature map output by the two-dimensional convolutional layer, extract inter-plane and intra-plane features, and obtain the final feature map.

[0074] In some specific embodiments, the four convolutional layers all use batch normalization (BatchNorm) to keep the input of each layer of the neural network in the same distribution. Batch normalization is a commonly used technique in deep learning. It normalizes the values ​​of each feature to make the data distribution more stable, making training simpler and faster. In terms of activation function, the parametric rectified linear unit (pReLu) is used as the activation function. At the same time, maximum pooling (Max Pooling) is performed after each convolutional layer to extract important information. In addition, in order to prevent overfitting, the Dropout coefficient of the two lower-dimensional two-dimensional convolutional layers is 0.1, and the two higher-dimensional three-dimensional convolutional layers are added with 0.2. Dropout refers to the random deletion of some nodes in the input layer and the hidden layer with a certain probability in the neural network layer. The Dropout coefficient of 0.1 or 0.2 can be understood as a 10% or 20% probability of each neuron in this layer being randomly discarded during training.

[0075] Step 122: linearly map the feature map to obtain a vector sequence of the feature map;

[0076] The Transformer model architecture can be mainly divided into two parts: encoder and decoder. The encoder is responsible for converting the input sequence into a series of hidden representations, which can contain the semantic and contextual information of the input sequence, and provide the necessary information basis for the subsequent decoder to generate the output sequence. In order to adapt to the one-dimensional input form of the Transformer encoder, this application needs to use a linear mapping method to perform dimensional transformation between the UNet architecture and the Transformer encoder after the fourth convolutional layer, so that the feature map is converted into a vector sequence format read by the transformer encoder.

[0077] Step 123: using the encoder to perform global semantic extraction on the vector sequence to obtain a global semantic encoding;

[0078] The encoder of the first neural network model is a 12-layer Transformer encoder module, which further integrates various features to obtain the global semantic code, and then outputs the global semantic code to the decoder part of the network.

[0079] In some specific embodiments, Figure 3 As shown in the encoder structure diagram, when using the Transformer encoder for global semantic extraction, first, the feature dimension of the magnetic resonance imaging input to the Transformer encoder module is set to formula (1):

[0080] x∈R C×W×H×D (1)

[0081] Among them, feature x is a four-dimensional tensor, R represents a real number set, C represents the number of modes or channels of magnetic resonance imaging, W represents the number of pixels of magnetic resonance imaging in the horizontal direction, H represents the number of pixels of magnetic resonance imaging in the vertical direction, and D represents the number of segmentations of magnetic resonance imaging.

[0082] Then, the vector sequence of the input feature map is divided into N non-overlapping small blocks x p , as shown in formula (2) and (3):

[0083]

[0084] Among them, x p Indicates that the size of the segmented voxel is P 3 , a small block with C modalities, N is the number of segmentations of magnetic resonance imaging, P is the unit length of each dimension of the voxel, and i is a counting unit between 1 and N.

[0085] Then, the small block is sent to the linear mapping layer to obtain the embedded projection information E that retains the content information of the small block. The dimension M of this layer is equal to P 3 Multiply by C.

[0086] In order to retain the location information, we need to add the location embedding information E on the basis of E p , we get the initial input embedded sequence a0 of the Transformer layer, as shown in formula (4):

[0087]

[0088] Among them, x p is the segmented image block, E is the embedded projection of each small block, E p is the position embedding information, N is the number of segmentations of the magnetic resonance imaging, and P is the unit length of each dimension of the voxel.

[0089] At the same time, the Transformer module also contains a multilayer perceptron (MLP). The multilayer perceptron contains two linear layers and uses GELU (Gaussian Error Linear Unit) as the activation function. The multi-headed self-attention (MSA) of the Transformer module consists of 12 self-attention heads to extract information from the input embedded sequence. In these self-attention heads, each element a in the embedded sequence i It contains three different vectors, namely: query vector q, key vector k and value vector v.

[0090] The output of the self-attention head is shown in formula (5):

[0091]

[0092] Among them, d n is the input data dimension, Represents the scaling factor of the feature map, used to prevent vector q and vector k T The inner product of is too large, SA() represents the self-attention head function, softmax() is the normalized exponential function, N is the number of segmentations of magnetic resonance imaging, M is the dimension of this layer, R represents the set of real numbers, T is the transpose operator, q is the query vector, k is the key identification vector, and v is the value vector.

[0093] The expression of multi-head self-attention is shown in formula (6):

[0094] MSA(a)=[SA1(a);SA2(a);…;SA 12 (a);]W msa (6)

[0095] Where MSA(a) represents the multi-head self-attention of the input embedded sequence a, W msa For the weight matrix used for training, the mean and variance of the input of each layer of neurons can be converted to the same value to speed up convergence.

[0096] In summary, the expressions of multi-head self-attention and multi-layer perceptron are shown in equations (7) and (8):

[0097]

[0098] in, Represents the feature map a l The multi-head self-attention output result is: Represents the feature map a l The output result of the multi-layer perceptron, norm() represents the normalization function, a l+1 and a l―1 Respectively represent the element a in the current embedded sequence l The previous and next element of , element a l The dimension is The meanings of N, M, P, H, W, and D are the same as those of the other expressions above.

[0099] Step 124: Using the decoder, perform a deconvolution operation on the global semantic code to obtain an image of the region of interest.

[0100] The decoder of the first neural network model is a mirror image deconvolution layer of the first four convolutional layers, and the parameter settings are consistent with the corresponding convolutional layers. Between each corresponding convolutional layer and deconvolution layer, a skip connection is used to directly connect the input to the output to solve the gradient vanishing problem and accelerate the training process.

[0101] The following is a specific example of a vestibular schwannoma (VS) to demonstrate the effect of the first neural network model in this application, that is, the 2.5D U-Net model.

[0102] The 2.5D U-Net model was trained, validated, and predicted using a private dataset. The total data volume was 98 patients, with two sequences of MRI data for each patient. The distribution of the training set, validation set, and test set was determined to be 7:1:2 based on k-fold cross validation, and the model with the highest Dice coefficient on the validation set was used to predict the test set. k-fold cross validation is a form of cross validation. The Dice coefficient is a set similarity metric function used to calculate the similarity between two sets. The model converged in about 300 epochs, and the training loss curve of the model is shown below. Figure 4 shown.

[0103] The total Dice coefficient and the average Dice coefficient obtained by the model are as follows Figure 5 As shown in the figure, the highest Dice coefficient is 0.9626, the lowest is 0.7909, and the average Dice coefficient is 0.9066±0.0389. From the results, we can see that both the highest Dice coefficient and the average Dice coefficient are higher than the segmentation accuracy of 0.8 required by the task, and the lowest Dice coefficient is also close to 0.8. It can be considered that the model in this paper can complete the VS segmentation task well.

[0104] The distribution histogram of the Dice coefficient of the test set is as follows Figure 6 As shown in the figure, we can see that most of the Dice coefficients fall within the range of 0.86-0.97, and the number of samples with a Dice coefficient near 0.93 is the largest. Although the Dice coefficients of some segmentation results are low, most of them are above 0.86. Therefore, the segmentation accuracy of the model in this paper is relatively good overall.

[0105] The total Hausdorff distance and average Hausdorff distance of the test set prediction results are as low as 1.41 and as high as 12.84, with an average of 4.67±3.19. As can be seen from the results, the average Hausdorff distance is low, indicating that the boundary between the true value and the predicted result is relatively consistent. When predicting the test set, the total sensitivity and average sensitivity of the results are close to 1, indicating that the proportion of correctly predicted areas to the true value is very high, and the results have good reliability.

[0106] The segmentation results are shown in the following example: Figure 7 From left to right are the original MRI image, the segmentation truth label, and the predicted output of the model of the present invention, and the corresponding Dice coefficient and Hausdorff distance are shown above the output graph.

[0107] By comparing the segmentation results obtained by the model of the present invention with the original MRI and the true value labels, it can be seen that the position and size of VS in the brain are not fixed and the shape is irregular, and the segmentation task is more difficult. However, the results show that the tumor area in the segmentation label of the model in this study is basically consistent with the true value label, and has a better segmentation result.

[0108] In addition, the segmentation results of the method of the present invention are compared with those of other methods. Table 1 compares the segmentation results of several classic methods on the dataset of this study. It can be seen that the model proposed in the present invention performs well on the self-collected dataset, and achieves the best results in both Hausdorff distance and Dice score. It has a significant improvement over the traditional 3D U-Net and 2.5DU-Net, especially in the accuracy of contour segmentation.

[0109]

[0110] Table 1 Comparison of results of other methods on self-collected datasets

[0111] At the same time, in order to further verify the effectiveness of the model proposed in this study, this application further verifies the model using the dataset published by Sharpey et al. on the Cancer Imaging Archive (TCIA). Table 2 compares the segmentation results of this method with several other different methods on the TCIA public dataset. It can be seen that the segmentation model proposed in this study obtained the best segmentation results in Hausdorff distance and Dice score, surpassing the SOTA (State-of-the-Art) method. In terms of Hausdorff distance, the method in this paper obtained results similar to the SOTA method.

[0112]

[0113]

[0114] Table 2 Comparison of results on TCIA datasets from other literature

[0115] In summary, the method proposed in the present invention effectively combines the Transformer and 2.5D U-Net network structures to achieve competitive tumor segmentation performance.

[0116] Step 130, extracting features from the image of the region of interest to obtain feature data of the magnetic resonance imaging;

[0117] Specifically, Pyradiomics is an open source python software package that can extract radiomics features from medical images. The Pyradiomics tool is used to extract features from the region of interest obtained by 2.5D Trans-UNet automatic segmentation to obtain the feature data of magnetic resonance imaging.

[0118] In a specific embodiment, the extracted imaging features of magnetic resonance imaging include one or more of the following features: First-Order, Gray Level Co-occurrence Matrix (GLCM), Gray Level Dependence Matrix (GLDM), Gray Level Run Length Matrix (GLRLM), Gray Level Size Zone Matrix (GLSZM), Neighboring Gray Tone Difference Matrix (NGTDM), three-dimensional shape, two-dimensional shape and wavelet transform (Wavelet).

[0119] Step 140, combining the filtering method and the embedding method to perform feature screening on the feature data;

[0120] Specifically, the radiomics features extracted in the above steps may have a high number of feature types and features, which may easily lead to the existence of a large number of redundant features and noise features, thereby affecting the performance of subsequent prediction models and increasing computational costs. Therefore, screening radiomics features can help improve the performance of subsequent prediction models, reduce the risk of overfitting, and reduce the computational costs of model training and inference. At the same time, screening out key radiomics features can help enhance the accuracy and interpretability of prediction results.

[0121] In existing feature screening methods, the interaction between features is usually ignored, or the training complexity is increased sharply in order to take care of feature correlation. This application combines the filtering method and the embedding method, combining the advantages of high computational efficiency, good adaptation to learning algorithms, and easy avoidance of overfitting, which can well meet the research objectives of this study on small sample size deep learning algorithms.

[0122] In some specific examples, the filtering method includes: one or more of the Spearman coefficient test, the independent sample t test and the Levene test. The embedding method includes: using the least absolute shrinkage and selection operator (LASSO) regression verification with the L1 norm. The feature data set is divided into two groups of samples according to the patient's postoperative neurological function injury level: mild and moderately severe. The mild sample group corresponds to a label of 0, and the moderately severe sample group corresponds to a label of 1. Each patient contains a large number of original imaging genomics features. In a more preferred embodiment, the feature screening of the original imaging genomics features in combination with the filtering method and the embedding method includes the above four verification methods, that is, the feature data is sequentially subjected to the Levene test, the independent sample t test, the Spearman coefficient test and the Lasso regression verification. The above three verification methods in the filtering method respectively remove abnormal samples, samples with no significance and samples with low stability, and preliminarily screen the extracted imaging genomics features in a relatively comprehensive manner. Afterwards, the features that have an important impact on the subsequent classification are selected using the Lasso regression verification to suppress model overfitting and improve model performance. Among them:

[0123] Levene's test is a statistical method used to test whether the variances between multiple groups are equal. It is usually used for variance homogeneity test, that is, to test whether the variances of different groups are equal. The null hypothesis of Levene's test is that the variances between the groups are equal, and the alternative hypothesis is that the variances of at least one group are unequal. The basic idea is to test the homogeneity of variance by comparing the difference between the observed values ​​in each group and the mean within the group. If the variances of the groups differ slightly, these differences should be small, while if the variances of the groups differ significantly, these differences may be large.

[0124] The Levene test includes: calculating the W value of the Levene test of the feature data, and the W value is a test statistic that directly measures the difference in variance between different groups. By looking up the F distribution table of the W value of the feature data, the p value corresponding to the W value of the feature data can be obtained. If the p value corresponding to the W value of the feature data is not greater than the first preset value, the current feature data is excluded. Here, the p value corresponding to the W value represents the probability of obtaining the current W value or a more extreme value than it under the assumption that variance homogeneity holds, which is used to determine whether the assumption of variance homogeneity holds. The first preset value is preferably 0.05.

[0125] Specifically, after the two groups of original imaging feature samples are subjected to the Levene test, the features with significant variance differences can be screened out, and the features with uneven variances caused by abnormal samples can be removed. For features that pass the Levene test, that is, the variances of the features in the two groups of samples are equal, the independent sample t test can be used for further feature screening.

[0126] Independent Samples t-test is a statistical test method used to compare whether the means of two independent samples are significantly different. It is used to determine whether there is a significant difference in the mean between two independent groups. The basic assumption of the independent sample t-test is that the two samples are independent samples from the same population and the distribution of the two populations is normally distributed. The null hypothesis of the test is that the means of the two populations are equal, and the alternative hypothesis is that the means of the two populations are not equal.

[0127] The independent sample t test includes: calculating the t value of the independent sample t test of the feature data, comparing the t value of the feature data with the theoretical t value distribution, obtaining the p value corresponding to the t value of the feature data, and excluding the current feature data if the p value corresponding to the t value of the feature data is less than a second preset value. Here, the p value corresponding to the t value represents the probability of the tail area corresponding to the t value in the t distribution. The second preset value is preferably 0.05.

[0128] Specifically, an independent sample t test is performed on each feature in the two groups of feature samples after the Levene test. Each time, the feature value corresponding to the same feature in the two groups of samples is taken to calculate the t value, and the p value of the feature is obtained. If the p value is less than 0.05, it indicates that the mean difference between the two groups is significant, and the feature is retained; otherwise, the feature is removed.

[0129] Spearman's rank correlation coefficient is a statistic used to measure the correlation between two variables. Spearman's coefficient is a non-parametric method that is applicable to cases of nonlinear relationships. Spearman's coefficient is often used to measure the rank relationship between variables, especially when the data does not meet the normal distribution assumption or there are extreme values. If two features are highly correlated, one of them can be considered redundant and thus removed. Such redundant features are often unstable, especially when the data dimension is high. Therefore, using the Spearman coefficient can screen out features with high redundancy and low stability.

[0130] The Spearman coefficient test includes: calculating the Spearman coefficient ρ of the feature data, and if the Spearman coefficient ρ of the feature is not greater than a third preset value, excluding the current feature data. The third preset value is preferably 0.8.

[0131] Specifically, the two groups of feature samples after the Levenne test and the independent sample t test are combined for the Spearman coefficient test, where the target label from the mild sample group is 0, and the target label from the moderate and severe sample group is 1. The Spearman correlation coefficient ρ between each feature and the target label is calculated. If the Spearman correlation coefficient ρ is not greater than 0.8, it indicates that the correlation between the feature and the target label is low, and the feature is excluded; otherwise, the feature is retained.

[0132] Lasso regression check (Least absolute shrinkage and selection operator) is a linear regression method that automatically selects those features that contribute less to the model and excludes them by adding L1 regularization terms to the loss function, making the coefficient 0, thereby achieving the purpose of feature selection and dimensionality reduction, and better solving the problem of multicollinearity in regression analysis.

[0133] Specifically, the Lasso regression test is performed on all feature sets containing two groups of feature samples after feature screening by the above three filtering methods, where the target label from the mild sample group is 0, and the target label from the moderate and severe sample group is 1. The Lasso regression model is trained using feature data and corresponding target labels, and the regularization coefficient of each feature is automatically determined by cross-validation during the training process. After the model is trained, the regularization coefficient of each feature is obtained. If the coefficient of a feature is 0, it means that the feature is considered unimportant and can be excluded. The features with coefficients not 0 are retained as the important imaging omics features finally screened.

[0134] Step 150, constructing a second neural network model using a one-dimensional convolutional architecture, a gated recurrent module, and a fully connected architecture, and using the second neural network model to perform neural function prediction based on the screened feature data;

[0135] Specifically, the one-dimensional convolutional architecture can be used as the backbone of the series hybrid neural network, and the gated recurrent unit (Gated Recurrent Unit, GRU) is an improved recurrent neural network that controls the flow of information through the update gate and the reset gate. The update gate determines the mixing ratio of the current hidden state and the previous state, and the reset gate controls how much past information is discarded. By fusing past and current state information, the long-term dependency problem in the sequence can be effectively dealt with, the structure of the long short-term memory network (Long Short-Term Memory, LSTM) is simplified, and it has the characteristics of easy training and easy implementation. Therefore, this application uses a one-dimensional convolutional architecture as the backbone network, introduces a gated recurrent module, and constructs a prognosis prediction model of the 1DCNN-GRU model, that is, a second neural network model, which is used to classify the previously extracted and screened imaging genomics features and predict according to the classification results.

[0136] More specifically, Figure 8 This is the 1DCNN-GRU model structure diagram of this application, that is, the second neural network model architecture diagram, combined with Figure 8 As shown, the second neural network model is used to predict neural function according to the filtered feature data, including steps 151-153:

[0137] Step 151, using three one-dimensional convolutional layers to extract spatial features of the filtered feature data;

[0138] Spatial features can be understood as the spatial entity location, topological relationship and geometric features of feature data.

[0139] In some specific embodiments, maximum pooling is also performed after each one-dimensional convolutional layer to extract important information.

[0140] Step 152, using a gated recurrent module to learn contextual information of spatial features to obtain sequence features of feature data;

[0141] The gated recurrent module is used to capture the sequence features of the input data, learn the contextual information of the spatial features, and obtain the sequence features of the feature data.

[0142] Step 153, using three fully connected layers, the neuron data is mapped to the probability interval to obtain the classification probability value, and classification is performed according to the classification probability value to predict the neural function according to the classification result.

[0143] The fully connected layer connects all nodes in the previous layers to each node in the latter layer, realizing the global mapping of features. This fully connected approach allows the network to learn complex features in the data and establish a nonlinear relationship between input and output. As the last layer of this application, the role of the fully connected layer is to integrate the features extracted by the previous convolutional layer or pooling layer and generate the final output result.

[0144] In some specific embodiments, the convolution kernel size of the three one-dimensional convolution layers in the one-dimensional convolution architecture is 3, the number of filters in the first convolution layer is 32, the number of filters in the second convolution layer is 64, and the number of filters in the third convolution layer is 128. A maximum pooling layer (MaxPooling1D) with a dimension of 1 is added after each convolution layer in the network, followed by a batch normalization (BN) operation.

[0145] The gated recurrent module is a 128-unit GRU layer used to learn sequence context information.

[0146] The fully connected architecture consists of three fully connected layers (Dense), and finally uses the normalized exponential function (Softmax) as the activation function of the classification network. A Dropout with a coefficient of 0.3 is also added to the three fully connected layers to discard the output of neurons with a probability of 30% to reduce the risk of network overfitting.

[0147] In terms of the output of the results, a normalized exponential function (Softmax) is used to perform binary classification on the model to convert the output of the neural network into a vector representing the probabilities of different categories.

[0148] At this point, the second neural network model can obtain a probability for prediction, classify the prognosis of the patient's nerve damage according to the probability value, and predict the nerve function accordingly.

[0149] The neural function prediction method provided in the present application uses a two-dimensional convolutional architecture, a three-dimensional convolutional architecture, an encoder and a decoder to construct a first neural network model to segment the region of interest in magnetic resonance imaging, and then extracts and screens features of the image of the region of interest. Finally, a one-dimensional convolutional architecture, a gated loop module and a fully connected architecture are used to construct a second neural network model for neural function prediction. By respectively improving the three steps of region of interest labeling, feature extraction and feature screening, and prediction model establishment, the function of predicting the postoperative neural function retention before surgery is realized, which can assist doctors in formulating individualized treatment plans in advance to avoid irreversible neural function damage during the treatment process, thereby better implementing individualized diagnosis and treatment of patients.

[0150] Below, in a specific embodiment of a vestibular schwannoma (VS), the effect of the second neural network model in the present application is demonstrated.

[0151] This application conducts experiments on a self-collected dataset to verify the effectiveness of the proposed imaging feature screening method combining filtering method and embedding method and the 1DCNN-GRU hybrid network model.

[0152] Fig. 9 Iteration curve of the 1DCNN-GRU network model proposed in the present invention during the training process. The two curves represent the training loss curve and the accuracy curve of the training set respectively.

[0153] Fig.10 This is the confusion matrix of the 1DCNN-GRU network model proposed in the present invention and other deep learning models on the test set. The input features of the prediction model come from the automatic segmentation results obtained by 2.5D Trans-UNet and the tumor area segmented manually. Fig.10(A) and (B) are the confusion matrices of the 1DCNN-LSTM model on the test set. Fig.10 (C) and 10(D) are the confusion matrices of the 1DCNN model on the test set; Fig.10 (E), 10(F) Confusion matrix of the 1DCNN-GRU model proposed in this study on the test set; Fig.10 (A), 10(C), and 10(E) are the confusion matrices predicted using the features of the automatic segmentation results obtained using 2.5D Trans-UNet; Fig.10 (B), 10(D), and 10(F) are the confusion matrices of prediction using the features of the manual segmentation results.

[0154] In order to further evaluate the performance of the proposed 1DCNN-GRU network model, this paper conducted multiple comparative experiments on self-collected data sets. The comparative models used are 1DCNN, 1DCNN-LSTM, and several representative machine learning algorithms, including support vector machine (SVM), logistic regression, bootstrap aggregating (Bagging), and decision tree. The comparison results are shown in Figure 2. Fig.11 As shown. Among them, Fig.11 (A) is the ROC (Receiver Operating Characteristic curve) curve for prognosis prediction using automatic segmentation results under different networks. Fig.11 (B) ROC curve of prognosis prediction using manual segmentation results under different networks.

[0155] From the above figure, we can see that in the two sets of experiments, the prediction effect of the 1DCNN-GRU hybrid network (black line) proposed in this study is better than that of the other four traditional neural networks, and it can reach the optimal value of the model faster.

[0156]

[0157]

[0158] Table 3 Comparison of prognosis prediction results using automatic segmentation results under different networks

[0159]

[0160] Table 4 Comparison of prognosis prediction results using manual segmentation results under different networks

[0161] Tables 3 and 4 show the evaluation indicators of the results of using 2.5D Trans-UNet to automatically segment tumors and manually segment tumor areas on the self-collected datasets, followed by feature extraction and screening, and then input into each deep neural network model for the prognosis prediction task.

[0162] From Tables 3 and 4, it can be seen that the model proposed in this study has almost achieved the best classification effect in terms of accuracy, ROC curve and recall rate. In addition, except for SVM, the evaluation indicators of other machine learning and deep learning models under the automatic segmentation group are better than those of the manual segmentation group, proving that the features extracted from the automatic segmentation results have advantages in improving model performance.

[0163] In summary, the method proposed in the present invention achieved the best results in multiple indicators, surpassing machine learning models such as SVM and two deep learning models, 1DCNN and 1DCNN-LSTM, and achieved the best performance, proving that the 2.5D Trans-UNet automatic segmentation model designed in the present invention, the imaging genomics feature screening method combining filtering method and embedding method, and the 1DCNN-GRU classification model have strong performance in the VS postoperative prognosis prediction task.

[0164] In addition, the present application also provides a neurological function prediction device based on imaging omics, which is used to implement the above-mentioned neurological function prediction method based on imaging omics, such as Fig.12 As shown, the device comprises:

[0165] An acquisition module 210 configured to acquire magnetic resonance imaging;

[0166] The first network module 220 is configured to construct a first neural network model using a two-dimensional convolutional architecture, a three-dimensional convolutional architecture, an encoder and a decoder, and segment the magnetic resonance imaging using the first neural network model to obtain an image of a region of interest;

[0167] The feature extraction module 230 is configured to extract features from the image of the region of interest to obtain feature data of the magnetic resonance imaging;

[0168] The feature screening module 240 is configured to perform feature screening on the feature data by combining a filtering method and an embedding method; wherein the filtering method includes: one or more of a Spearman coefficient test, an independent sample t test, and a Levene test; and the embedding method includes: a Lasso regression test;

[0169] The second network module 250 is configured to construct a second neural network model using a one-dimensional convolutional architecture, a gated loop module and a fully connected architecture, and use the second neural network model to predict neural function based on the screened feature data.

[0170] The effect of a neurological function prediction device based on imaging omics disclosed in the present application is the same as the above-mentioned neurological function prediction method based on imaging omics, which will not be repeated here.

[0171] Below, reference Fig.13 The electronic device 11 according to an embodiment of the present application is described. Fig.13 A block diagram of an electronic device according to an embodiment of the present application is shown.

[0172] like Fig.13 As shown, the electronic device 11 includes one or more processors 111 and a memory 112 .

[0173] The processor 111 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 11 to perform desired functions.

[0174] The memory 112 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 111 may run the program instructions to implement the test methods of the various embodiments of the present application described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

[0175] In one example, the electronic device 11 may further include: an input device 113 and an output device 114 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0176] The input device 113 may include, for example, a keyboard, a mouse, etc.

[0177] The output device 114 can output various information to the outside, including determined distance information, direction information, etc. The output device 114 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0178] Of course, to simplify, Fig.13Only some of the components related to the present application in the electronic device 11 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 11 may also include any other appropriate components.

[0179] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the testing method according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

[0180] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0181] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the testing method according to various embodiments of the present application described in the above “Exemplary Method” section of this specification.

[0182] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0183] The preferred embodiments of the present application are described in detail above; however, the present application is not limited to the specific details in the above embodiments. Within the technical concept of the present application, a variety of simple modifications can be made to the technical solution of the present application, and these simple modifications all fall within the protection scope of the present application.

[0184] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this application will not further describe various possible combinations.

[0185] In addition, the various implementation modes of the present application may be arbitrarily combined, and as long as they do not violate the concept of the present application, they should also be regarded as the contents disclosed by the present invention.

Claims

1. A neurological function prediction method based on radiomics, characterized in that: The method comprises: Obtain magnetic resonance imaging; Using a two-dimensional convolutional architecture, a three-dimensional convolutional architecture, an encoder and a decoder to construct a first neural network model, and using the first neural network model to segment the magnetic resonance imaging to obtain an image of a region of interest; Extracting features from the image of the region of interest to obtain feature data of magnetic resonance imaging; The feature data is subjected to feature screening by combining filtering method and embedding method; wherein the filtering method includes: one or more of Spearman coefficient test, independent sample t test and Levene test; the embedding method includes: Lasso regression test; A second neural network model is constructed using a one-dimensional convolutional architecture, a gated recurrent module, and a fully connected architecture, and the second neural network model is used to predict neural function based on the screened feature data.

2. The method according to claim 1, characterized in that Segmenting the magnetic resonance imaging using the first neural network model to obtain an image of a region of interest includes: sequentially using two two-dimensional convolutional layers and two three-dimensional convolutional layers to perform convolution operations on the magnetic resonance imaging to obtain a feature map; Performing linear mapping on the feature map to obtain a vector sequence of the feature map; Using the encoder, performing global semantic extraction on the vector sequence to obtain a global semantic encoding; The decoder is used to perform a deconvolution operation on the global semantic code to obtain an image of the region of interest.

3. The method according to claim 2, characterized in that: The two two-dimensional convolutional layers are two two-dimensional convolutional layers with residual connections, so that the in-plane resolution and inter-plane resolution of the feature map are consistent.

4. The method according to claim 2, characterized in that: The encoder is a Transformer encoder; The decoder is a mirrored deconvolution layer of the two two-dimensional convolutional layers and the two three-dimensional convolutional layers.

5. The method according to claim 1, characterized in that The Levene test includes: Calculate the W value of the Levene test of the characteristic data; Searching the F distribution table of the W value of the characteristic data to obtain the p value corresponding to the W value of the characteristic data; If the p value corresponding to the W value of the characteristic data is not greater than a first preset value, the characteristic data is excluded.

6. The method according to claim 1, characterized in that The independent sample t test includes: Calculate the t value of the independent sample t test of the characteristic data; Compare the t value of the characteristic data with the theoretical t value distribution to obtain the p value corresponding to the t value of the characteristic data; If the p value corresponding to the t value of the characteristic data is smaller than a second preset value, the characteristic data is excluded.

7. The method according to claim 1, characterized in that The Spearman coefficient test includes: Calculate the Spearman coefficient ρ of the characteristic data; If the Spearman coefficient ρ of the feature data is not greater than a third preset value, the feature data is excluded.

8. The method according to claim 1, characterized in that The combining filtering method and embedding method to perform feature screening on the feature graph includes: The characteristic data are sequentially subjected to Levene test, independent sample t test, Spearman coefficient test and Lasso regression test.

9. The method according to claim 1, characterized in that: Using the second neural network model to predict neural function according to the filtered feature data includes: Using three one-dimensional convolutional layers, extracting spatial features of the filtered feature data; Using the gated recurrent module, learning the context information of the spatial features to obtain sequence features of feature data; Using three fully connected layers, the sequence features are mapped to probability intervals to obtain classification probability values; A neural function prediction is performed according to the classification probability value.

10. A neurological function prediction device based on radiomics, characterized in that: The device comprises: an acquisition module configured to acquire a magnetic resonance image; A first network module is configured to construct a first neural network model using a two-dimensional convolutional architecture, a three-dimensional convolutional architecture, an encoder, and a decoder, and segment the magnetic resonance imaging using the first neural network model to obtain an image of a region of interest; A feature extraction module is configured to extract features from the image of the region of interest to obtain feature data of magnetic resonance imaging; A feature screening module is configured to perform feature screening on the feature data by combining a filtering method and an embedding method; wherein the filtering method includes: one or more of a Spearman coefficient test, an independent sample t test, and a Levene test; and the embedding method includes: a Lasso regression test; The second network module is configured to construct a second neural network model using a one-dimensional convolutional architecture, a gated recurrent module and a fully connected architecture, and use the second neural network model to predict neural function based on the screened feature data.

11. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1 to 9.

12. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the method according to any one of claims 1 to 9.

Images