A multi-modal magnetic resonance imaging data analysis system
By constructing a multimodal magnetic resonance imaging (MRI) data analysis system and combining ensemble learning and deep learning, the problem of poor model performance in multimodal MRI data analysis was solved, achieving more efficient disease prediction and feature interpretation, and improving prediction performance and interpretability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SOUTH CHINA UNIV OF TECH
- Filing Date
- 2024-03-14
- Publication Date
- 2026-07-21
Smart Images
Figure CN118115807B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical field of machine learning and data analysis, and in particular to a multimodal magnetic resonance imaging data analysis system. Background Technology
[0002] Magnetic resonance imaging (MRI), such as functional magnetic resonance imaging (fMRI), structural magnetic resonance imaging (SMRI), diffusion tensor imaging (DTI), and arterial spin labeling perfusion MRI, is a powerful tool for auxiliary diagnosis of diseases. For example, there are many types of brain diseases, such as cerebrovascular diseases, neurodegenerative diseases, epilepsy, and brain tumors, and data from a single modality of MRI is often used to identify diseases or disease progressions. However, different MRI modalities contain heterogeneous and complementary information, and researchers are working to develop an intelligent multimodal MRI analysis tool for clinical auxiliary diagnosis.
[0003] In recent years, for complex diseases such as brain diseases, deep learning models have often been used to predict diseases or disease courses in multimodal magnetic resonance imaging (MRI) data analysis, taking into account the complexity of multimodal MRI data. However, multimodal MRI data analysis based on intelligent methods such as deep learning still faces many challenges, such as poor model performance, opaque model decision-making, and difficulty in understanding. Summary of the Invention
[0004] The purpose of this invention is to overcome the shortcomings and deficiencies of the prior art and propose a multimodal magnetic resonance imaging data analysis system, which can effectively solve the limitation of single-modal magnetic resonance imaging information source, fully capture the heterogeneous and complementary characteristics between different modal magnetic resonance imaging, thereby improving the predictive performance and interpretability of the model.
[0005] To achieve the above objectives, the technical solution provided by this invention is: a multimodal magnetic resonance imaging data analysis system, comprising:
[0006] The data import module is used to acquire multimodal magnetic resonance imaging data and preprocess the data to obtain a set of feature matrices for the multimodal magnetic resonance imaging data.
[0007] The intelligent recognition training module uses the feature matrix set of multimodal magnetic resonance imaging data obtained by the data import module to train machine learning models and deep learning models based on optimization algorithms. Then, it uses ensemble learning to fuse the trained machine learning models and deep learning models to construct the required ensemble learning classification model.
[0008] The intelligent identification and prediction module predicts diseases based on the integrated learning classification model constructed by the intelligent identification training module for the multimodal magnetic resonance imaging data to be tested; wherein, the multimodal magnetic resonance imaging data to be tested is collected and preprocessed by the data import module;
[0009] The prediction result interpretation module is used to interpret the feature importance and perform feature interaction testing based on the prediction results of the intelligent recognition prediction module.
[0010] Furthermore, the data import module includes a data acquisition module and a data preprocessing module, wherein:
[0011] The data acquisition module reads multimodal magnetic resonance imaging data and clinical data from the local machine, including nii, gz, dcm, CSV and xlsx formats. The multimodal magnetic resonance imaging data includes T1-weighted magnetic resonance imaging and arterial spin label perfusion magnetic resonance imaging, and the clinical data includes sample age, gender and years of education.
[0012] The data preprocessing module is used to remove samples with missing modalities, divide the data into several groups according to category labels, remove samples that fail the t-test for age and years of education, remove the mean of samples that fail the chi-square test for gender, and for magnetic resonance imaging data, use the SPM, CAT12, and ExploreASL toolkits to extract the feature matrix set {A1, A2, A3, ..., A...} for each modality of magnetic resonance imaging data. n}, where A1 is the feature matrix of the first modal magnetic resonance imaging data, A n Let A be the feature matrix of the nth modal magnetic resonance imaging data, where n is the number of modalities of the magnetic resonance imaging data, and A = nRois × w is the size of the feature matrix, where nRois is the number of features, and w is the number of samples.
[0013] Furthermore, the intelligent recognition training module includes a sub-model training module and a sub-model fusion module, wherein:
[0014] The sub-model training module is used to perform feature fusion on samples and construct three machine learning models and one deep learning model. The three machine learning models are a linear kernel function support vector machine model, a radial basis function kernel function support vector machine model, and a random forest model, respectively. The deep learning model is a multilayer perceptron model, and specifically includes the following steps:
[0015] 1) Set the feature matrices of each modal magnetic resonance imaging data sample {A1, A2, A3, ..., A...} n The vectors {F1, F2, F3, ..., F} are concatenated and converted into the corresponding vector form {F1, F2, F3, ..., F}. m}, where F represents a feature vector, m = nRois × n represents the total number of feature vectors, the transformed feature vectors are divided into training set, validation set and test set, and data normalization is performed;
[0016] 2) Construct a set of sub-models {M1, M2, M3, M4}. Sub-model M1 is a linear kernel function support vector machine model, sub-model M2 is a radial basis function kernel function support vector machine model, sub-model M3 is a random forest model, and sub-model M4 is a multilayer perceptron model. The multilayer perceptron model M4 contains one input layer, two FC-BN-LeakyReLU modules, and one output layer. A dropout layer is added after the two FC-BN-LeakyReLU modules to avoid overfitting. The sigmoid function is applied to the output layer to predict the probability that any input instance is a disease patient. The FC-BN-LeakyReLU module consists of a fully connected neural network layer FC, a batch normalization layer BN, and a LeakyReLU activation function layer.
[0017] 3) Train the sub-models using the training set. For traditional machine learning models, use grid search to select the optimal hyperparameters of sub-models M1, M2 and M3 and use the validation set to validate them. Finally, the optimal sub-models M1, M2 and M3 are obtained. For deep learning models, the Adam optimization algorithm is used to optimize sub-model M4. The loss function is the binary cross-entropy loss function. Finally, the optimal sub-model M4 is obtained.
[0018] The sub-model fusion module uses ensemble learning to extract predictions from the four sub-models and uses the average of the predictions as the final prediction. Let the ensemble learning classification model be y = f(M1, M2, M3, M4), where y is the class predicted by the model and f(·) is the classification model. Then the final prediction result is:
[0019]
[0020] In the formula, p uv For u th The prediction result of the v-th sub-model of class label, u th Let y represent the u-th group of samples grouped by category label. * This is the predicted result.
[0021] Furthermore, the intelligent recognition and prediction module includes the following steps:
[0022] 1) Use the data acquisition module to load the multimodal magnetic resonance imaging data, and use the data preprocessing module to process the multimodal magnetic resonance imaging data;
[0023] 2) The multimodal magnetic resonance imaging data processed in step 1) is input into the integrated learning classification model constructed by the intelligent recognition training module for disease prediction.
[0024] Furthermore, the prediction result interpretation module includes a feature importance interpretation module and a feature interaction interpretation module, wherein:
[0025] The feature importance interpretation module is used to calculate the contribution value of each feature to the model prediction, i.e., feature importance, based on the performance error method, including the following steps:
[0026] 1) Select samples from the test set where the model correctly predicts the class y′, where y′ is one of multiple class labels, and replace the given feature F. i Calculate the permutation given feature F i The error in prediction performance before and after classification accuracy is considered as the importance of the feature, and is calculated using the following formula:
[0027] E i =f(F1,F2,F3,...,F i ,...,F m )-f(F1,F2,F3,...,perm(F i ),...,F m )
[0028] In the formula, F i E represents the i-th feature among all features. i Representing feature F i The importance of perm(F) i ) represents the permutation feature F i E i The larger the value, the stronger the feature F. i The greater the importance of E in prediction tasks, i The negative value of F indicates that the feature F i It may hinder or affect accurate prediction;
[0029] 2) Sort the importance of all features obtained in step 1), and take the top K features in terms of final importance as the important features for the model to predict category y′;
[0030] 3) The selected important features were validated. The test set obtained by masking the top 1, top 5, top 10, ..., top K important features respectively was input into the ensemble learning classification model and the test results were compared with the test results of the original test set. This fully validated the effectiveness of the selected top K features in terms of importance.
[0031] The feature interaction interpretation module is used to test whether the interaction effect between various important features and the interaction effect between important features and features of user interest are significant, including the following steps:
[0032] 1) Calculate the important feature F that is simultaneously permuted i And important features or features of interest to users F j Prediction performance error E i,j :
[0033] E i,j =f(F1,F2,F3,...,F i ,F j ...,F m )-f(F1,F2,F3,...,perm(F i ,F j ),...,F m )
[0034] In the formula, F j Let perm(F) represent the j-th feature among all features. i ,F j ) represents the permutation feature F i and feature F j ;
[0035] 2) Feature interaction strength is measured by the error in prediction performance; therefore, feature F is calculated. i and feature F j Interaction intensity I ij :
[0036] I ij =E i +E j -E i,j
[0037] In the formula, E j It refers to feature F j Performance error, I ij The larger the value, the more characteristic F is. i and feature F j The greater the interaction strength between them, the stronger I ij A value of 0 indicates feature F i and feature F j There is no interaction between them; a positive value indicates feature F. i and feature F j There is a positive interaction between them; a negative value indicates a feature F. i and feature F j There is negative interaction between them;
[0038] 3) Calculate the pairwise interaction strength between the K important features obtained by the feature importance interpretation module, and at the same time calculate the pairwise interaction strength between the important features and the user's interest features, sort them, and take the feature pairs with the highest final interaction strength as the important interaction feature pairs with the greatest impact on the model prediction.
[0039] Calculate the feature interaction strength between Q×2 features and other modal features of interest to the user, rank them, and select the top T feature pairs with the largest feature interaction strength as the feature pairs with the greatest impact on model prediction from different modal feature groups.
[0040] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0041] 1. By using deep learning methods to uncover the complementary characteristics of each modality, the predictive performance of the model is improved.
[0042] 2. By combining ensemble learning with deep learning, an ensemble learning classification model is constructed, which improves the model's expressive and generalization abilities and yields more robust and accurate prediction results.
[0043] 3. Explaining the importance and interaction effects of key or interesting features helps researchers gain a deeper understanding of the disease's pathogenesis.
[0044] 4. Combining multimodal magnetic resonance imaging data can help to further explore disease biomarkers. Attached Figure Description
[0045] Figure 1 This is a schematic diagram showing the relationship between the various modules of the system of the present invention.
[0046] Figure 2 This is a flowchart of the system training and prediction process of the present invention.
[0047] Figure 3 This is a schematic diagram of the structure of the multilayer perceptron model used in the system of the present invention. Detailed Implementation
[0048] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.
[0049] This embodiment provides a multimodal magnetic resonance imaging (MRI) data analysis system. It is a multimodal MRI data analysis system developed using the Python language and capable of running on Windows devices. The relationships between the system modules are as follows: Figure 1 As shown, the system training and prediction process is as follows: Figure 2 As shown. It includes:
[0050] The data import module is used to acquire multimodal magnetic resonance imaging data and preprocess the data to obtain a set of feature matrices for the multimodal magnetic resonance imaging data.
[0051] The intelligent recognition training module uses the feature matrix set of multimodal magnetic resonance imaging data obtained by the data import module to train machine learning models and deep learning models based on optimization algorithms. Then, it uses ensemble learning to fuse the trained machine learning models and deep learning models to construct the required ensemble learning classification model.
[0052] The intelligent identification and prediction module predicts diseases based on the integrated learning classification model constructed by the intelligent identification training module for the multimodal magnetic resonance imaging data to be tested; wherein, the multimodal magnetic resonance imaging data to be tested is collected and preprocessed by the data import module;
[0053] The prediction result interpretation module is used to interpret the feature importance and perform feature interaction testing based on the prediction results of the intelligent recognition prediction module.
[0054] Specifically, the data import module includes a data acquisition module and a data preprocessing module, wherein:
[0055] The data acquisition module reads multimodal magnetic resonance imaging data and clinical data from the local machine, including nii, gz, dcm, CSV and xlsx formats. The multimodal magnetic resonance imaging data includes T1-weighted magnetic resonance imaging and arterial spin label perfusion magnetic resonance imaging, and the clinical data includes sample age, gender and years of education.
[0056] The data preprocessing module is used to remove samples with missing modalities, divide the data into several groups according to category labels, remove samples that fail the t-test for age and years of education, remove the mean of samples that fail the chi-square test for gender, and for magnetic resonance imaging data, use the SPM, CAT12, and ExploreASL toolkits to extract the feature matrix set {A1, A2, A3, ..., A...} for each modality of magnetic resonance imaging data. n}, where A1 is the feature matrix of the first modal magnetic resonance imaging data, A n Let A be the feature matrix of the nth modal magnetic resonance imaging data, where n is the number of modalities of the magnetic resonance imaging data, and A = nRois × w is the size of the feature matrix, where nRois is the number of features, and w is the number of samples.
[0057] Specifically, the intelligent recognition training module includes a sub-model training module and a sub-model fusion module, wherein:
[0058] The sub-model training module is used to perform feature fusion on samples and construct three machine learning models and one deep learning model. The three machine learning models are a linear kernel function support vector machine model, a radial basis function kernel function support vector machine model, and a random forest model, respectively. The deep learning model is a multilayer perceptron model, and specifically includes the following steps:
[0059] 1) Set the feature matrices of each modal magnetic resonance imaging data sample {A1, A2, A3, ..., A...} n The vectors {F1, F2, F3, ..., F} are concatenated and converted into the corresponding vector form {F1, F2, F3, ..., F}. m}, where F represents a feature vector, m = nRois × n represents the total number of feature vectors, the transformed feature vectors are divided into training set, validation set and test set, and data normalization is performed;
[0060] 2) Construct a set of sub-models {M1, M2, M3, M4}, where sub-model M1 is a linear kernel function support vector machine model, sub-model M2 is a radial basis function kernel function support vector machine model, sub-model M3 is a random forest model, and sub-model M4 is a multilayer perceptron model; for example... Figure 3 As shown, the multilayer perceptron model M4 consists of one input layer, two FC-BN-LeakyReLU modules, and one output layer. A dropout layer is added after the two FC-BN-LeakyReLU modules to avoid overfitting. The sigmoid function is applied to the output layer to predict the probability that any input instance is a patient with the disease. The FC-BN-LeakyReLU module consists of a fully connected neural network layer FC, a batch normalization layer BN, and a LeakyReLU activation function layer.
[0061] 3) Train the sub-models using the training set. For traditional machine learning models, use grid search to select the optimal hyperparameters of sub-models M1, M2 and M3 and use the validation set to validate them. Finally, the optimal sub-models M1, M2 and M3 are obtained. For deep learning models, the Adam optimization algorithm is used to optimize sub-model M4. The loss function is the binary cross-entropy loss function. Finally, the optimal sub-model M4 is obtained.
[0062] The sub-model fusion module uses ensemble learning to extract predictions from the four sub-models and uses the average of the predictions as the final prediction. Let the ensemble learning classification model be y = f(M1, M2, M3, M4), where y is the class predicted by the model and f(·) is the classification model. Then the final prediction result is:
[0063]
[0064] In the formula, p uv For u th The prediction result of the v-th sub-model of class label, u th Let y represent the u-th group of samples grouped by category label. * This is the predicted result.
[0065] Specifically, the intelligent recognition and prediction module includes the following steps:
[0066] 1) Use the data acquisition module to load the multimodal magnetic resonance imaging data, and use the data preprocessing module to process the multimodal magnetic resonance imaging data;
[0067] 2) The multimodal magnetic resonance imaging data processed in step 1) is input into the integrated learning classification model constructed by the intelligent recognition training module for disease prediction.
[0068] Specifically, the prediction result interpretation module includes a feature importance interpretation module and a feature interaction interpretation module, wherein:
[0069] The feature importance interpretation module is used to calculate the contribution value of each feature to the model prediction, i.e., feature importance, based on the performance error method, including the following steps:
[0070] 1) Select samples from the test set where the model correctly predicts the class y′, where y′ is one of multiple class labels, and replace the given feature F. i Calculate the permutation given feature F i The error in prediction performance before and after classification accuracy is considered as the importance of the feature, and is calculated using the following formula:
[0071] E i =f(F1,F2,F3,...,F i ,...,F m )-f(F1,F2,F3,...,perm(F i ),...,F m )
[0072] In the formula, F i E represents the i-th feature among all features. i Representing feature F i The importance of perm(F) i ) represents the permutation feature F i E i The larger the value, the stronger the feature F. i The greater the importance of E in prediction tasks, i The negative value of F indicates that the feature F i It may hinder or affect accurate prediction;
[0073] 2) Sort the importance of all features obtained in step 1), and take the top K features in terms of final importance as the important features for the model to predict category y′;
[0074] 3) The selected important features were validated. The test set obtained by masking the top 1, top 5, top 10, ..., top K important features was input into the ensemble learning classification model and the test results were compared with the test results of the original test set. The accuracy index decreased by 15% and the F1 value decreased by 18% in the test results, which fully verified the effectiveness of the selected top K features.
[0075] The feature interaction interpretation module is used to test whether the interaction effect between various important features and the interaction effect between important features and features of user interest are significant, including the following steps:
[0076] 1) Calculate the important feature F that is simultaneously permuted i And important features or features of interest to users F j Prediction performance error E i,j :
[0077] E i,j =f(F1,F2,F3,...,F i ,F j ...,F m )-f(F1,F2,F3,...,perm(F i ,F j ),...,F m )
[0078] In the formula, F j Let perm(F) represent the j-th feature among all features. i ,F j ) represents the permutation feature F i and feature F j ;
[0079] 2) Feature interaction strength is measured by the error in prediction performance; therefore, feature F is calculated. i and feature F j Interaction intensity I ij :
[0080] I ij =E i +E j -E i,j
[0081] In the formula, E j It refers to feature F j Performance error, I ij The larger the value, the more characteristic F is. i and feature F j The greater the interaction strength between them, the stronger I ij A value of 0 indicates feature F i and feature F jThere is no interaction between them; a positive value indicates feature F. i and feature F j There is a positive interaction between them; a negative value indicates a feature F. i and feature F j There is negative interaction between them;
[0082] 3) Calculate the pairwise interaction strength between the K important features obtained by the feature importance interpretation module and sort them. At the same time, calculate the pairwise interaction strength between the important features and the user's interest features and sort them. Take the feature pairs with the highest final interaction strength as the important interaction feature pairs with the greatest impact on the model prediction.
[0083] Calculate the feature interaction strength between Q×2 features and other modal features of interest to the user, rank them, and select the top T feature pairs with the largest feature interaction strength as the feature pairs with the greatest impact on model prediction from different modal feature groups.
[0084] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.
Claims
1. A multimodal magnetic resonance imaging data analysis system, characterized in that, include: The data import module is used to acquire multimodal magnetic resonance imaging data and preprocess the data to obtain a set of feature matrices for the multimodal magnetic resonance imaging data. The intelligent recognition training module uses the feature matrix set of multimodal magnetic resonance imaging data obtained by the data import module to train machine learning models and deep learning models based on optimization algorithms. Then, it uses ensemble learning to fuse the trained machine learning models and deep learning models to construct the required ensemble learning classification model. The intelligent identification and prediction module predicts diseases based on the integrated learning classification model constructed by the intelligent identification training module for the multimodal magnetic resonance imaging data to be tested; wherein, the multimodal magnetic resonance imaging data to be tested is collected and preprocessed by the data import module; The prediction result interpretation module is used to interpret the feature importance and perform feature interaction testing based on the prediction results of the intelligent recognition prediction module. The intelligent recognition training module includes a sub-model training module and a sub-model fusion module, wherein: The sub-model training module is used to perform feature fusion on samples and construct three machine learning models and one deep learning model. The three machine learning models are a linear kernel function support vector machine model, a radial basis function kernel function support vector machine model, and a random forest model, respectively. The deep learning model is a multilayer perceptron model, and specifically includes the following steps: 1) Set the feature matrices of each modal magnetic resonance imaging data of the sample. Concatenate the data and convert it into the corresponding vector form. ,in, This is the feature matrix of the first modal magnetic resonance imaging data. For the first Feature matrix of modal magnetic resonance imaging data The number of modes in magnetic resonance imaging data. The size of the feature matrix, For the number of features, For the sample size, Represents an eigenvector. This represents the total number of feature vectors. The transformed feature vectors are divided into training, validation, and test sets, and then the data is normalized. 2) Construct a set of sub-models Sub-model For linear kernel function support vector machine models, sub-models For radial basis function kernel function support vector machine models, sub-models For the random forest model, sub-model This is a multilayer perceptron model, where the multilayer perceptron model... It consists of one input layer, two FC-BN-LeakyReLU modules, and one output layer. A dropout layer is added after the two FC-BN-LeakyReLU modules to avoid model overfitting. The sigmoid function is applied to the output layer to predict the probability that any input instance is a disease patient. The FC-BN-LeakyReLU module consists of a fully connected neural network layer FC, a batch normalization layer BN, and a LeakyReLU activation function layer. 3) Train sub-models using the training set. For traditional machine learning models, use grid search to select sub-models. , and The optimal hyperparameters were determined and validated using a validation set, ultimately yielding the optimal sub-model. , and For deep learning models, the Adam optimization algorithm is used to optimize sub-models. The loss function used is the binary cross-entropy loss function, which ultimately yields the optimal sub-model. ; The sub-model fusion module utilizes ensemble learning to extract predictions from the four sub-models and uses the average of these predictions as the final prediction; let the ensemble learning classification model be... ,in, For the category predicted by the model, For a classification model, the final prediction result is: ； In the formula, for The first class tag The prediction results of each sub-model This indicates the first group of samples grouped by category label. Group, This is the predicted result.
2. The multimodal magnetic resonance imaging data analysis system according to claim 1, characterized in that, The data import module includes a data acquisition module and a data preprocessing module, wherein: The data acquisition module reads multimodal magnetic resonance imaging data and clinical data from the local machine, including nii, gz, dcm, CSV and xlsx formats. The multimodal magnetic resonance imaging data includes T1-weighted magnetic resonance imaging and arterial spin label perfusion magnetic resonance imaging, and the clinical data includes sample age, gender and years of education. The data preprocessing module is used to remove samples with missing modalities, divide the data into several groups according to category labels, remove samples that fail the t-test for age and years of education, remove the mean of samples that fail the chi-square test for gender, and for magnetic resonance imaging data, use the SPM, CAT12, and ExploreASL toolkits to extract the feature matrix set of each modality of magnetic resonance imaging data. .
3. The multimodal magnetic resonance imaging data analysis system according to claim 2, characterized in that, The intelligent recognition and prediction module includes the following steps: 1) Use the data acquisition module to load multimodal magnetic resonance imaging data, and use the data preprocessing module to process the multimodal magnetic resonance imaging data; 2) The multimodal magnetic resonance imaging data processed in step 1) is input into the integrated learning classification model constructed by the intelligent recognition training module for disease prediction.
4. The multimodal magnetic resonance imaging data analysis system according to claim 3, characterized in that, The prediction result interpretation module includes a feature importance interpretation module and a feature interaction interpretation module, wherein: The feature importance interpretation module is used to calculate the contribution value of each feature to the model prediction, i.e., feature importance, based on the performance error method, including the following steps: 1) Select model prediction categories from the test set Correct sample, For one of multiple category labels, replace a given feature. Calculate the permutation of given features The error in prediction performance before and after classification accuracy is considered as the importance of the feature, and is calculated using the following formula: ； In the formula, Represents the first of all features One characteristic, Representation of features The importance of Represents permutation features , The larger the value, the stronger the feature. The greater its importance in prediction tasks, Negative values indicate features It may hinder or affect accurate prediction; 2) Rank the importance of all features obtained in step 1), and take the feature with the highest final importance ranking. Features, as predicted categories by the model Key features; 3) Verify the selected key features by masking the top 1, top 5, top 10, ..., top ... The test set obtained after modifying key features is input into the ensemble learning classification model, and the test results are compared with the test results of the original test set. This fully verifies the effectiveness of the selected key features. The effectiveness of the importance of each feature; The feature interaction interpretation module is used to test whether the interaction effect between various important features and the interaction effect between important features and features of user interest are significant, including the following steps: 1) Calculate important features by simultaneous permutation and important features or features that users are interested in Prediction performance error : ； In the formula, Represents the first of all features One characteristic, Represents permutation features and characteristics ; 2) Feature interaction strength is measured by the error in prediction performance; therefore, calculating features... and characteristics Interaction intensity : ； In the formula, It refers to characteristics Performance error, The larger the value, the more characteristic it indicates. and characteristics The greater the interaction strength between them, A value of 0 indicates a feature and characteristics There is no interaction between them; a positive value indicates a feature. and characteristics There is a positive interaction between them; negative values indicate features. and characteristics There is negative interaction between them; 3) The feature importance interpretation module obtained The interaction strength between each pair of important features is calculated, and the interaction strength between important features and user-interested features is also calculated. These are then ranked, with the final interaction strength ranked first. The feature pairs are the most important interactive feature pairs that have the greatest impact on model prediction as feature interaction effects. calculate The strength of the feature interaction between each feature and other modalities of interest to the user is used to rank and select the top-ranked features. The feature pair with the strongest interaction strength is the feature pair that has the greatest impact on the model prediction due to the interaction of different modal feature groups.