Executive function prediction method, device and equipment based on Riemannian manifold and multimodal connectome

Abstract

The present application relates to the field of multimodal neuroimaging and machine learning technology, and in particular to an executive function prediction method, device and equipment based on Riemann manifolds and multimodal connection groups. The present application constructs a multimodal brain connection group by mapping the structural connection group and the functional connection group to the tangent space of the Riemann manifold, and combines the random forest algorithm for predictive modeling, thereby constructing a model that can effectively predict the performance of individual executive functions; by integrating the data of the structural connection group and the functional connection group, the extracted individual connection group is projected into the tangent space to construct a Tangent connection matrix that reflects individual differences; and then the random forest algorithm is used in combination with the Tangent connection matrix. The present application not only overcomes the limitation of relying on a single data source, but also optimizes the information extraction process, improves the ability to identify relevant patterns from complex neural network data, and improves the accuracy and robustness of the prediction results.

Description

Technical Field

[0001] The present application relates to the field of multimodal neuroimaging and machine learning technology, and in particular to a method, device and equipment for predicting executive function based on Riemannian manifolds and multimodal connectomes. Background Art

[0002] Executive function is a high-level cognitive ability that allows individuals to flexibly adjust and integrate cognitive resources according to goal requirements to guide and regulate their own behavior and achieve goal-oriented activities. Existing studies have shown that executive function is closely related to the coordinated activities of brain networks. The large-scale brain network distributed control framework provides a more reasonable theoretical basis for explaining this high-order cognitive process. Under this framework, the connectome (referring to the functional and structural connections between various regions in the brain) plays a key role in characterizing macroscopic brain networks.

[0003] However, traditional research methods have limitations in exploring the relationship between executive function and brain network activity, such as over-reliance on simple correlation analysis, ignoring the overall context of neural activity, and difficulty in establishing causal relationships between behavior and brain networks. This leads to inaccuracies in traditional research methods in exploring the relationship between executive function and brain activity.

[0004] Therefore, how to accurately predict executive functions based on brain networks has become an urgent problem that needs to be solved. Summary of the invention

[0005] The present application provides an executive function prediction method, apparatus and device based on Riemannian manifolds and multimodal connection groups, which can accurately predict executive functions according to brain networks.

[0006] In a first aspect, an embodiment of the present application provides an executive function prediction method based on Riemann manifolds and multimodal connection groups, the method comprising: obtaining historical executive function measurement indicators and historical brain neuroimaging data, the historical brain neuroimaging data comprising different categories of historical T1 structural image data, historical functional magnetic resonance data and historical diffusion weighted magnetic resonance imaging data; determining executive function measurement indicator preprocessing data and brain neuroimaging preprocessing data by preprocessing the historical executive function measurement indicators and historical brain neuroimaging data; determining structural connection groups and function preprocessing data based on the brain neuroimaging preprocessing data by AAL-116 atlas segmentation and Shen-268 atlas segmentation. The method comprises the steps of: mapping the structural connection group and the functional connection group to the tangent space of the Riemann manifold to determine the characteristic matrix of the structural connection group and the functional connection group; preprocessing the data based on the characteristic matrix and the executive function measurement index, training the data through the random forest algorithm, and determining a prediction model, wherein the prediction model is used to predict the executive function; after determining the prediction model, obtaining the brain neuroimaging data to be detected, wherein the brain neuroimaging data to be detected includes T1 structural image data, functional magnetic resonance imaging data, and diffusion-weighted magnetic resonance imaging data; determining the executive function corresponding to the brain neuroimaging data to be detected based on the brain neuroimaging data to be detected and the prediction model.

[0007] In a possible implementation, mapping the structural connection group and the functional connection group to the tangent space of the Riemann manifold using the logarithmic Euclidean average as a reference matrix to determine the feature matrix of the structural connection group and the functional connection group includes: determining the reference matrix C by calculating the logarithmic Euclidean average based on the structural connection group and the functional connection group ref ; Based on the matrix C ref The points on the Riemann manifold are converted to the tangent space by a logarithmic mapping formula to determine the characteristic matrices of the structural connection group and the functional connection group. The logarithmic mapping formula is:

[0008]

[0009] in, Characterization reference matrix C ref The square root of the inverse matrix, log m Represents the logarithmic transformation of the matrix, It represents the characteristic matrix after the transformation of the structural connectome and the functional connectome, and Q represents the symmetric positive definite matrix.

[0010] In a possible implementation, the historical execution function measurement indicators are preprocessed to determine execution function measurement indicator preprocessing data, including: based on the historical execution function measurement indicators, determining reaction time data and accuracy data through data screening; based on the reaction time data and accuracy data, filling missing values ​​in the data with the mode of the data and removing outliers in the data through data cleaning, processing the missing values ​​and outliers in the reaction time and accuracy data, and determining cleaned data; based on the cleaned data determined, determining standardized data through standardization; based on the standardized data, calculating the correlation matrix between the standardized data; based on the correlation matrix, performing eigenvalue decomposition to obtain eigenvalues ​​and eigenvectors; based on the eigenvalues ​​and eigenvectors, determining the execution function measurement indicator preprocessing data, the execution function measurement indicator preprocessing data includes a first principal component, and the first principal component represents the eigenvector of the maximum eigenvalue.

[0011] In a possible implementation, the feature matrix and the executive function measurement index preprocessing data are trained by a random forest algorithm to determine the prediction model, including: determining the correlation by a correlation formula based on the feature matrix and the executive function measurement index preprocessing data; dividing the correlation into a positive network, a negative network and a whole brain network based on the direction corresponding to the correlation; summing the edges contained in the positive network, the negative network and the whole brain network respectively to determine the prediction feature; determining the prediction model by a random forest algorithm based on the prediction feature and the executive function measurement index preprocessing data; the correlation formula is:

[0012]

[0013] Among them, r represents the Pearson correlation calculated between the first principal component and each edge in the feature matrix, n is the number of test samples, x represents an edge in each test sample's feature matrix, and y represents the first principal component of the test sample's execution function.

[0014] In a possible implementation, after the data is preprocessed based on the prediction features and the executive function measurement indicators and the prediction model is determined through a random forest algorithm, it also includes: based on the prediction model, through model evaluation, calculating the root mean square error between the actual behavior variables and the predicted behavior variables; based on the root mean square error, evaluating the measure of the strength of the relationship between the predicted behavior variables and the actual behavior variables of the prediction model; performing a permutation test on the prediction model to evaluate the significance of the prediction model; wherein the permutation test step includes: shuffling data labels, refitting the model, generating a permutation distribution, and evaluating the significance of the prediction model based on the permutation distribution.

[0015] In a possible implementation, after the data is preprocessed based on the prediction features and the executive function measurement indicators and the prediction model is determined through a random forest algorithm, it also includes: based on the random forest algorithm, obtaining the importance characteristic values ​​of the structural connection group and the functional connection group; sorting the structural connection group and the functional connection group according to the size of the importance characteristic value to determine the importance of the structural connection group and the functional connection group; based on the importance and the brain map, determining a connection weight heat map, and the connection weight heat map is used to characterize the importance of brain regions and the structural connection group and the functional connection group in executive function prediction.

[0016] In a possible implementation, the historical T1 structural image data is preprocessed, including: based on the historical T1 structural image data, determining brain tissue data through brain extraction processing; based on the brain tissue data, performing brain segmentation, dividing the brain tissue into gray matter, white matter and cerebrospinal fluid, and determining brain segmentation image data; based on the brain segmentation image data, mapping the T1 structural segmentation image data to a standardized template through spatial standardization, and determining the linear and nonlinear transformation matrix from individual space to standard space; based on the transformation matrix, aligning the historical diffusion weighted magnetic resonance imaging data and the historical functional magnetic resonance data to the T1 structural image data through a linear algorithm; and applying the transformation matrix to the aligned historical diffusion weighted magnetic resonance imaging data and historical functional magnetic resonance data to achieve standard space alignment.

[0017] In a second aspect, an embodiment of the present application provides an executive function prediction device based on a Riemannian manifold and a multimodal connection group, the device comprising: the device comprising:

[0018] An acquisition module is used to acquire historical executive function measurement indicators and historical brain neuroimaging data, wherein the historical brain neuroimaging data includes different categories of historical T1 structural image data, historical functional magnetic resonance data, and historical diffusion-weighted magnetic resonance imaging data; a preprocessing module is used to determine executive function measurement indicator preprocessing data and brain neuroimaging preprocessing data by preprocessing the historical executive function measurement indicators and historical brain neuroimaging data; a determination module is used to determine a structural connection group and a functional connection group based on the brain neuroimaging preprocessing data by AAL-116 atlas segmentation and Shen-268 atlas segmentation; the determination module is also used to map the structural connection group and the functional connection group to the tangent space of the Riemann manifold, and determine the characteristic matrices of the structural connection group and the functional connection group; the determination module is also used to pre-process the data based on the characteristic matrix and the executive function measurement index, and determine the prediction model through training by the random forest algorithm, and the prediction model is used to predict the executive function; the acquisition module is also used to acquire the brain neural imaging data to be detected after determining the prediction model, and the brain neural imaging data to be detected includes T1 structural image data, functional magnetic resonance data and diffusion-weighted magnetic resonance imaging data; the determination module is also used to determine the executive function corresponding to the brain neural imaging data to be detected based on the brain neural imaging data to be detected and the prediction model.

[0019] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method described in the first aspect or any one of the implementation methods thereof is implemented.

[0020] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect or any one of the implementation methods thereof is implemented.

[0021] This application determines the executive function corresponding to the brain neuroimaging data to be tested through the brain neuroimaging data to be tested and the prediction model. In the process of building the prediction model, the structural connection group and the functional connection group are determined based on the historical brain neuroimaging data, and the structural connection group and the functional connection group are mapped to the tangent space of the Riemann manifold to determine the feature matrix. The use of Riemann manifold geometry can maintain the intrinsic geometric properties of the data while simplifying pattern recognition and machine learning tasks. By operating on the Riemann manifold, the brain connection data can be processed and interpreted more realistically, and the accuracy of subsequent recognition and prediction can be improved. Then, combined with the random forest algorithm, multiple decision trees are constructed and their prediction results are integrated to improve the accuracy and robustness of the prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A flowchart of an executive function prediction method based on Riemannian manifold and multimodal connection group provided in one embodiment of the present application;

[0024] Figure 2 A schematic diagram of determining the structural connectome and the functional connectome by AAL-116 atlas segmentation and Shen-268 atlas segmentation provided in this application;

[0025] Figure 3 A structural block diagram of an executive function prediction device based on Riemannian manifold and multimodal connection group provided in one embodiment of the present application;

[0026] Figure 4 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the implementation methods of this application to clearly and completely describe the technical solutions in the implementation methods of this application. Obviously, the described implementation methods are only part of the implementation methods of this application, not all of the implementation methods. Based on the implementation methods in this application, all other implementation methods obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] Executive function is a high-level cognitive ability that allows individuals to flexibly adjust and integrate cognitive resources according to goal requirements to guide and regulate their own behavior and achieve goal-oriented activities. Existing studies have shown that executive function is closely related to the coordinated activities of brain networks. The large-scale brain network distributed control framework provides a more reasonable theoretical basis for explaining this high-order cognitive process. Under this framework, the connectome (referring to the functional and structural connections between various regions in the brain) plays a key role in characterizing macroscopic brain networks.

[0029] However, traditional research methods have limitations when exploring the relationship between executive function and brain network activity, such as over-reliance on simple correlation analysis, ignoring the overall context of neural activity, and difficulty in establishing causal relationships between behavior and brain networks. These limitations make it an important challenge to accurately predict individual executive function. With the advancement of multimodal neuroimaging technology (such as functional magnetic resonance imaging, diffusion tensor imaging, etc.) and machine learning technology, new possibilities have been provided for in-depth exploration of the detailed relationship between brain connectomes and executive function. These technologies can capture the structural and functional information of the brain at different levels, laying a data foundation for establishing more accurate prediction models.

[0030] In this context, the Tangent matrix method, as an innovative mathematical tool, has shown great potential in the field of neuroscience, especially in brain connectome research. The Tangent matrix is ​​based on Riemann geometry theory and can effectively capture the complex relationships between data points in high-dimensional space. In brain connectome analysis, the Tangent matrix method has the following significant advantages: (1) Enhanced capture of variability: Accurately capturing differences in connectivity between individuals helps to understand individual specificity and inter-group variation. (2) Efficient processing of high-dimensional data: While retaining important structural information, it effectively reduces the data dimension and is suitable for complex machine learning tasks. (3) Adaptability to a variety of analysis methods: It can be combined with a variety of statistical and machine learning techniques to enhance the flexibility of analysis. (4) Improved stability and robustness: Through logarithmic Euclidean averaging, it reduces the impact of noise and outliers and provides more reliable results. Although the Tangent matrix method has shown many advantages in brain connectome research, how to effectively integrate multimodal data, extract key features from massive information, and design prediction algorithms that can make full use of this information are still the main challenges facing current research.

[0031] Therefore, this application proposes an executive function prediction method, device and equipment based on Riemann manifold and multimodal connectome. The method is to use the Tangent matrix method to build a brain connectome model, and integrate data from different neuroimaging modalities and combine the prediction model to predict executive function. This application can not only significantly improve the accuracy of the prediction, but also provide a new perspective for in-depth understanding of the neural basis of executive function. This application overcomes the limitations of existing methods, gives full play to the advantages of Tangent matrix in multimodal neuroimaging data analysis, and provides a more reliable and effective tool for the prediction and research of executive function.

[0032] Figure 1 A flowchart of an executive function prediction method based on Riemann manifold and multimodal connection group provided in one embodiment of the present application, the method comprising:

[0033] S110, obtaining historical executive function measurement indicators and historical brain neuroimaging data, wherein the historical brain neuroimaging data includes different categories of historical T1 structural image data, historical functional magnetic resonance data, and historical diffusion-weighted magnetic resonance imaging data. Historical T1 structural image data, historical functional magnetic resonance data, and historical diffusion-weighted magnetic resonance imaging data.

[0034] Specifically, before creating a prediction model, it is necessary to first obtain multiple groups of historical data to train the model. The acquired data include different categories of historical functional magnetic resonance data (Functional Magnetic Resonance Imaging, fMRI) and historical diffusion weighted magnetic resonance imaging data (Diffusion Weighted Imaging, DWI) and historical T1 structural image data.

[0035] S120, preprocessing the historical executive function measurement indicators and the historical brain nerve image data to determine executive function measurement indicator preprocessing data and brain nerve image preprocessing data.

[0036] By preprocessing historical fMRI, historical DWI and historical T1 structural image data to reduce the impact of noise and extreme values ​​on model building, the first principal component is determined for historical executive function measurement indicators.

[0037] In a possible implementation, the historical T1 structural image data is preprocessed, including: based on the historical T1 structural image data, removing non-brain tissues such as scalp and skull through brain extraction processing, retaining only brain substance, determining brain tissue data, and improving accuracy; based on the brain tissue data, performing brain segmentation, dividing brain tissue into gray matter, white matter and cerebrospinal fluid, determining brain segmentation image data, and providing high-quality individual anatomical feature data for subsequent registration; based on the brain segmentation image data, mapping the T1 structural segmentation image data to a standardized template through spatial standardization, and determining linear and nonlinear transformation matrices from individual space to standard space; based on the transformation matrix, registering the historical diffusion weighted magnetic resonance imaging data and the historical functional magnetic resonance data to the T1 structural image data through a linear algorithm; applying the transformation matrix to the registered historical diffusion weighted magnetic resonance imaging data and historical functional magnetic resonance data to achieve registration of the two modalities of historical diffusion weighted magnetic resonance imaging data and historical functional magnetic resonance data to the standard space. The registration based on T1 structure is across subjects but not across modalities, while the direct registration without T1 structure is across subjects and across modalities. The former has better effect.

[0038] As an example, the historical fMRI is preprocessed to determine the preprocessed fMRI, specifically including:

[0039] 1. Brain extraction is performed on the converted NIfTI format fMRI data. Brain extraction is performed on the structural image. A centralized iterative algorithm (robust brain center estimation) is used to optimize the brain extraction by updating and optimizing the surface model based on the interface between brain and non-brain tissues through multiple iterations.

[0040] 2. Perform head motion correction on the extracted data to correct the head movement during the scanning process. Usually, a rigid body transformation (6 degrees of freedom) is used to align the images of all time points to the reference body to reduce motion-induced artifacts and improve the quality of time series data;

[0041] 3. Perform time layer correction to correct the acquisition time difference between different brain slices. Use linear interpolation technology to ensure that the BOLD signal of the whole brain is aligned in time;

[0042] 4. Perform distortion correction on the time-layer corrected data to correct the geometric distortion caused by magnetic field inhomogeneity, and use a combination of field maps and methods based on inverse phase encoding to improve the spatial accuracy of functional images;

[0043] 5. Co-register the distortion-corrected data to align the functional image with the high-resolution structural image. Use affine transformations such as mutual information or correlation ratios to map functional activity to precise anatomical locations.

[0044] 6. After the co-registration process, a spatial normalization process is performed, which is similar to the spatial normalization of historical T1 structural image data, but only the functional image is normalized;

[0045] 7. Finally, based on the standardized data, spatial smoothing was performed to increase the signal-to-noise ratio, and convolution was performed using a Gaussian kernel to compensate for anatomical differences between individual samples, with a full width at half maximum of 6 mm.

[0046] As an example, preprocessing the historical DWI data to determine the preprocessed DWI data specifically includes:

[0047] 1. According to the converted DWI data in NIfTI format, noise reduction is performed to reduce thermal noise and physiological noise, and a PCA-based method is used to improve image quality;

[0048] 2. For the image data after noise reduction, Gibbs ring artifact correction is used to eliminate the ringing artifact caused by fast Fourier transform. By analyzing and correcting the edge of k-space data or image space, the streak artifact in the image is reduced and the accuracy of fiber tracking is improved.

[0049] 3. Combined with head motion correction and EPI distortion correction, it corrects the distortion caused by motion and magnetic field inhomogeneity, and improves image quality and spatial accuracy;

[0050] 4. For the data after head motion and distortion correction, bias field correction is performed to correct the signal intensity changes caused by the unevenness of the RF coil. The low-frequency signal changes are estimated and removed using the N4 algorithm. This ensures the consistency of image intensity and improves the accuracy of subsequent analysis;

[0051] 5. Use the preprocessed T1 data to register the diffusion-weighted MRI data to the standard space: First, linearly register the diffusion MRI data to the individual's T1-weighted image. Second, register the T1-weighted image to the standard space to obtain the linear and nonlinear transformation matrix. Finally, apply this transformation matrix to the diffusion MRI data to ensure accurate alignment with the standard space and obtain the standardized diffusion MRI data.

[0052] 7. Finally, tissue segmentation is performed on the co-registered image data in diffusion space, and the segmentation results in T1 space (the boundary between gray matter and white matter) are converted to diffusion space to provide anatomical constraints for fiber tracking and microstructural analysis.

[0053] In a possible implementation, historical executive function measurement indicators are preprocessed to determine executive function measurement indicator preprocessing data, including: based on the historical executive function measurement indicators, determining reaction time data and accuracy data through data screening; based on the reaction time data and accuracy data, filling missing values ​​in the data with the mode of the data and removing outliers in the data through data cleaning, processing missing values ​​and outliers in the reaction time and accuracy data, and determining cleaned data; based on the determined cleaned data, determining standardized data through standardization, wherein standardization is Z-score standardization, standardizing the data to between 0 and 1 to eliminate the influence of dimensional units so that different measurements can be compared; based on the standardized data, calculating the correlation matrix between the standardized data; based on the correlation matrix, performing eigenvalue decomposition to obtain eigenvalues ​​and eigenvectors; based on the eigenvalues ​​and eigenvectors, determining executive function measurement indicator preprocessing data, the executive function measurement indicator preprocessing data includes a first principal component, the first principal component represents the eigenvector of the maximum eigenvalue, and then the first principal component is stored for subsequent analysis.

[0054] S130, based on the brain neuroimaging preprocessing data, determines the structural connectome and functional connectome through AAL-116 atlas segmentation and Shen-268 atlas segmentation.

[0055] Figure 2The schematic diagram of determining the structural connection group and the functional connection group by AAL-116 atlas segmentation and Shen-268 atlas segmentation provided in this application is as follows Figure 2 As shown in the figure, based on the preprocessed fMRI data, the functional connection group is determined by segmentation of the Shen-268 atlas. The Shen-268 atlas divides the brain into 268 nodes, covering the entire cortex, subcutaneous area and cerebellum, and further organized into 10 typical brain networks. For each sample, the BOLD time series of different brain regions are first extracted from the preprocessed fMRI data, and the Pearson correlation coefficient between these time series is calculated to construct a 268*268 functional connection group for each individual, that is, a symmetrical correlation matrix.

[0056] like Figure 2 As shown in the figure, according to the preprocessed DWI data, AAL-116 defines the nodes of the brain structure connection group, uses the constrained spherical deconvolution algorithm to estimate the fiber orientation distribution, and performs fiber tract tracking to finally determine the structure connection group. Specifically: First, the Fournier algorithm is used to estimate the response function for the constrained spherical deconvolution (CSD) in the DWI data, and the diffusion direction function of each voxel is calculated to obtain the fiber orientation distribution image. In the fiber tract tracking process, the iFOD2 algorithm is used to track the white matter fibers to obtain the fiber tract connection of the whole brain of each sample, and three tissue type segmentation images (segmentation results of T1 space) are used as labels for the fiber tracking area, including white matter, gray matter and cerebrospinal fluid. In order to enhance the connectivity of the fiber tract, a backtracking strategy is adopted and the fiber tracking is terminated at the junction of gray matter and white matter to avoid crossing between different tissue types. The sensitivity and selectivity of fiber tracking are controlled by setting the relative threshold and the maximum turning angle threshold, and the fiber length is limited to between 25 mm and 250 mm to filter out fiber tracts that are too short or too long. The junction of gray matter and white matter was selected as the starting point for tracking, and one million fiber bundles were tracked for each test sample. The iFOD2 algorithm was used to optimize the probabilistic fiber tracking process, calculate the average weight and linear combination coefficient of each fiber bundle, and thus determine the important fiber bundles. When constructing the streamline count matrix between different nodes based on important fiber bundles, the streamline count was normalized by the node volume to solve the problem of node size differences, thereby determining the streamline count matrix between different nodes of important fiber bundles. At the same time, in order to avoid over-representation caused by too many fiber bundles, the SIFT algorithm was used to offset overfitting, and finally a weighted optimized 90*90 structural connection group was obtained for each sample.

[0057] S140, mapping the structural connection group and the functional connection group to the tangent space of the Riemann manifold, and determining the feature matrices of the structural connection group and the functional connection group.

[0058] By mapping all functional and structural connectomes to the tangent space of the Riemann manifold, with the logarithmic Euclidean mean as the reference, the tangent feature matrix of each subject is obtained, and the feature matrix of each subject sample is obtained by tangent space mapping for subsequent analysis and model construction.

[0059] In a possible implementation, the structural connection group and the functional connection group are mapped to the tangent space of the Riemann manifold to determine the feature matrices of the structural connection group and the functional connection group, including: based on the structural connection group and the functional connection group, for each structural connection group and the functional connection group, the main diagonal regularization is first performed to improve the subsequent recognition ability, and the reference matrix C is determined by calculating the logarithmic Euclidean average. ref ; Based on matrix C ref Through the logarithmic mapping formula, the points on the Riemann manifold are transformed into the tangent space to determine the characteristic matrices of the structural connection group and the functional connection group. The logarithmic mapping formula is:

[0060]

[0061] in, Characterization reference matrix C ref The square root of the inverse matrix, log m Represents the logarithmic transformation of the matrix, Characterizes the feature matrix after the transformation of the structural connectome and the functional connectome, which quantifies the difference between the original connection matrix and the reference connection matrix. Q represents the symmetric positive definite matrix.

[0062] The key to transforming a point on a Riemann manifold to a tangent space is to use a logarithmic map to "linearize" the nonlinear structure on the Riemann manifold. The logarithmic map transforms a point on a Riemann manifold (i.e., the structural and functional connectome matrices) to a tangent space, which is a Euclidean space corresponding to the tangent space of the point on the manifold. Represents the tangent space matrix of each test sample finally mapped to the tangent space.

[0063] In this process, Riemannian manifolds provide a natural framework to process connectome data. These matrices are symmetric, positive definite, and form a complex geometric structure. Using Riemannian geometry, especially its metric and logarithmic mapping, can maintain the intrinsic geometric properties of the data while simplifying pattern recognition and machine learning tasks. By operating on this manifold, we can more realistically process and interpret brain connectivity data and improve the accuracy of recognition and prediction.

[0064] S150, preprocessing data based on the feature matrix and the executive function measurement index, training through a random forest algorithm, and determining a prediction model, where the prediction model is used to predict the executive function.

[0065] The tangent space matrix of each sample was used as the characteristic variable, and the preprocessed data of executive function measurement indicators were subjected to principal component analysis. The first principal component was extracted as the representation of executive function and as the predictor variable. The random forest method was used for model fitting, and the prediction model was finally determined.

[0066] In a possible implementation, based on the feature matrix and the executive function measurement index preprocessing data, the prediction model is determined by training through a random forest algorithm, including: based on the feature matrix and the executive function measurement index preprocessing data, the correlation is determined through a correlation formula; based on the direction corresponding to the correlation, the correlation is divided into a positive network, a negative network and a whole brain network; the edges contained in the positive network, the negative network and the whole brain network are summed up respectively to determine the prediction feature; based on the prediction feature and the executive function measurement index preprocessing data, the prediction model is determined through a random forest algorithm; the correlation formula is:

[0067]

[0068] Among them, r represents the Pearson correlation calculated between the first principal component and each edge in the feature matrix, n is the number of test samples, x represents an edge in the feature matrix of each test subject, and y represents the first principal component of the test subject's executive function.

[0069] Specifically, based on the feature matrix and the executive function measurement index preprocessing data, the correlation is determined by the correlation formula, including: based on the feature matrix and the first principal component in the executive function measurement index preprocessing data, the Pearson correlation is determined by the correlation formula. Because the first principal component represents the eigenvector of the maximum eigenvalue, when calculating the Pearson correlation, only the most significantly correlated edges are selected as predictive features. In order to obtain the optimal correlation threshold between the first principal component and the edge, a total of 500 iterations were performed from 0.0001 to 0.05 with an increment of 0.001. After determining the correlation, the directionality of the correlation in the feature selection process is divided into positive networks and negative networks, as well as the whole brain network according to the correlation. Each modality includes these three networks, and then the edges contained in each predicted network are summarized and summed to form the final predicted features of each subject sample, and then the prediction model is determined by the random forest algorithm based on the predicted features and the executive function measurement index preprocessing data (first principal component).

[0070] In a possible implementation, after preprocessing the data based on the prediction features and the executive function measurement indicators and determining the prediction model through the random forest algorithm, it also includes: evaluating the measure of the strength of the relationship between the predicted behavior variables and the actual behavior variables of the prediction model based on the root mean square error; performing a permutation test on the prediction model to evaluate the significance of the prediction model; wherein the steps of the permutation test include: shuffling the data labels, refitting the model, generating a permutation distribution, and evaluating the significance of the prediction model based on the permutation distribution.

[0071] Exemplarily, shuffling the data labels, refitting the model, and generating a permutation distribution includes: shuffling the data labels of the first principal component of the executive function measurement of each subject, for example: the sample of subject No. 1 is 4, the sample of subject No. 2 is 3, and the sample of subject No. 3 is 10. After random shuffling, the sample of subject No. 1 is 10, the sample of subject No. 2 is 4, and the sample of subject No. 3 is 3. Then, the shuffled data is used to refit the model to obtain the 0 distribution (permutation distribution) of the prediction effect of the shuffled labels.

[0072] Furthermore, when the result of the prediction model is in a preset position in the distribution of the refitted model, the model effect is considered to be significant.

[0073] Specifically, based on the prediction features as the predictive variables and the preprocessed data of the executive function measurement indicators (the first principal component) as the dependent variable, the prediction model is determined through the random forest algorithm; after the prediction model is determined, the model needs to be further evaluated.

[0074] In the optimization process, in order to avoid overfitting and improve the generalization ability of the constructed prediction model for new samples, the leave one out cross validation (LOOCV) method was used for optimization. This method involves using N-1 test samples as the training set for model training and feature selection, while leaving out one test sample as the test set. The model is then used to predict the executive function behavior variables of the left-out test sample, and iterated n times to obtain the prediction score for each test sample.

[0075] After cross-validation is completed, in order to evaluate the predictive performance of the model, the root mean square error (RMSE) between the actual behavior variables and the predicted behavior variables is calculated. The evaluation index provides a measure of the strength of the relationship between the predicted and true behavior variables;

[0076]

[0077] Where n is the total number of samples tested, y i is the actual behavioral performance measurement of executive function of the ith is the predicted value of executive function based on the i-th multimodal connectome.

[0078] Furthermore, in order to evaluate the statistical significance of the model, a permutation test was performed to check the reliability of the model results. A permutation distribution was generated by randomly shuffling the data labels and refitting the model multiple times. This process was repeated 10,000 times to produce an empirical distribution of statistical significance. The statistics of the prediction model were compared with the permutation distribution, and the p-value obtained was calculated to judge the significance of the original effect. If the results of the original model are at an extreme position in the permutation distribution, the model effect is considered to be significant.

[0079] In a possible implementation, after determining the prediction model, it also includes: obtaining the importance characteristic values ​​of the structural connection group and the functional connection group based on the random forest algorithm; sorting the structural connection group and the functional connection group according to the size of the importance characteristic value to determine the importance of the structural connection group and the functional connection group; based on the importance and the brain map, determining the connection weight heat map, the connection weight heat map is used to characterize the importance of brain regions and the structural connection group and the functional connection group in performing function prediction.

[0080] Exemplarily, random forest is an ensemble learning method that improves the accuracy and robustness of predictions by building multiple decision trees and integrating their predictions. In the process of building a multimodal connectome prediction model, in order to determine the contribution of each connection in the model prediction, the importance feature values ​​of the structural connectome and the functional connectome are obtained using the random forest algorithm. The importance feature values ​​in the random forest algorithm are calculated by observing the extent to which the segmentation of each feature in the structural connectome and the functional connectome in the decision tree improves the prediction performance. These importance values ​​can help identify the connections that are most critical for performing functional predictions.

[0081] Specifically, first, the feature importance values ​​of each connection group are extracted from the random forest algorithm, which represent the importance of each connection in the prediction model. Through these importance values, we can identify the connections that are significantly related to executive function. Next, the connections are sorted according to the size of the feature importance to identify the most critical connections. These important connections may correspond to specific neural networks in the brain that are closely related to executive function. In order to more intuitively show the importance of each connection, the feature importance information can be mapped to the brain map to generate a heat map of the connection weights. In this way, it is possible to more clearly observe which brain regions and connections play a key role in the prediction of executive function. In order to verify the significance of these feature importance values, statistical tests are performed, such as the permutation test method to evaluate the significance level of feature importance. The specific steps include: randomly shuffling the labels, refitting the model, generating a permutation distribution, and then comparing the relationship between the original model feature importance and the permutation distribution to determine the significance.

[0082] S160, after determining the prediction model, obtaining the brain nerve imaging data to be detected, the brain nerve imaging data to be detected includes T1 structural image data, functional magnetic resonance imaging data and diffusion weighted magnetic resonance imaging data.

[0083] S170, determining the execution function corresponding to the brain neuroimage data to be detected based on the brain neuroimage data to be detected and the prediction model.

[0084] The method described in this application proposes an executive function prediction technology, which uses the Tangent matrix method to construct a multimodal brain connection group and combines the random forest algorithm for predictive modeling. First, functional magnetic resonance imaging and diffusion-weighted magnetic resonance imaging data are used to construct functional connection groups and white matter structural connection groups, and then these connection group data are input into the random forest algorithm for fitting and parameter optimization, so as to construct a model that can effectively predict the performance of individual executive functions. The method described in this application can not only deepen the understanding of the complex neural basis of executive function, but also provide more reliable evidence for the early diagnosis of defects or abnormalities in individual executive function. The present application provides a method, by integrating the data of the structural connection group and the functional connection group, the extracted individual connection group is projected into the tangent space, and the Tangent connection matrix reflecting individual differences is constructed. This multimodal data integration and analysis technology not only overcomes the limitation of relying on a single data source by using the random forest algorithm and the Tangent connection matrix that can better reflect individual differences, but also optimizes the information extraction process and improves the ability to identify relevant patterns from complex neural network data. Through this data integration, a more accurate brain function map can be revealed, and further understanding of how structural and functional connections jointly affect individual executive functions can be achieved.

[0085] Figure 3 This is a structural block diagram of an execution function prediction device based on Riemann manifold and multimodal connection group provided in one embodiment of the present application. For the sake of convenience, only the part related to the embodiment of the present application is shown. Figure 3 The execution function prediction device 300 includes an acquisition module 301, a preprocessing module 302, and a determination module 303.

[0086] In one implementation, the apparatus 300 may be used to implement the above Figure 1 For example, the acquisition module 301 is used to implement S110 and S160, the preprocessing module 302 is used to implement S120, and the determination module 303 is used to implement S130 to S150, S170.

[0087] The method described in this application proposes an executive function prediction device, which uses the Tangent matrix method to construct a multimodal brain connection group and combines the random forest algorithm for predictive modeling. First, functional magnetic resonance imaging and diffusion-weighted magnetic resonance imaging data are used to construct functional connection groups and white matter structural connection groups, and then these connection group data are input into the random forest algorithm for fitting and parameter optimization, so as to construct a model that can effectively predict the performance of individual executive functions. The device described in this application can not only deepen the understanding of the complex neural basis of executive function, but also provide more reliable evidence for the early diagnosis of defects or abnormalities in individual executive function. This application projects the extracted individual connection group into the tangent space by integrating the data of the structural connection group and the functional connection group, and constructs a Tangent connection matrix reflecting individual differences. This multimodal data integration and analysis technology not only overcomes the limitation of relying on a single data source by using the random forest algorithm and the Tangent connection matrix that can better reflect individual differences, but also optimizes the information extraction process and improves the ability to identify relevant patterns from complex neural network data. Through this data integration, a more accurate brain function map can be revealed, and further understanding of how structural and functional connections jointly affect individual executive functions can be achieved.

[0088] Figure 4 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 4 As shown, the electronic device 4 of this embodiment includes: at least one processor 40 ( Figure 4 Only one is shown in the figure) a processor, a memory 41, and a computer program 42 stored in the memory 41 and executable on the at least one processor 40, wherein the processor 40 implements the steps in any of the above method embodiments when executing the computer program 42.

[0089] The electronic device 4 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device may include, but is not limited to, a processor 40 and a memory 41. Those skilled in the art will appreciate that Figure 4 It is only an example of the electronic device 4 and does not constitute a limitation on the electronic device 4. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0090] The processor 40 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0091] In some embodiments, the memory 41 may be an internal storage unit of the electronic device 4, such as a hard disk or memory of the electronic device 4. In other embodiments, the memory 41 may also be an external storage device of the electronic device 4, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 4. Further, the memory 41 may also include both an internal storage unit of the electronic device 4 and an external storage device. The memory 41 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory 41 may also be used to temporarily store data that has been output or is to be output.

[0092] The above description is only a preferred implementation mode of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for predicting executive function based on Riemannian manifolds and multimodal connectomes, characterized in that: The method comprises: Acquiring historical executive function measurement indicators and historical brain neuroimaging data, wherein the historical brain neuroimaging data includes different categories of historical T1 structural image data, historical functional magnetic resonance imaging data, and historical diffusion-weighted magnetic resonance imaging data; Determine executive function measurement index preprocessing data and brain neuroimage preprocessing data by preprocessing the historical executive function measurement index and the historical brain neuroimage data; Based on the brain neuroimaging preprocessed data, determining the structural connectome and the functional connectome through AAL-116 atlas segmentation and Shen-268 atlas segmentation; Mapping the structural connection group and the functional connection group to the tangent space of the Riemann manifold to determine the feature matrices of the structural connection group and the functional connection group; Based on the feature matrix and the executive function measurement index preprocessing data, training is performed through a random forest algorithm to determine a prediction model, wherein the prediction model is used to predict the executive function; After determining the prediction model, obtaining brain neuroimage data to be detected, wherein the brain neuroimage data to be detected includes T1 structural image data, functional magnetic resonance imaging data, and diffusion-weighted magnetic resonance imaging data; Determine the execution function corresponding to the brain nerve image data to be detected based on the brain nerve image data to be detected and the prediction model; Mapping the structural connection group and the functional connection group to the tangent space of the Riemann manifold to determine the feature matrix of the structural connection group and the functional connection group includes: Based on the structural connectome and functional connectome, the reference matrix C is determined by calculating the log-Euclidean mean. ref ; Based on the reference matrix C ref The points on the Riemann manifold are converted to the tangent space by a logarithmic mapping formula to determine the characteristic matrices of the structural connection group and the functional connection group. The logarithmic mapping formula is: in, Characterization reference matrix C ref The square root of the inverse matrix, log m Represents the logarithmic transformation of the matrix, It represents the characteristic matrix after the transformation of the structural connectome and the functional connectome, and Q represents the symmetric positive definite matrix.

2. The method according to claim 1, characterized in that The historical execution function measurement indicators are preprocessed to determine execution function measurement indicator preprocessing data, including: Based on the historical executive function measurement indicators, determining reaction time data and accuracy data through data screening; Based on the reaction time data and accuracy data, through data cleaning, the missing values ​​in the data are filled with the mode of the data, the outliers in the data are removed, the missing values ​​and outliers in the reaction time and accuracy data are processed, and the cleaned data are determined; Based on the cleaned data determined, determine standardized data by standardization; Based on the standardized data, calculating a correlation matrix between the standardized data; Based on the correlation matrix, eigenvalue decomposition is performed to obtain eigenvalues ​​and eigenvectors; Based on the eigenvalues ​​and eigenvectors, executive function measurement index preprocessing data is determined, wherein the executive function measurement index preprocessing data includes a first principal component, and the first principal component represents the eigenvector of the maximum eigenvalue.

3. The method according to claim 2, characterized in that The preprocessing of data based on the feature matrix and the executive function measurement index, training through a random forest algorithm, and determining a prediction model includes: Determining correlation based on the feature matrix and the executive function measurement index preprocessing data by using a correlation formula; Based on the direction corresponding to the correlation, the correlation is divided into a positive network, a negative network and a whole brain network; Summing the edges contained in the positive network, the negative network and the whole brain network respectively to determine the prediction features; Determining a prediction model by a random forest algorithm based on the prediction features and the executive function measurement index preprocessing data; The correlation formula is: Among them, r represents the Pearson correlation calculated between the first principal component and each edge in the feature matrix, n is the number of test samples, x represents an edge in each test feature matrix, and y represents the first principal component of the test subject's executive function.

4. The method according to claim 3, characterized in that After the preprocessing data based on the prediction features and the executive function measurement indexes and determining the prediction model by the random forest algorithm, the method further includes: Based on the prediction model, calculating the root mean square error between the actual behavior variable and the predicted behavior variable through model evaluation; Based on the root mean square error, a measure of the strength of the relationship between the predicted behavior variables and the actual behavior variables of the prediction model is evaluated; Performing a permutation test on the prediction model to evaluate the significance of the prediction model; The permutation test step includes: disrupting data labels, refitting the model, generating a permutation distribution, and evaluating the significance of the prediction model based on the permutation distribution.

5. The method according to claim 4, characterized in that After the preprocessing data based on the prediction features and the executive function measurement indexes and determining the prediction model by the random forest algorithm, the method further includes: Based on the random forest algorithm, obtaining the importance feature values ​​of the structural connection group and the functional connection group; Sorting the structural connection group and the functional connection group according to the magnitude of the importance eigenvalues ​​to determine the importance of the structural connection group and the functional connection group; Based on the importance and the brain map, a connection weight heat map is determined, and the connection weight heat map is used to characterize the importance of brain regions and the structural connection groups and functional connection groups in performing function prediction.

6. The method according to claim 1, characterized in that Preprocessing the historical T1 structural image data includes: Based on the historical T1 structural image data, determining brain tissue data through brain extraction processing; Based on the brain tissue data, performing brain segmentation, dividing the brain tissue into gray matter, white matter and cerebrospinal fluid, and determining brain segmentation image data; Based on the brain segmentation image data, the T1 structural segmentation image data is mapped to a standardized template through spatial standardization to determine the linear and nonlinear transformation matrices from the individual space to the standard space; Based on the transformation matrix, the historical diffusion-weighted magnetic resonance imaging data and the historical functional magnetic resonance data are registered to the T1 structural image data by a linear algorithm; The transformation matrix is ​​applied to the registered historical diffusion weighted magnetic resonance imaging data and historical functional magnetic resonance data to achieve standard spatial registration.

7. An executive function prediction device based on Riemannian manifold and multimodal connection group, characterized in that: The device comprises: An acquisition module, used to acquire historical executive function measurement indicators and historical brain neuroimaging data, wherein the historical brain neuroimaging data includes different categories of historical T1 structural image data, historical functional magnetic resonance imaging data, and historical diffusion-weighted magnetic resonance imaging data; A preprocessing module, used for determining executive function measurement index preprocessing data and brain nerve image preprocessing data by preprocessing the historical executive function measurement index and the historical brain nerve image data; A determination module, used to determine the structural connection group and the functional connection group based on the brain neuroimaging preprocessing data through AAL-116 atlas segmentation and Shen-268 atlas segmentation; The determination module is further used to map the structural connection group and the functional connection group to the tangent space of the Riemann manifold to determine the feature matrix of the structural connection group and the functional connection group; The determination module is further used to pre-process data based on the feature matrix and the executive function measurement index, train through a random forest algorithm, and determine a prediction model, wherein the prediction model is used to predict the executive function; The acquisition module is further used to acquire brain nerve image data to be detected after determining the prediction model, wherein the brain nerve image data to be detected includes T1 structural image data, functional magnetic resonance imaging data and diffusion-weighted magnetic resonance imaging data; The determination module is further used to determine the execution function corresponding to the detected brain nerve image data based on the brain nerve image data to be detected and the prediction model; Mapping the structural connection group and the functional connection group to the tangent space of the Riemann manifold to determine the feature matrix of the structural connection group and the functional connection group includes: Based on the structural connectome and functional connectome, the reference matrix C is determined by calculating the log-Euclidean mean. ref ; Based on the reference matrix C ref The points on the Riemann manifold are converted to the tangent space by a logarithmic mapping formula to determine the characteristic matrices of the structural connection group and the functional connection group. The logarithmic mapping formula is: in, Characterization reference matrix C ref The square root of the inverse matrix, log m Represents the logarithmic transformation of the matrix, It represents the characteristic matrix after the transformation of the structural connectome and the functional connectome, and Q represents the symmetric positive definite matrix.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images