Magnetic resonance image classification method based on adaptive high-order hyperedge time sequence multi-feature fusion

Abstract

The invention discloses a magnetic resonance image classification method based on adaptive high-order hyperedge time sequence multi-feature fusion, and the method comprises the steps: extracting a hyperedge time sequence through employing an LASSO method and a self-attention mechanism, and then extracting an adaptive high-order hyperedge time sequence based on Kalman filtering; and then global and local time features are extracted by using statistical indexes and the optimized expansion convolutional network, feature fusion is carried out by using a graph attention network, and finally classification is carried out. According to the method, on the basis of better simulating complex multi-element interaction of a human brain, the problems that an existing high-order brain function super-network ignores time dependence of different brain regions in a hyperedge and spurious fluctuation occurs when a sliding time window is adopted are solved, so that the spatial-temporal characteristics of the brain function super-network are represented more accurately, the classification accuracy is improved, and the classification efficiency is improved. And the accuracy of disease diagnosis of doctors is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a magnetic resonance image classification method based on adaptive high-order hyperedge time series multi-feature fusion. Background Art

[0002] Exploring the intrinsic structure and function of the human brain is a huge challenge in the field of brain imaging science. The combination of resting-state functional magnetic resonance imaging technology and complex network theory provides a new perspective for studying the construction of brain network models. In recent years, brain functional network models have been successfully applied to the evaluation and diagnosis of clinical brain diseases. However, the activity of brain neurons is very complex, and using simple binary networks to describe the interactive relationship between brain regions will lose high-order information. In this context, brain functional hypernetworks are introduced into the modeling of functional brain networks. Numerous studies have shown that resting-state brain functional hypernetworks can reflect the relationship between multiple interactions between brain regions, and conducting research on resting-state brain functional hypernetworks can obtain more pathological information. Therefore, resting-state brain functional hypernetworks are considered to have great potential in the field of brain disease diagnosis.

[0003] Research on brain functional hypernetworks usually uses the node features of the hypernetwork to characterize the topological information of the brain network, and uses a sliding time window to study the time variability of the brain. This method ignores the time dependence between different brain regions within a hyperedge and between different hyperedges. In addition, the sliding time window method can only estimate the connectivity strength of each time period and cannot obtain the connectivity strength at each time point. At the same time, the selection of sliding window parameters will bring about spurious fluctuation problems, thereby affecting the accuracy of classification and even the accuracy of doctors' diagnosis of diseases. Summary of the invention

[0004] In order to solve the problems existing in the prior art, the present invention provides a magnetic resonance image classification method based on adaptive high-order hyperedge time series multi-feature fusion. While retaining the multivariate information of the original hypergraph, it pays attention to the time dependence between different brain regions within the hyperedge and the spurious fluctuation problem caused by the time-varying nature of the traditional resting-state brain function hypernetwork representation, so as to more accurately characterize the high-order spatiotemporal characteristics of the human brain, improve the classification accuracy, and solve the problems mentioned in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: a magnetic resonance image classification method based on adaptive high-order hyperedge time series multi-feature fusion, comprising the following steps:

[0006] S1. Acquire resting-state functional magnetic resonance imaging data and perform preprocessing to obtain preprocessed imaging data; perform brain region segmentation on the preprocessed imaging data according to a selected standardized brain atlas, and extract the average time series of each brain region;

[0007] S2. Based on the average time series of each brain region, the weighted minimum shrinkage operator LASSO method was used to construct a brain function hypernetwork model;

[0008] S3, based on the brain function hypernetwork model, using the self-attention mechanism to extract the hyperedge time series of the subjects;

[0009] S4, based on the hyperedge time series, the Kalman filter algorithm is used to extract the subject's adaptive high-order hyperedge time series;

[0010] S5. Calculate the standard deviation, root mean square, skewness and kurtosis based on the adaptive high-order hyperedge time series, and use these four statistical indicators as global time features;

[0011] S6, based on adaptive high-order hyperedge time series, using the optimized time dilation convolutional network TDCN to extract local time features;

[0012] S7. Use the graph attention network to aggregate global temporal features and local temporal features to obtain high-order spatiotemporal aggregation features. Based on the high-order spatiotemporal aggregation features, use the fully connected layer classification network to classify the subject's magnetic resonance imaging data.

[0013] Preferably, in step S1, the preprocessing step specifically includes time layer correction, head motion correction, space standardization, and smoothing; the standardized brain map is an internationally used automatic anatomical labeling template; the brain region segmentation divides the image into 90 regions, including 45 regions in each of the left and right hemispheres.

[0014] Preferably, in step S1, the extracting the average time series of each brain region specifically includes: extracting activation signals of all voxels contained in each brain region at different time points, and then arithmetically averaging the activation signals of each voxel at different time points to obtain the average time series of the brain region.

[0015] Preferably, in step S2, it specifically includes:

[0016] Firstly, based on the average time series of brain regions, the weighted minimum shrinkage operator LASSO method was used to construct the brain function hypernetwork, which is specifically defined as:

[0017]

[0018] Among them, w represents the w-th brain region after atlas division, w∈{1,2,...,W}, W represents the number of divided brain regions; represents the time series of the wth brain region in the nth subject; represents the time series of the remaining brain regions except the wth brain region in the nth subject, is a weight vector, which indicates the influence of the remaining brain regions in the nth subject on the wth brain region; λ1 is a penalty parameter, λ1>0;

[0019] Then, for each subject n among the N subjects, λ1 is fixed and the weight vector corresponding to each brain region is calculated respectively. The non-zero weight vectors form a hyperedge of the corresponding brain region, and all hyperedges form a brain function supernetwork.

[0020] Preferably, in step S3, firstly, based on the brain function hypernetwork of each subject, the time series of all brain region nodes in each hyperedge of the brain function hypernetwork are extracted. represents the time series of the oth brain region under the nth subject, o, p, q represent the oth brain region node, the pth brain region node and the qth brain region node in the brain function supernetwork, e represents the eth hyperedge in the brain function supernetwork, and H represents the number of hyperedges;

[0021] Based on the brain function hypernetwork of each subject, the time series of all brain region nodes in each hyperedge are used as the input vector of the self-attention module. The dot product similarity between the query matrix Q and the key matrix K in the three trainable matrices is calculated to obtain the attention distribution of the value matrix V. Finally, the output attention weight vector at each different time point is:

[0022]

[0023] where d k is the dimension of the key-value vector, softmax normalizes the similarity scores row by row, and then the self-attention module assigns each softmax score to the corresponding value vector v t (t)(t=1,2,…,T), t represents the tth time point, T represents all time points in the time series, and finally the weighted sum of different time points is calculated to obtain the hyperedge time series

[0024] Preferably, in step S4, a Kalman filter algorithm is used to extract the subject's adaptive high-order hyperedge time series, as follows:

[0025]

[0026] in, represents the kth hyperedge time series of the nth subject, represents the value at the xth moment in the kth hyperedge time series of the nth subject, represents the functional connection strength between the kth hyperedge and the lth hyperedge at time t, and E(·) represents the mathematical expectation;

[0027] After removing weak connections through Kalman filtering, each subject finally obtains an adaptive high-order hyperedge time series. in represents the lth adaptive high-order hyperedge time series; represents the value of the tth time point in the lth adaptive high-order hyperedge time series, [·] T Represents the transpose of this vector.

[0028] Preferably, in step S5, the specific calculation formula of the standard deviation is as follows:

[0029]

[0030] in represents the standard deviation of the lth adaptive high-order hyperedge time series, represents the value of the tth time point in the lth adaptive high-order hyperedge time series, T represents the total number of time points in the time series, express , n is the nth subject;

[0031] The specific calculation formula of the root mean square is as follows:

[0032]

[0033] in represents the root mean square of the lth adaptive high-order hyperedge time series, represents the value of the t-th time point in the l-th adaptive high-order hyperedge time series, T represents the total number of time points, and n represents the n-th subject;

[0034] The specific calculation formula of the skewness is as follows:

[0035]

[0036] in represents the skewness of the lth adaptive high-order hyperedge time series, represents the value of the tth time point in the lth adaptive high-order hyperedge time series, T represents the total number of time points, express The mean of represents the standard deviation of the lth adaptive high-order hyperedge time series, and n is the nth subject;

[0037] The specific calculation formula of the kurtosis is as follows:

[0038]

[0039] in represents the kurtosis of the lth adaptive high-order hyperedge time series, represents the value of the tth time point in the lth adaptive high-order hyperedge time series, express The mean of represents the standard deviation of the lth adaptive high-order hyperedge time series, and n is the nth subject;

[0040] In the nth subject, calculate each adaptive high-order hyperedge Finally, the global time characteristics of the nth subject are obtained in [·] T Represents the transpose of this vector.

[0041] Preferably, in step S6, the optimized time dilation convolutional network TDCN includes branch 1, branch 2 and branch 3; branch 1 uses dilated causal convolution to extract causal time characteristics from the sliding window; branch 2 aggregates the time axis information of the sequence to obtain a complete local time feature representation; branch 3 uses 1D convolution to extract longer time scale dependencies and maintain network stability by adjusting the output dimension;

[0042] The adaptive high-order hyperedge time series η n As the input sequence, the optimized time dilation convolutional network TDCN is used to extract local time features and obtain LT n , the specific steps are as follows:

[0043] S61, for the adaptive high-order hyperedge time series η n , Branch 1 uses two dilated convolutional layers as well as weight normalization and linear rectification activation function strategies to extract features;

[0044] The first dilated convolutional layer DCC1 extracts the first-level features M1, and the second dilated convolutional layer DCC2 extracts the second-level features M2;

[0045] M1=Relu(WeightNorm(DCC1(η n )))

[0046] M1 is the first-level feature, where η n represents an adaptive high-order hyperedge time series;

[0047] M2=Relu(WeightNorm(DCC2(M1)))

[0048] M2 is the second-level feature, where M1 is the first-level feature;

[0049] S62, branch 2 uses a convolution layer with a dilation factor of 1. First, the adaptive high-order hyperedge time series information is aggregated through adaptive average pooling to obtain M3, and then through weight normalization and activation function Relu, M4 is obtained. Then, through a convolution layer with a convolution kernel size of 1, weight normalization and Relu, M5 is obtained. Finally, the time dimension of feature M5 is expanded to be equal to η through the fully connected layer FC. n The time dimension is consistent, and we get M6;

[0050] M3=AdaptiveAvgPool(η n )

[0051] M3 is the third-level feature, where η n represents an adaptive high-order hyperedge time series;

[0052] M4=Relu(WeightNorm(Conv1(M3)))

[0053] M4 is the fourth-level feature, where M3 is the third-level feature;

[0054] M5=Relu(WeightNorm(Conv2(M4)))

[0055] M5 is the fifth-level feature, where M4 is the fourth-level feature;

[0056] M6=FC(M5)

[0057] M6 is the sixth-level feature, where M5 is the fifth-level feature;

[0058] S63. As the network depth increases, the TDCN network captures longer functional connection time dependencies. In order to maintain the stability of the network, branch 3 is added. If the channel dimension of the input sequence is Channel In Channel dimension of the output sequence Out If they are inconsistent, a one-dimensional convolution Conv3 with a convolution kernel size of 1 is used for dimension transformation; if the two are consistent, the convolution operation is not selected;

[0059]

[0060] M7 is the seventh level feature, where η n represents the extracted adaptive high-order hyperedge time series;

[0061] S64, add the outputs of branch 1, branch 2, and branch 3, and then use ReLU to obtain the final residual block output F Output :

[0062] F Output=Relu(M2+M6+M7)

[0063] F Output is the final local time feature, where M2 is the second-level feature, M6 is the sixth-level feature, and M7 is the seventh-level feature;

[0064] In each subject, the local temporal feature F of each adaptive high-order hyperedge is calculated. Output The local temporal features of all adaptive high-order hyperedges constitute the local temporal features LT of the corresponding subjects. n .

[0065] Preferably, in step S7, a graph attention network is used to aggregate global temporal features and local temporal features to obtain high-order spatiotemporal aggregate features, specifically including the following:

[0066] Firstly, based on the adaptive high-order hyperedge time series, the sparse group lasso algorithm is used to estimate the correlation between the adaptive high-order hyperedge time series to characterize the high-order spatial characteristics of the brain network and use it as prior information. Then, the local temporal features F are concatenated. Output and global temporal features is the fusion feature vector Then the attention mechanism is used to calculate the attention coefficient of each dimension of the fused feature. Finally, according to the attention coefficient, it is linearly combined with the fused feature to generate a high-order spatiotemporal aggregation feature.

[0067] Preferably, the adaptive high-order hyperedge time series are first grouped using a clustering algorithm, and then the sparse group lasso algorithm is used to estimate the correlation between the adaptive high-order hyperedge time series. The specific formula is:

[0068]

[0069] in represents the weight vector, i.e., the connection weight between the lth adaptive high-order hyperedge and other adaptive high-order hyperedges in the nth subject, where express The weight vector in the gth array in ; η and μ are regularization parameters; represents the lth adaptive high-order hyperedge time series; ||.| |2 is the l2 norm, ||.||1 is the l1 norm;

[0070] Calculate the high-order connection weights Afterwards, the Xavier initialization method is used to initialize the feedforward neural network and the attention coefficient matrix Perform random initialization; first, fuse the feature vector Multiply it with the attention coefficient matrix AT, and then pass it through the LeakyRelu function Multiply them and update the attention coefficient AT through the softmax function; the attention coefficient reflects the similarity between different adaptive high-order hyperedge time series. The stronger the high-order connectivity, the larger the attention coefficient. The Adam optimizer is used to iteratively update the feedforward neural network in the whole process. Finally, based on Multiply and sum with the median of the lth column AT respectively and generate through the LeakyRelu function And use it as a high-order spatiotemporal aggregation feature. The specific formula is:

[0071]

[0072] represents the attention coefficient of the i-th row and l-th column in the attention coefficient matrix AT, is the fusion feature vector, It is a high-order spatiotemporal aggregation feature.

[0073] The beneficial effects of the present invention are as follows: the magnetic resonance image classification method based on adaptive high-order hyperedge time series multi-feature fusion described in the present invention obtains a high-order adaptive high-order hyperedge time series through a self-attention mechanism and a Kalman filter algorithm, and uses statistical indicators and a time-expanded convolutional network to obtain classification features and perform feature aggregation, thereby improving the classification accuracy. The present invention solves the problem of the traditional brain function hypernetwork model ignoring the time dependency between different brain regions within the hyperedge and the problem of stray fluctuations caused by the traditional sliding time window method to characterize the time-varying nature of the resting brain function hypernetwork, so as to more accurately characterize the spatiotemporal characteristics of the human brain and improve the classification performance of brain diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 It is a flow chart of a magnetic resonance image classification method based on adaptive high-order hyperedge time series multi-feature fusion in an embodiment of the present invention;

[0075] Figure 2 Schematic diagram of the optimized time dilation convolutional network model structure in an embodiment of the present invention;

[0076] Figure 3 It is a schematic diagram for comparing the method of the present invention with the traditional magnetic resonance brain function supernetwork classification method. DETAILED DESCRIPTION

[0077] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0078] See also Figure 1 The present invention provides a technical solution: a magnetic resonance image classification method based on adaptive high-order hyperedge time series multi-feature fusion, comprising the following steps:

[0079] S1. Acquire resting-state functional magnetic resonance imaging data and perform preprocessing to obtain preprocessed imaging data; perform brain region segmentation on the preprocessed imaging data according to a selected standardized brain atlas, and extract the average time series of each brain region.

[0080] Furthermore, the preprocessing steps specifically include time layer correction, head movement correction, space standardization, and smoothing; the standardized brain map is an internationally used automatic anatomical labeling (AAL) template, which divides the brain into 90 regions, including 45 regions in each of the left and right hemispheres; therefore, the brain region segmentation described in the present invention is to divide the image into 90 regions, including 45 regions in each of the left and right hemispheres.

[0081] Furthermore, the extracting of the average time series of each brain region specifically includes: extracting activation signals of all voxels contained in each brain region at different time points, and then arithmetically averaging the activation signals of each voxel at different time points to obtain the average time series of the brain region.

[0082] S2. Based on the average time series of each brain region, a brain functional hypernetwork model was constructed using the weighted minimum shrinkage and selection operator (LASSO) method.

[0083] Specifically, based on the average time series of brain regions, the weighted minimum shrinkage operator LASSO method is used to construct a brain function supernetwork, which is specifically defined as:

[0084]

[0085] Among them, w represents the w-th brain region after atlas division, w∈{1,2,...,W}, W represents the number of divided brain regions; represents the time series of the wth brain region in the nth subject; represents the time series of the remaining brain regions except the wth brain region in the nth subject, is a weight vector, which indicates the influence of the remaining brain regions in the nth subject on the wth brain region; λ1 is a penalty parameter, λ1>0;

[0086] Then, for each subject n among the N subjects, λ1 is fixed and the weight vector corresponding to each brain region is calculated respectively. The non-zero weight vectors form a hyperedge of the corresponding brain region, and all hyperedges form a brain function supernetwork.

[0087] S3. Based on the brain function hypernetwork model, the self-attention mechanism is used to extract the hyperedge time series of the subjects. First, based on the brain function hypernetwork of each subject, the time series of all brain region nodes in each hyperedge of the brain function hypernetwork are extracted. represents the time series of the oth brain region under the nth subject, o, p, q represent the oth brain region node, the pth brain region node and the qth brain region node in the brain function hypernetwork, e represents the eth hyperedge in the brain function hypernetwork, and H represents the number of hyperedges; for example, the first hyperedge of the nth subject contains the three brain region nodes numbered 2, 24, and 89, then the node time series corresponding to the first hyperedge of the nth subject is The node time series corresponding to the remaining hyperedges are deduced in the same way.

[0088] Based on the brain function hypernetwork of each subject, the time series of all brain region nodes in each hyperedge are used as the input vector of the self-attention module. The dot product similarity between the query matrix Q and the key matrix K in the three trainable matrices is calculated to obtain the attention distribution of the value matrix V. Finally, the output attention weight vector at each different time point is:

[0089]

[0090] where d k is the dimension of the key-value vector, softmax normalizes the similarity scores row by row, and then the self-attention module assigns each softmax score to the corresponding value vector v t (t)(t=1,2,…,T), t represents the tth time point, T represents all time points in the time series, and finally calculates the weighted sum of different time points (for example ), and obtain the hyperedge time series

[0091] S4. Based on the hyperedge time series, the Kalman filter algorithm is used to extract the subject's adaptive high-order hyperedge time series.

[0092] The Kalman filter algorithm is used to extract the subject's adaptive high-order hyperedge time series, as follows:

[0093]

[0094] in, represents the kth hyperedge time series of the nth subject, represents the value at the xth moment in the kth hyperedge time series of the nth subject, represents the functional connection strength between the kth hyperedge and the lth hyperedge at time t, and E(·) represents the mathematical expectation;

[0095] After removing weak connections through Kalman filtering, each subject finally obtains an adaptive high-order hyperedge time series. in represents the lth adaptive high-order hyperedge time series; represents the value of the tth time point in the lth adaptive high-order hyperedge time series, [·] T Represents the transpose of this vector.

[0096] S5. Calculate the standard deviation, root mean square, skewness and kurtosis based on the adaptive high-order hyperedge time series, and use these four statistical indicators as global time features.

[0097] Furthermore, the specific calculation formula of the standard deviation is as follows:

[0098]

[0099] in represents the standard deviation of the lth adaptive high-order hyperedge time series, represents the value of the tth time point in the lth adaptive high-order hyperedge time series, T represents the total number of time points in the time series, express , n is the nth subject;

[0100] The specific calculation formula of the root mean square is as follows:

[0101]

[0102] in represents the root mean square of the lth adaptive high-order hyperedge time series, represents the value of the t-th time point in the l-th adaptive high-order hyperedge time series, T represents the total number of time points, and n represents the n-th subject;

[0103] The specific calculation formula of the skewness is as follows:

[0104]

[0105] in represents the skewness of the lth adaptive high-order hyperedge time series, represents the value of the tth time point in the lth adaptive high-order hyperedge time series, T represents the total number of time points, express The mean of represents the standard deviation of the lth adaptive high-order hyperedge time series, and n is the nth subject;

[0106] The specific calculation formula of the kurtosis is as follows:

[0107]

[0108] in represents the kurtosis of the lth adaptive high-order hyperedge time series, represents the value of the tth time point in the lth adaptive high-order hyperedge time series, T represents the total number of time points, express The mean of represents the standard deviation of the lth adaptive high-order hyperedge time series, and n is the nth subject.

[0109] In the nth subject, calculate each adaptive high-order hyperedge Finally, the global time characteristics of the nth subject are obtained in [·] T Represents the transpose of this vector.

[0110] S6. Based on the adaptive high-order hyperedge time series, the optimized time dilation causal convolution network (TDCN) is used to extract local time features.

[0111] In step S6, based on the existing temporal convolutional network, a local temporal characteristic characterization module is innovatively introduced. By adding branch 2, a new optimized time dilation causal convolution network (TDCN) is constructed to adjust the receptive field of the convolution kernel by changing the dilation coefficient, thereby mining local temporal features more accurately. Figure 2 As shown, the optimized time dilation convolutional network TDCN includes branch 1, branch 2 and branch 3; branch 1 uses dilated causal convolution to extract causal time characteristics from the sliding window; branch 2 aggregates the time axis information of the sequence to obtain a complete local time feature representation; branch 3 uses 1D convolution to adjust the output dimension to extract dependencies on a longer time scale and maintain the stability of the network;

[0112] The adaptive high-order hyperedge time series η n As the input sequence, the optimized time dilation convolutional network TDCN is used to extract local time features and obtain LT n , the specific steps are as follows:

[0113] S61, for the adaptive high-order hyperedge time series η n , branch 1 uses two dilated convolution layers (DCC) and weight normalization (WeighNorm) and rectified linear unit (Relu) strategies to extract features;

[0114] The first dilated convolutional layer DCC1 extracts the first-level features M1, and the second dilated convolutional layer DCC2 extracts the second-level features M2;

[0115] M1=Relu(WeightNorm(DCC1(η n )))

[0116] M1 is the first-level feature, where η n represents an adaptive high-order hyperedge time series;

[0117] M2=Relu(WeightNorm(DCC2(M1)))

[0118] M2 is the second-level feature, where M1 is the first-level feature;

[0119] S62, branch 2 uses a convolution layer with a dilation factor of 1. First, the adaptive high-order hyperedge time series information is aggregated through adaptive averaging pooling (Avg) to obtain M3, and then through weight normalization and activation function Relu, M4 is obtained. Then, a convolution layer with a convolution kernel size of 1, weight normalization and Relu is obtained to obtain M5. Finally, the time dimension of feature M5 is expanded to be equal to η through the fully connected layer FC. n The time dimension is consistent, and we get M6;

[0120] M3=AdaptiveAvgPool(η n )

[0121] M3 is the third-level feature, where η n represents an adaptive high-order hyperedge time series;

[0122] M4=Relu(WeightNorm(Conv1(M3)))

[0123] M4 is the fourth-level feature, where M3 is the third-level feature;

[0124] M5=Relu(WeightNorm(Conv2(M4)))

[0125] M5 is the fifth-level feature, where M4 is the fourth-level feature;

[0126] M6=FC(M5)

[0127] M6 is the sixth-level feature, where M5 is the fifth-level feature;

[0128] S63. As the network depth increases, the TDCN network captures longer functional connection time dependencies. In order to maintain the stability of the network, branch 3 is added. If the channel dimension of the input sequence is Channel In Channel dimension of the output sequence Out If they are inconsistent, a one-dimensional convolution Conv3 with a convolution kernel size of 1 is used for dimension transformation; if the two are consistent, the convolution operation is not selected;

[0129]

[0130] M7 is the seventh level feature, where η n represents the extracted adaptive high-order hyperedge time series;

[0131] S64, add the outputs of branch 1, branch 2, and branch 3, and then use ReLU to obtain the final residual block output F Output :

[0132] F Output =Relu(M2+M6+M7)

[0133] F Output is the final local time feature, where M2 is the second-level feature, M6 is the sixth-level feature, and M7 is the seventh-level feature.

[0134] In each subject, the local temporal feature F of each adaptive high-order hyperedge is calculated. Output The local temporal features of all adaptive high-order hyperedges constitute the local temporal features LT of the corresponding subjects. n .

[0135] S7. Based on the topological structure information between adaptive high-order hyperedge time series, the graph attention network is used to aggregate global time features and local time features to obtain high-order spatiotemporal aggregation features. Based on the high-order spatiotemporal aggregation features, the fully connected layer classification network is used to classify the magnetic resonance imaging data of the subjects.

[0136] Furthermore, the graph attention network is used to aggregate global temporal features and local temporal features to obtain high-order spatiotemporal aggregation features, including the following:

[0137] Firstly, based on the adaptive high-order hyperedge time series, the sparse group lasso algorithm is used to estimate the correlation between the adaptive high-order hyperedge time series to characterize the high-order spatial characteristics of the brain network and use it as prior information. Then, the local temporal features F are concatenated.Output and global temporal features is the fusion feature vector Then the attention mechanism is used to calculate the attention coefficient of each dimension of the fused feature. Finally, according to the attention coefficient, it is linearly combined with the fused feature to generate a high-order spatiotemporal aggregation feature.

[0138] Further, the details are as follows: first, the adaptive high-order hyperedge time series are grouped using a clustering algorithm, and then the sparse group lasso algorithm is used to estimate the correlation between the adaptive high-order hyperedge time series. The specific formula is:

[0139]

[0140] in represents the weight vector, i.e., the connection weight between the lth adaptive high-order hyperedge and other adaptive high-order hyperedges in the nth subject, where express The weight vector in the gth array in ; η and μ are regularization parameters; represents the lth adaptive high-order hyperedge time series; ||.| |2 is the l2 norm, ||.||1 is the l1 norm;

[0141] Calculate the high-order connection weights Afterwards, the Xavier initialization method is used to initialize the feedforward neural network and the attention coefficient matrix Perform random initialization; first, fuse the feature vector Multiply it with the attention coefficient matrix AT, and then pass it through the LeakyRelu function Multiply them and update the attention coefficient AT through the softmax function; the attention coefficient reflects the similarity between different adaptive high-order hyperedge time series. The stronger the high-order connectivity, the larger the attention coefficient. The Adam optimizer is used to iteratively update the feedforward neural network in the whole process. Finally, based on Multiply and sum with the median of the lth column AT respectively and generate through the LeakyRelu function And use it as a high-order spatiotemporal aggregation feature. The specific formula is:

[0142]

[0143] represents the attention coefficient of the i-th row and l-th column in the attention coefficient matrix AT, is the fusion feature vector, It is a high-order spatiotemporal aggregation feature.

[0144] like Figure 3The figure is a schematic diagram comparing the method proposed by the invention with the traditional magnetic resonance brain function super network classification method. Figure 3 It can be seen that the classification accuracy of the present invention is significantly higher than that of the traditional magnetic resonance imaging data classification method.

[0145] Taking into account the problem of spurious fluctuations caused by the traditional brain function hypernetwork model ignoring the temporal dependence between different brain regions within the hyperedge and the use of a sliding time window method to characterize the time-varying nature of the resting-state brain function hypernetwork, the present invention introduces an attention mechanism and a Kalman filter algorithm to extract the subjects' adaptive high-order hyperedge time series to more accurately characterize the spatiotemporal characteristics of the human brain. Then, statistical indicators and an expanded time convolutional network are used to comprehensively extract the global and local temporal features of the adaptive high-order hyperedge time series, and a graph attention network is used to aggregate features, thereby further accurately extracting high-order spatiotemporal features of the human brain and improving classification performance.

[0146] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0147] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0148] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0149] The "first\second" mentioned in the embodiments is only to distinguish similar objects, and does not represent a specific order for the objects. It is understandable that the "first\second" can be interchanged with the specific order or sequence where permitted. It should be understood that the objects distinguished by "first\second" can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0150] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A magnetic resonance image classification method based on adaptive high-order hyperedge time series multi-feature fusion, characterized in that: The steps include: S1. Acquire resting-state functional magnetic resonance imaging data and perform preprocessing to obtain preprocessed imaging data; perform brain region segmentation on the preprocessed imaging data according to a selected standardized brain atlas, and extract the average time series of each brain region; S2. Based on the average time series of each brain region, the weighted minimum shrinkage operator LASSO method was used to construct a brain function hypernetwork model; S3, based on the brain function hypernetwork model, using the self-attention mechanism to extract the hyperedge time series of the subjects; S4, based on the hyperedge time series, the Kalman filter algorithm is used to extract the subject's adaptive high-order hyperedge time series; S5. Calculate the standard deviation, root mean square, skewness and kurtosis based on the adaptive high-order hyperedge time series, and use these four statistical indicators as global time features; S6, based on adaptive high-order hyperedge time series, using the optimized time dilation convolutional network TDCN to extract local time features; S7. Use the graph attention network to aggregate global temporal features and local temporal features to obtain high-order spatiotemporal aggregation features. Based on the high-order spatiotemporal aggregation features, use the fully connected layer classification network to classify the subject's magnetic resonance imaging data.

2. The magnetic resonance image classification method based on adaptive high-order hyperedge time series multi-feature fusion according to claim 1 is characterized by: In step S1, the preprocessing steps specifically include time layer correction, head motion correction, space standardization, and smoothing; the standardized brain atlas is an internationally used automatic anatomical labeling template; the brain region segmentation divides the image into 90 regions, including 45 regions in each of the left and right hemispheres.

3. The magnetic resonance image classification method based on adaptive high-order hyperedge time series multi-feature fusion according to claim 1 is characterized by: In step S1, the extracting of the average time series of each brain region specifically includes: extracting the activation signals of all voxels contained in each brain region at different time points, and then arithmetically averaging the activation signals of each voxel at different time points to obtain the average time series of the brain region.

4. The magnetic resonance image classification method based on adaptive high-order hyperedge time series multi-feature fusion according to claim 1, characterized in that: In step S2, it specifically includes: Firstly, based on the average time series of brain regions, the weighted minimum shrinkage operator LASSO method was used to construct the brain function hypernetwork, which is specifically defined as: Among them, w represents the w-th brain region after atlas division, w∈{1,2,...,W}, W represents the number of divided brain regions; represents the time series of the wth brain region in the nth subject; represents the time series of the remaining brain regions except the wth brain region in the nth subject, is a weight vector, which indicates the influence of the remaining brain regions in the nth subject on the wth brain region; λ1 is a penalty parameter, λ1>0; Then, for each subject n among the N subjects, λ1 is fixed and the weight vector corresponding to each brain region is calculated respectively. The non-zero weight vectors form a hyperedge of the corresponding brain region, and all hyperedges form a brain function supernetwork.

5. The magnetic resonance image classification method based on adaptive high-order hyperedge time series multi-feature fusion according to claim 1, characterized in that: In step S3, firstly, based on the brain function hypernetwork of each subject, the time series of all brain region nodes in each hyperedge of the brain function hypernetwork are extracted. represents the time series of the oth brain region under the nth subject, o, p, q represent the oth brain region node, the pth brain region node and the qth brain region node in the brain function supernetwork, e represents the eth hyperedge in the brain function supernetwork, and H represents the number of hyperedges; Based on the brain function hypernetwork of each subject, the time series of all brain region nodes in each hyperedge are used as the input vector of the self-attention module. The dot product similarity between the query matrix Q and the key matrix K in the three trainable matrices is calculated to obtain the attention distribution of the value matrix V. Finally, the output attention weight vector at each different time point is: where d k is the dimension of the key-value vector, softmax normalizes the similarity scores row by row, and then the self-attention module assigns each softmax score to the corresponding value vector v t (t)(t=1,2,…,T), t represents the tth time point, T represents all time points in the time series, and finally the weighted sum of different time points is calculated to obtain the hyperedge time series 6. The magnetic resonance image classification method based on adaptive high-order hyperedge time series multi-feature fusion according to claim 1, characterized in that: In step S4, the Kalman filter algorithm is used to extract the subject's adaptive high-order hyperedge time series, as follows: in, represents the kth hyperedge time series of the nth subject, represents the value at the xth moment in the kth hyperedge time series of the nth subject, represents the functional connection strength between the kth hyperedge and the lth hyperedge at time t, and E(·) represents the mathematical expectation; After removing weak connections through Kalman filtering, each subject finally obtains an adaptive high-order hyperedge time series. in represents the lth adaptive high-order hyperedge time series; represents the value of the tth time point in the lth adaptive high-order hyperedge time series, [·] T Represents the transpose of this vector.

7. The magnetic resonance image classification method based on adaptive high-order hyperedge time series multi-feature fusion according to claim 1 is characterized by: In step S5, the specific calculation formula of the standard deviation is as follows: in represents the standard deviation of the lth adaptive high-order hyperedge time series, represents the value of the tth time point in the lth adaptive high-order hyperedge time series, T represents the total number of time points in the time series, express , n is the nth subject; The specific calculation formula of the root mean square is as follows: in represents the root mean square of the lth adaptive high-order hyperedge time series, represents the value of the t-th time point in the l-th adaptive high-order hyperedge time series, T represents the total number of time points, and n represents the n-th subject; The specific calculation formula of the skewness is as follows: in represents the skewness of the lth adaptive high-order hyperedge time series, represents the value of the tth time point in the lth adaptive high-order hyperedge time series, T represents the total number of time points, express The mean of represents the standard deviation of the lth adaptive high-order hyperedge time series, and n is the nth subject; The specific calculation formula of the kurtosis is as follows: in represents the kurtosis of the lth adaptive high-order hyperedge time series, represents the value of the tth time point in the lth adaptive high-order hyperedge time series, T represents the total number of time points, express The mean of represents the standard deviation of the lth adaptive high-order hyperedge time series, and n is the nth subject; In the nth subject, calculate each adaptive high-order hyperedge Finally, the global time characteristics of the nth subject are obtained in [·] T Represents the transpose of this vector.

8. The magnetic resonance image classification method based on adaptive high-order hyperedge time series multi-feature fusion according to claim 1, characterized in that: In step S6, the optimized time dilation convolutional network TDCN includes branch 1, branch 2 and branch 3; branch 1 uses dilated causal convolution to extract causal time characteristics from the sliding window; branch 2 aggregates the time axis information of the sequence to obtain a complete local time feature representation; branch 3 uses 1D convolution to adjust the output dimension to extract dependencies on a longer time scale and maintain the stability of the network; The adaptive high-order hyperedge time series η n As the input sequence, the optimized time dilation convolutional network TDCN is used to extract local time features and obtain LT n , the specific steps are as follows: S61, for the adaptive high-order hyperedge time series η n , Branch 1 uses two dilated convolutional layers as well as weight normalization and linear rectification activation function strategies to extract features; The first dilated convolutional layer DCC1 extracts the first-level features M1, and the second dilated convolutional layer DCC2 extracts the second-level features M2; M1=Relu(WeightNorm(DCC1(η n ))) M1 is the first-level feature, where η n represents an adaptive high-order hyperedge time series; M2=Relu(WeightNorm(DCC2(M1))) M2 is the second-level feature, where M1 is the first-level feature; S62, branch 2 uses a convolution layer with a dilation factor of 1. First, the adaptive high-order hyperedge time series information is aggregated through adaptive average pooling to obtain M3, and then through weight normalization and activation function Relu, M4 is obtained. Then, through a convolution layer with a convolution kernel size of 1, weight normalization and Relu, M5 is obtained. Finally, the time dimension of feature M5 is expanded to be equal to η through the fully connected layer FC. n The time dimension is consistent, and we get M6; M3=AdaptiveAvgPool(η n ) M3 is the third-level feature, where η n represents an adaptive high-order hyperedge time series; M4=Relu(WeightNorm(Conv1(M3))) M4 is the fourth-level feature, where M3 is the third-level feature; M5=Relu(WeightNorm(Conv2(M4))) M5 is the fifth-level feature, where M4 is the fourth-level feature; M6=FC(M5) M6 is the sixth-level feature, where M5 is the fifth-level feature; S63. As the network depth increases, the TDCN network captures longer functional connection time dependencies. In order to maintain the stability of the network, branch 3 is added. If the channel dimension of the input sequence is Channel In Channel dimension of the output sequence Out If they are inconsistent, a one-dimensional convolution Conv3 with a convolution kernel size of 1 is used for dimension transformation; if the two are consistent, the convolution operation is not selected; M7 is the seventh level feature, where η n represents the extracted adaptive high-order hyperedge time series; S64, add the outputs of branch 1, branch 2, and branch 3, and then use ReLU to obtain the final residual block output F Output : <h2 style=";text-align:left;direction:ltr">F<h2 style=";text-align:left;direction:ltr"> Output <h2 style=";text-align:left;direction:ltr"> Relu(M2+M6+M7) F Output is the final local time feature, where M2 is the second-level feature, M6 is the sixth-level feature, and M7 is the seventh-level feature; In each subject, the local temporal feature F of each adaptive high-order hyperedge is calculated. Output The local temporal features of all adaptive high-order hyperedges constitute the local temporal features LT of the corresponding subjects. n .

9. The magnetic resonance image classification method based on adaptive high-order hyperedge time series multi-feature fusion according to claim 1, characterized in that: In step S7, the graph attention network is used to aggregate the global temporal features and the local temporal features to obtain high-order spatiotemporal aggregate features, which specifically include the following: Firstly, based on the adaptive high-order hyperedge time series, the sparse group lasso algorithm is used to estimate the correlation between the adaptive high-order hyperedge time series to characterize the high-order spatial characteristics of the brain network and use it as prior information. Then, the local temporal features F are concatenated. Output and the global temporal feature GT l n is the fusion feature vector Then the attention mechanism is used to calculate the attention coefficient of each dimension of the fused feature. Finally, according to the attention coefficient, it is linearly combined with the fused feature to generate a high-order spatiotemporal aggregation feature.

10. The magnetic resonance image classification method based on adaptive high-order hyperedge time series multi-feature fusion according to claim 9, characterized in that: First, the clustering algorithm is used to group the adaptive high-order hyperedge time series, and then the sparse group lasso algorithm is used to estimate the correlation between the adaptive high-order hyperedge time series. The specific formula is: in represents the weight vector, i.e., the connection weight between the lth adaptive high-order hyperedge and other adaptive high-order hyperedges in the nth subject, where express The weight vector in the gth array in ; η and μ are regularization parameters; represents the lth adaptive high-order hyperedge time series; ||.| |2 is the l2 norm, ||.||1 is the l1 norm; Calculate the high-order connection weights Afterwards, the Xavier initialization method is used to initialize the feedforward neural network and the attention coefficient matrix Perform random initialization; first, fuse the feature vector Multiply it with the attention coefficient matrix AT, and then pass it through the LeakyRelu function Multiply them and update the attention coefficient matrix AT through the softmax function; the attention coefficient reflects the similarity between different adaptive high-order hyperedge time series. The stronger the high-order connectivity, the larger the attention coefficient. The Adam optimizer is used to iteratively update the feedforward neural network in the whole process; finally, based on Multiply and sum with the median of the lth column AT respectively and generate through the LeakyRelu function And use it as a high-order spatiotemporal aggregation feature. The specific formula is: represents the attention coefficient of the i-th row and l-th column in the attention coefficient matrix AT, is the fusion feature vector, It is a high-order spatiotemporal aggregation feature.

Images