Multi-view-angle-based fMRI classification analysis method and device

Abstract

The invention provides a multi-view-angle-based fMRI classification analysis method and device, and relates to the technical field of medical instruments. Comprising the steps that a multi-view fMRI classification model and an interpretability analysis module are constructed, and the fMRI classification model comprises a dynamic and static cross attention encoder and a time attention encoder; inputting the medical image to be classified into an fMRI classification model trained by using the preprocessed multi-view fMRI sample set, and outputting a classification result; and inputting the classification result, the multi-view fMRI sample set and the behavior data into an interpretability analysis module, and outputting the multi-view fMRI sample set and disease-related significant brain regions, significant connections, and the correlation between the decision result of the fMRI classification model and the difference between categories and the correlation between the decision result of the fMRI classification model and the behavior data. Complex interrelations can be completely captured, and the interpretability of a multi-view fMRI classification model is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical devices, and in particular to a multi-view functional magnetic resonance imaging (fMRI) classification and analysis method and device. Background Art

[0002] With the advancement of science and technology, medical imaging technology has provided important imaging information for the diagnosis and treatment of diseases, greatly assisting clinical decision-making. Among them, functional magnetic resonance imaging (fMRI) is an important neuroimaging technology for understanding the structure and function of the brain in healthy and diseased individuals, and is widely used in neuroscience, clinical diagnosis, and treatment planning.

[0003] fMRI reflects brain activity by detecting changes in the blood oxygen level dependent (BOLD) signal. The intensity of the BOLD signal over time is represented as a time series (TS), which reflects the time-varying neural activity pattern of a specific brain region and is an important basis for studying the dynamic characteristics of brain function. Static functional connectivity (sFC) focuses on the common activation patterns and connection strengths between different brain regions by calculating the Pearson correlation between TS series of different brain regions, providing valuable insights into the organizational structure of brain networks. Dynamic functional connectivity (dFC) can capture changing brain connectivity patterns, thereby revealing the evolution of interactions between brain regions on a shorter time scale. Measurements from all three perspectives are widely used in the diagnosis and explanation of diseases. Therefore, the classification and analysis of multi-perspective fMRI is particularly important in the medical field.

[0004] Currently, fMRI classification models typically employ deep learning methods. However, most of these methods use late-stage fusion or utilize only one or two fMRI views as input, which prevents the model from fully capturing complex interrelationships. Furthermore, existing techniques simply extract salient features without further analysis, resulting in insufficient interpretability of the model results. Summary of the Invention

[0005] The purpose of the embodiments of the present invention is to provide a multi-perspective fMRI classification analysis method and apparatus to solve the problems that the existing technology may not be able to fully capture complex interrelationships; and the model results are insufficiently interpretable.

[0006] To solve the above technical problems, the embodiments of the present invention provide the following technical solutions:

[0007] A first aspect of the present invention provides an fMRI classification and analysis method based on multiple perspectives, the fMRI classification and analysis method based on multiple perspectives comprising:

[0008] Obtain fMRI sample sets and behavioral data;

[0009] Preprocess the fMRI sample set to obtain a multi-view fMRI sample set;

[0010] Construct a multi-perspective fMRI classification model and an interpretability analysis module. The fMRI classification model includes a dynamic-static cross-attention encoder and a temporal attention encoder. The interpretability analysis module includes a decision statistics difference algorithm submodule and a decision behavior-related algorithm submodule.

[0011] The fMRI classification model is trained using a multi-view fMRI sample set to obtain a trained fMRI classification model;

[0012] Inputting the medical image to be classified into the trained fMRI classification model so that the trained fMRI classification model outputs a classification result;

[0013] The classification results, multi-view fMRI sample sets and behavioral data are input into the interpretability analysis module, so that the interpretability analysis module outputs the multi-view fMRI sample sets and the significant brain regions and significant connections related to the disease, as well as the correlation between the decision results of the fMRI classification model and the differences between categories and the correlation with the behavioral data.

[0014] A second aspect of the present invention provides an fMRI classification and analysis device based on multiple perspectives, the fMRI classification and analysis device based on multiple perspectives comprising:

[0015] Acquisition module, used to acquire fMRI sample sets and behavioral data;

[0016] A preprocessing module is used to preprocess the fMRI sample set to obtain a multi-view fMRI sample set;

[0017] A construction module is used to construct a multi-view fMRI classification model and an interpretability analysis module. The fMRI classification model includes a dynamic-static cross attention encoder and a temporal attention encoder. The interpretability analysis module includes a decision statistics difference algorithm submodule and a decision behavior related algorithm submodule.

[0018] A training module is used to train the fMRI classification model using a multi-view fMRI sample set to obtain a trained fMRI classification model;

[0019] A classification module is used to input the medical image to be classified into the trained fMRI classification model so that the trained fMRI classification model outputs a classification result;

[0020] The correlation analysis module is used to input the classification results, multi-view fMRI sample sets and behavioral data into the interpretability analysis module, so that the interpretability analysis module outputs the significant brain regions and significant connections related to the disease in the multi-view fMRI sample sets, as well as the correlation between the decision results of the fMRI classification model and the differences between categories and the correlation with the behavioral data.

[0021] Compared with the prior art, the present invention provides a multi-perspective fMRI classification and analysis method and device, which obtains an fMRI sample set and behavioral data; preprocesses the fMRI sample set to obtain a multi-perspective fMRI sample set; constructs a multi-perspective fMRI classification model and an interpretability analysis module, wherein the fMRI classification model includes a dynamic-static cross-attention encoder and a temporal attention encoder, and the interpretability analysis module includes a decision statistics difference algorithm submodule and a decision behavior correlation algorithm submodule; uses a training set of the multi-perspective fMRI sample set to train the fMRI classification model to obtain a trained fMRI classification model; inputs the medical image to be classified into the trained fMRI classification model so that the trained fMRI classification model outputs a classification result; inputs the classification result, the multi-perspective fMRI sample set and the behavioral data into the interpretability analysis module so that the interpretability analysis module outputs the multi-perspective fMRI sample set and the significant brain regions and significant connections related to the disease, as well as the correlation between the decision results of the fMRI classification model and the differences between categories and the correlation with the behavioral data. In this way, through the dynamic-static cross-attention encoder and the temporal attention encoder, information interaction between dynamic functional connections and static functional connections can be carried out, and the correlation between them can be calculated, so that the model can fully capture complex relationships; through the interpretability analysis module, the correlation between significant brain areas, significant connections and the decision results of the fMRI classification model and the differences between categories and the correlation with behavioral data can be determined, so that the model can improve the interpretability of multi-perspective fMRI classification models. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0023] Figure 1 The flowchart of the multi-view fMRI classification analysis method is schematically shown;

[0024] Figure 2 The schematic diagram of the overall model structure of fMRI classification analysis based on multiple perspectives is shown;

[0025] Figure 3 Schematic diagram of the dynamic and static cross attention encoder and the temporal attention encoder;

[0026] Figure 4 The schematic diagram of the explainability analysis module is shown schematically;

[0027] Figure 5 Schematic diagrams of the decision statistics difference algorithm submodule and the decision behavior related algorithm submodule are shown;

[0028] Figure 6 The structure of the fMRI classification and analysis device based on multiple perspectives is schematically shown. DETAILED DESCRIPTION

[0029] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0030] It should be noted that, unless otherwise specified, the technical or scientific terms used in the present invention should have the common meanings understood by those skilled in the art to which the present invention belongs.

[0031] The method in the embodiment of the present invention is described in detail below.

[0032] Figure 1 The flowchart of the multi-view fMRI classification analysis method in the embodiment of the present invention is schematically shown. Figure 1 As shown, the multi-view fMRI classification analysis method may include:

[0033] S101. Obtain fMRI sample sets and behavioral data.

[0034] The fMRI sample set is a sample set containing fMRI medical images.

[0035] S102 : Preprocess the fMRI sample set to obtain a multi-view fMRI sample set.

[0036] The format of the fMRI sample set is the Digital Imaging and Communications in Medicine (DICOM) standard format.

[0037] Specifically, Figure 2 The schematic diagram of the overall model structure of fMRI classification analysis based on multiple perspectives is shown in Figure 2 As shown, the fMRI sample set is preprocessed to obtain a multi-view fMRI sample set, including:

[0038] Step A1: Convert the digital medical imaging and communication standard format into the Neuroimaging Informatics Technology Initiative (NII) format to construct an fMRI sample set from a time series perspective.

[0039] Among them, the fMRI sample set from the time series perspective consists of image sequences reflecting the blood oxygen level dependent effect (BOLD) at multiple time points.

[0040] Convert the DICOM format to the nii format to obtain an fMRI sample set from a time series perspective.

[0041] Step A2: Delete the first five time points from the plurality of time points to obtain the first fMRI data.

[0042] To ensure magnetization equilibrium, the first five time points among the multiple time points were discarded.

[0043] Step A3: performing time correction and head motion correction on the first fMRI data to obtain corrected fMRI data.

[0044] Step A4: The corrected fMRI data are sequentially registered and confounding factors are eliminated to obtain confounding-free fMRI data.

[0045] The corrected fMRI data are registered to a standard template and confounding factors such as head motion parameters, white matter signals and cerebrospinal fluid signals are eliminated to obtain confounded fMRI data.

[0046] Step A5: performing band-pass filtering and Gaussian kernel smoothing filtering on the fMRI data after eliminating contamination, to obtain filtered fMRI data.

[0047] The filtered fMRI data includes data from multiple brain regions.

[0048] Bandpass filtering can be performed using a bandpass filter, or a 6mm Gaussian kernel can be used for Gaussian kernel smoothing filtering.

[0049] Step A6: Calculate the Pearson correlation between multiple brain regions to obtain an fMRI sample set from a static functional connectivity perspective.

[0050] Step A7: For the filtered fMRI data, a sliding window method is used to determine the fMRI sample set of the dynamic functional connectivity perspective to obtain a multi-perspective fMRI sample set.

[0051] S103. Construct a multi-perspective fMRI classification model and interpretability analysis module.

[0052] in, Figure 2 The schematic diagram of the overall model structure of fMRI classification analysis based on multiple perspectives is shown in Figure 2 As shown in the figure, the fMRI classification model includes a static-dynamic cross attention (SDCA) encoder and a temporal attention (TA) encoder, and the interpretability analysis module includes a decision statistical difference algorithm submodule (DiA) and a decision behavior related algorithm submodule (Physiological Relevance, PR).

[0053] Specifically, the fMRI classification model also includes a static functional connection embedding layer, a dynamic functional connection embedding layer, a temporal embedding layer, a Token extraction fusion layer and a classification result output layer. The static functional connection embedding layer, the static-dynamic cross attention encoder, the token extraction fusion layer and the classification result output layer are connected in sequence. The dynamic functional connection embedding layer, the static-dynamic cross attention encoder, the token extraction fusion layer and the classification result output layer are connected in sequence. The temporal embedding layer, the temporal attention encoder, the token extraction fusion layer and the classification result output layer are connected in sequence. The static functional connection embedding layer and the dynamic functional connection embedding layer are both spatial row embedding layers.

[0054] The multi-perspective fMRI classification model is constructed based on a codec structure, which includes an encoder and a decoder. The encoder includes a static functional connectivity embedding layer, a dynamic functional connectivity embedding layer, a time embedding layer, an SDCA encoder, and a TA encoder, and the decoder includes a token extraction fusion layer and a classification result output layer. The multi-perspective fMRI sample set in the present invention can be an fMRI sample set from three perspectives. In the encoder, the fMRI sample sets from three perspectives (fMRI sample set from the time series perspective, fMRI sample set from the static functional connectivity perspective, and fMRI sample set from the dynamic functional connectivity perspective) are respectively input into three embedding layers (sFC embedding layer, dFC embedding layer, and time embedding layer), so that the three embedding layers respectively output the embedded features Y of sFC. s , dFC embedded feature Y d and the embedded features Y of TS t The embedded features of sFC and dFC are then fed into the dynamic-static cross attention encoder, and the embedded features of TS are fed into the temporal encoder. The token extraction and fusion layer in the decoder extracts dynamic tokens, static tokens, and temporal tokens, which are then fused as input to the classification result output layer. The classification result output layer outputs the classification result from the fused tokens.

[0055] The spatial row embedding layer uses row embedding to reconstruct features of the fMRI sample set from the static functional connectivity perspective and the fMRI sample set from the dynamic functional connectivity perspective. The spatial row embedding layer first performs dimensionality reduction on both the fMRI sample set from the static functional connectivity perspective and the fMRI sample set from the dynamic functional connectivity perspective. Then, feature reconstruction is performed on the fMRI sample set from the static functional connectivity perspective and the fMRI sample set from the dynamic functional connectivity perspective after dimensionality reduction. The formula for feature reconstruction is as follows:

[0056] a d ,a s ∈R1 W×N×D →R1 W×M×(P,D) ;

[0057] Among them, a d is the fMRI sample set from the perspective of dynamic functional connectivity after dimensionality reduction, a s is the fMRI sample set of the static functional connectivity perspective after dimensionality reduction, W is the number of windows, N is the window size, D is the width of the block, P is the length of the block, P is 1, M = D / P, M is the number of embedded blocks, R1 W×N×D The dimensions of the fMRI sample set from the dynamic and static functional connectivity perspective under the number of windows W, window size N and block width D.

[0058] The spatial row embedding layer proposed in the present invention has significant advantages, enabling the multi-head attention mechanism to focus on the connection relationship between brain regions, rather than the simple relationship between square regions that has no practical significance in traditional methods. Through the spatial row embedding layer, the spatial relationship between brain regions can be more accurately captured and modeled during the processing of fMRI data. This spatial row embedding layer effectively retains the connection information between brain regions, allowing the multi-view fMRI classification model to understand and learn the interactions between different brain regions, thereby improving the multi-view fMRI classification model's ability to express brain network structure.

[0059] Specifically, Figure 3 The schematic diagram of the dynamic and static cross attention encoder and the temporal attention encoder is shown in Figure 3 As shown, Figure 3 (a) is a schematic diagram of the SDCA encoder. The dynamic-static cross attention encoder includes a dynamic-static cross attention layer, a first residual module and a second residual module connected to the static cross attention layer respectively, a first forward propagation layer connected to the first residual module, a second forward propagation layer connected to the second residual module, a third residual module connected to the first forward propagation layer, and a fourth residual module connected to the second forward propagation layer. The dynamic-static cross attention layer includes a first linear layer, a second linear layer, a third linear layer, a fourth linear layer and a dynamic-static multi-head attention mechanism calculation layer connected in sequence.

[0060] The SDCA encoder is used to fuse dynamic and static functional connectivity information, and to build a more efficient network by utilizing the similarity and complementarity of the fused dynamic and static functional connectivity information. The sFC embedding layer, dFC embedding layer, and temporal embedding layer are used to map the data into a pattern that is easy for deep networks to learn. The dynamic-static cross attention layer is used to calculate the static features in the fMRI sample set from the static functional connectivity perspective and the dynamic features in the fMRI sample set from the dynamic functional connectivity perspective through a cross-attention mechanism. The temporal attention encoder is used to forward propagate the fMRI sample set from the time series perspective, and is used to remap the temporal features in the fMRI sample set from the time series perspective to enhance the expressive power of the model. All residual modules are used to supplement the missing features after the attention mechanism and forward propagation. The Token extraction fusion layer is used to extract dynamic tokens, static tokens, and temporal tokens, and fuse them. All linear layers are used to generate dynamic queries Q based on the embedded features. d , dynamic key K d , dynamic value V d and static query Q s , static key K s , static value V s The dynamic and static multi-head attention mechanism calculation layer is used to exchange Ks 、V s and K d 、V d The dynamic and static multi-head attention mechanism adopts a multi-head attention mechanism.

[0061] The multi-head attention mechanism divides the query Q, key K, and value V into multiple independent attention mechanisms, providing different perspectives across data dimensions. The expression of the multi-head attention mechanism is:

[0062]

[0063] in, is the static query from the static functional connection of the i-th attention head, is the dynamic key of the i-th attention head derived from the dynamic functional connection, is the dynamic value of the i-th attention head derived from the dynamic functional connection, is the dynamic query of the i-th attention head from the dynamic functional connection, is the static key of the i-th attention head derived from the static-dynamic functional connection, V s (i) is the static value of the i-th attention head from the static functional connection, SA(·) is the attention mechanism, is the i-th attention head in static state, is the i-th attention head under dynamics, MHSA(q s ,K d ,V d ) is used to calculate the static query Q s , dynamic key K d and dynamic value V d Multi-head attention mechanism, To concatenate all the attention heads in static state, is the first attention head in static state, is the hth attention head in static state, W s O is the static projection weight, MHSA(q d ,K s ,V s ) is used to calculate the dynamic query Q d , static key K s and the static value V s Multi-head attention mechanism, In order to splice all the attention heads under the dynamics, is the first attention head under dynamics, is the h-th attention head under dynamics, is the dynamic projection weight, and h is the total number of attention heads.

[0064] Each attention head uses its corresponding learned weight matrix. After calculation using the multi-head attention mechanism, the results of all attention heads are concatenated and passed through their respective output projection matrices.

[0065] The expression of attention mechanism SA(·) is:

[0066]

[0067] Among them, SA(Q s ,K d ,V d ) is a static query Q s , dynamic key K d and dynamic value V d The corresponding attention mechanism, SA(Q d ,K s ,V s ) is a dynamic query Q d , static key K s and static value V s The corresponding attention mechanism, softmax(·) is a normalization operation, K is the dynamic key d The transpose of is the static key K s The transpose of d k is the dimension of the key vector k.

[0068] In order to achieve cross attention between the embedding features of sFC and dFC, K d 、V d and K s 、V s Exchange, allowing information from a dynamic perspective to focus on information from a static perspective, or allowing information from a static perspective to focus on information from a dynamic perspective.

[0069] For details, see Figure 3 As shown, Figure 3 (b) is a schematic diagram of the TA encoder. The temporal attention encoder includes a multi-head attention mechanism layer, a fifth residual module, a third forward propagation layer, and a sixth residual module connected in sequence.

[0070] The static-dynamic cross-attention layer combines the cross-attention mechanism, cleverly fusing the embedded features of sFC and dFC, and also fusing the embedded features of TS through a multi-head attention mechanism. This multi-view fusion approach enables the multi-view fMRI classification model to fully utilize the multiple measurement dimensions of fMRI and deeply analyze the functional activity of the brain from different perspectives. In particular, by introducing the static-dynamic cross-attention (SDCA) mechanism, the multi-view fMRI classification model can effectively learn the correlation and complementarity between sFC and dFC. The static-dynamic cross-attention layer enables the multi-view fMRI classification model to simultaneously capture static and dynamic information, further enhancing the expressive power of brain connectivity features, providing more accurate brain activity patterns, and providing more comprehensive and reliable information support for subsequent classification tasks, greatly improving the depth of cognition and prediction accuracy of brain function.

[0071] The present invention uses the SDCA encoder to interact with dynamic and static functional connectivity based on fMRI, calculating the correlation between them to extract features and achieve seamless fusion of dynamic and static information. Furthermore, the TA encoder provides time series information to supplement functional connectivity, addressing the challenges posed by short time series.

[0072] Specifically, Figure 4 The schematic diagram of the explainability analysis module is shown schematically, see Figure 4 As shown in FIG, the interpretability analysis module further includes a back-propagation saliency map submodule and a joint masking saliency map submodule, and the decision statistics difference algorithm submodule and the decision behavior related algorithm submodule are both connected to the back-propagation saliency map submodule.

[0073] The backpropagation saliency map submodule backpropagates a multi-view fMRI classification model and outputs backpropagation saliency maps for the three views. The joint masked saliency map submodule masks the input brain region information and outputs masked saliency maps for the three views. The decision statistical difference algorithm submodule determines whether there is a correlation between the static saliency map in the fMRI classification model's decision results and the differences between the fMRI sample sets from the two static functional connectivity views. The decision behavioral correlation algorithm submodule analyzes the static saliency map for correlation with behavioral data.

[0074] Specifically, Figure 5 The schematic diagram of the decision statistics difference algorithm submodule and the decision behavior related algorithm submodule is shown schematically, see Figure 5 As shown, Figure 5 (a) is a schematic diagram of a decision statistics difference algorithm submodule, which includes a two-sample t-test unit and a difference correlation calculation unit connected in sequence. Figure 5(b) is a schematic diagram of the decision-making behavior-related algorithm submodule, which includes a measurement-related calculation unit and a behavior matrix decision-related unit connected in sequence. The measurement-related calculation unit is a correlation calculation unit between the fMRI sample set and the behavioral data from the static functional connection perspective.

[0075] S104 , using the multi-view fMRI sample set to train the fMRI classification model to obtain a trained fMRI classification model.

[0076] The multi-view fMRI sample set is input into the fMRI classification model, and the weight parameters of the fMRI classification model are optimized using a fixed cross entropy loss function and an optimizer to obtain a trained fMRI classification model, that is, an fMRI classification model with optimal weight parameters.

[0077] S105 , inputting the medical image to be classified into the trained fMRI classification model, so that the trained fMRI classification model outputs a classification result.

[0078] The medical images to be classified are multi-view fMRI medical images.

[0079] The medical images to be classified are input into the trained fMRI classification model to obtain accurate classification results.

[0080] S106. Input the classification results, multi-view fMRI sample set and behavioral data into the interpretability analysis module, so that the interpretability analysis module outputs the multi-view fMRI sample set and the significant brain regions and significant connections related to the disease, as well as the correlation between the decision results of the fMRI classification model and the differences between categories and the correlation with the behavioral data.

[0081] The fMRI sample set from the static functional connectivity perspective in the multi-perspective fMRI sample set includes a first type of fMRI sample set from the static functional connectivity perspective and a second type of fMRI sample set from the static functional connectivity perspective.

[0082] Specifically, step S106 includes:

[0083] Step B1: Input the multi-view fMRI sample set into the joint masked saliency map submodule, so that the joint masked saliency map submodule masks the brain area information in the multi-view fMRI sample set, outputs a multi-view masked saliency map, and extracts the salient brain areas from the multi-view masked saliency map.

[0084] Step B2: Input the classification result into the back-propagation saliency map submodule, so that the back-propagation saliency map submodule back-propagates the classification result to obtain a multi-view back-propagation saliency map, and extract significant connections from the static saliency map of the multi-view back-propagation saliency map.

[0085] Step B3: Input the fMRI sample set from the first static functional connectivity perspective, the fMRI sample set from the second static functional connectivity perspective, and the static saliency map into the decision statistical difference algorithm submodule, so that the decision statistical difference algorithm submodule determines the corresponding first mean and first sample variance based on the fMRI sample set from the first static functional connectivity perspective, and determines the corresponding second mean and second sample variance based on the fMRI sample set from the second static functional connectivity perspective, and determines the correlation between the static saliency map and the difference between categories in the decision result of the fMRI classification model based on the first mean, first sample variance, second mean, second sample variance, and the static saliency map.

[0086] Specifically, step B3 includes:

[0087] Step B31: Input the fMRI sample set from the first static functional connectivity perspective and the fMRI sample set from the second static functional connectivity perspective into the two-sample t-test unit, so that the two-sample t-test unit determines the difference between the fMRI sample set from the first static functional connectivity perspective and the fMRI sample set from the second static functional connectivity perspective based on the first mean, the first sample variance, the second mean, and the second sample variance, and constructs a difference matrix based on the difference.

[0088] The two-sample t-test unit is used to calculate the difference between the fMRI sample sets of two types of static functional connectivity perspectives. The expression of the two-sample t-test unit is:

[0089]

[0090] Among them, D′ l,q is the difference between the fMRI sample set of the first type of static functional connectivity perspective and the fMRI sample set of the second type of static functional connectivity perspective, that is, an element of the difference matrix with the lth row and the qth column, is the first mean corresponding to the fMRI sample set of the first type of static functional connectivity perspective, is the first sample variance corresponding to the fMRI sample set of the first type of static functional connectivity perspective, is the second mean corresponding to the fMRI sample set of the second type of static functional connectivity perspective, is the second sample variance corresponding to the fMRI sample set from the second static functional connectivity perspective, n1 is the sample size of the fMRI sample set from the first static functional connectivity perspective, and n2 is the sample size of the fMRI sample set from the second static functional connectivity perspective. Connection difference D′ l,q is an element in the lth row and qth column of the difference matrix D.

[0091] Based on the difference between the fMRI sample set from the first static functional connectivity perspective and the fMRI sample set from the second static functional connectivity perspective, a difference matrix D is determined. The difference matrix represents the connectivity difference between the fMRI sample set from the first static functional connectivity perspective and the fMRI sample set from the second static functional connectivity perspective, and stores the t value of each connection.

[0092] Step B32: Input the difference, the mean corresponding to the difference matrix, and the mean corresponding to the static saliency map into the difference correlation calculation unit, so that the difference correlation calculation unit determines the correlation between the difference between the static saliency map and the category in the decision result of the fMRI classification model based on the difference, the mean corresponding to the difference matrix, and the mean corresponding to the static saliency map.

[0093] The difference correlation calculation unit is used to calculate the correlation between the difference matrix and the static saliency map in the decision result of the fMRI classification model. The expression of the difference correlation calculation unit is:

[0094]

[0095] where corr(D,S) is the correlation between the static saliency map and the difference between categories in the decision result of the fMRI classification model, that is, the correlation between the difference between the static saliency map S and the difference matrix D, and D′ l,q is the difference between the fMRI sample set of the first type of static functional connectivity perspective and the fMRI sample set of the second type of static functional connectivity perspective, that is, an element of the difference matrix with the lth row and the qth column, is the mean corresponding to the difference matrix, is the mean value corresponding to the static saliency map, S l,q is the static saliency map corresponding to the element in the lth row and qth column of the difference matrix, and corr(·) is the Pearson correlation calculation, which generates the R value and p value corresponding to the correlation between the static saliency map S and the difference matrix D. The r value and p value generated by the Pearson correlation calculation quantify the linear relationship between the static saliency map S and the difference matrix D, and the corresponding p value can indicate the significance of the linear relationship between the static saliency map S and the difference matrix D.

[0096] Step B4: Input the fMRI sample set, behavioral data, and static saliency map from the static functional connectivity perspective into the decision-making behavior-related algorithm submodule, so that the decision-making behavior-related algorithm submodule determines the corresponding third mean based on the fMRI sample set from the static functional connectivity perspective, and determines the corresponding fourth mean based on the behavioral data, and determines the correlation between the static saliency map and the behavioral data in the decision result of the fMRI classification model based on the fMRI sample set, the third mean, the behavioral data, and the fourth mean from the static functional connectivity perspective.

[0097] The measurement correlation calculation unit is used to calculate the correlation between the fMRI sample set from the static functional connectivity perspective and the behavioral data, thereby linking connectivity with behavior.

[0098] Specifically, step B4 specifically includes: inputting the fMRI sample set and behavioral data from the static functional connectivity perspective into the measurement-related calculation unit, so that the measurement-related calculation unit determines the corresponding third mean based on the fMRI sample set from the static functional connectivity perspective, and determines the corresponding fourth mean based on the behavioral data; determining the target correlation between the connection (l, q) in the matrix corresponding to the fMRI sample set from the static functional connectivity perspective and the behavioral data based on the fMRI sample set, the third mean, the behavioral data and the fourth mean; and determining the transformation value of the Z behavioral matrix of the target correlation based on the target correlation between the connection (l, q) in the matrix corresponding to the fMRI sample set from the static functional connectivity perspective and the behavioral data; inputting the transformation value of the Z behavioral matrix, the static saliency map and the mean of the static saliency map into the behavioral matrix decision-related unit, so that the behavioral matrix decision-related unit determines the correlation between the static saliency map and the behavioral data based on the transformation value of the Z behavioral matrix, the static saliency map and the mean of the static saliency map, that is, the correlation between the static saliency map and the transformation value of the Z behavioral matrix.

[0099] Specifically, the expression of the target correlation between the connection (l,q) and the behavioral data in the matrix corresponding to the fMRI sample set from the static functional connectivity perspective is:

[0100]

[0101] in, is the target correlation between the connection (l,q) and m behavioral data in the matrix corresponding to the fMRI sample set from the static functional connectivity perspective, F l,q,k1 is the value of the connection (l,q) of the k1th sample in the fMRI sample set from the static functional connectivity perspective, is the third mean corresponding to the connection (l,q) of the fMRI sample set from the static functional connectivity perspective, is the m behavioral data of the k1th sample, is the fourth mean corresponding to the m behavioral data.

[0102] The target correlation between the connection (l,q) and the behavioral data in the matrix corresponding to the fMRI sample set from the static functional connectivity perspective, and the expression of the transformation value of the corresponding Z behavioral matrix is:

[0103]

[0104] in, is the transformation value of the Z behavioral matrix connecting the target correlation between (l,q) and m behavioral data in the matrix corresponding to the fMRI sample set from the static functional connectivity perspective, The target correlation between the connection (l,q) and m behavioral data in the matrix corresponding to the fMRI sample set from the static functional connectivity perspective.

[0105] The behavior matrix decision correlation unit is used to calculate the correlation between the Z behavior matrix and the static saliency map S.

[0106] The expression for the correlation between the static saliency map and the transformed value of the Z behavior matrix is:

[0107]

[0108] Among them, corr(Z,S) is the correlation between the static saliency map S and the behavior matrix Z, l,q for A transformation value in is the mean of the Z behavioral matrix, is the mean value corresponding to the static saliency map, S l,q is the static saliency map corresponding to the element of the difference matrix in the lth row and the qth column.

[0109] The proposed decision-making statistical variance algorithm submodule aims to effectively quantify the interpretability of deep learning models by combining the decision-making process of the fMRI classification model with the statistical variance of the fMRI sample data itself. By comparing the fMRI classification model's predictions with the statistical characteristics of the data, the decision-making statistical variance algorithm submodule reveals the statistical laws and data characteristics that may underlie the fMRI classification model's decisions, partially addressing the "black box" problem commonly found in deep learning models.

[0110] The decision-making behavior-related algorithm submodule proposed in the present invention further enhances the transparency and credibility of the fMRI classification model. By combining the decisions of the fMRI classification model with the behavioral data behind the fMRI sample set data, the decision-making behavior-related algorithm submodule clearly demonstrates the physiological significance of the fMRI classification model's decisions. Behavioral data generally reflects the patient's cognitive or behavioral state. By combining this information with the fMRI sample set data, the fMRI classification model can not only provide predictions about brain activity, but also explain the relationship between these predictions and patient behavior. In this way, the prediction results of the fMRI classification model can be explained at the physiological level, further enhancing the credibility of the fMRI classification model and ensuring that its decision-making process is consistent with actual physiological activities.

[0111] The decision statistics difference algorithm submodule and the decision behavior related algorithm submodule not only improve the interpretability of the fMRI classification model, but also provide comparable indicators for the fMRI-based classification model, promoting the integration of neuroimaging and deep learning.

[0112] The present invention's decision-making statistical difference algorithm submodule and decision-making behavioral correlation algorithm submodule are used to analyze the correlation between fMRI classification model decisions and statistical differences and behavioral data, effectively addressing the low transparency issue of deep learning functional magnetic resonance imaging models. As a result, the present invention can perform interpretable classification tasks for three different fMRI sample sets.

[0113] Exemplarily, the present invention can be applied to depression. Specifically, it distinguishes the categories of fMRI sample sets, mines the significant brain regions and significant connections of the analyzed fMRI sample set data, and evaluates the correlation between the decisions of the fMRI classification model and statistical differences and behavioral data to evaluate the credibility of the fMRI classification model. A preprocessing method is used to map the fMRI sample set into fMRI sample sets of three perspectives. The preprocessed fMRI sample sets of three perspectives are divided into training sets and test sets in a 4:1 ratio. Then, a five-fold cross-validation is applied to the training set to ensure the robustness of the model. The initial learning rate, learning rate decay method, number of network iterations, optimization method and optimizer of the fMRI classification analysis are set. The fMRI classification model is trained using the medical images to be classified. After the training is completed, the classification effect of the fMRI classification model can be evaluated and the interpretability analysis can be performed using the test set.

[0114] Based on the above Figure 1It can be seen from the implementation method that the embodiment of the present invention obtains an fMRI sample set and behavioral data; preprocesses the fMRI sample set to obtain a multi-perspective fMRI sample set; constructs a multi-perspective fMRI classification model and an interpretability analysis module, the fMRI classification model includes a dynamic and static cross-attention encoder and a temporal attention encoder, and the interpretability analysis module includes a decision statistics difference algorithm submodule and a decision behavior related algorithm submodule; uses the multi-perspective fMRI sample set to train the fMRI classification model to obtain a trained fMRI classification model; inputs the medical image to be classified into the trained fMRI classification model, so that the trained fMRI classification model outputs a classification result; inputs the classification result, the multi-perspective fMRI sample set and the behavioral data into the interpretability analysis module, so that the interpretability analysis module outputs the multi-perspective fMRI sample set and the significant brain regions and significant connections related to the disease, as well as the correlation between the decision results of the fMRI classification model and the differences between categories and the correlation with the behavioral data. In this way, through the dynamic-static cross-attention encoder and the temporal attention encoder, information interaction between dynamic functional connections and static functional connections can be carried out, and the correlation between them can be calculated, so that complex relationships can be fully captured; through the interpretability analysis module, the correlation between significant brain areas, significant connections and the decision results of the fMRI classification model and the differences between categories and the correlation with behavioral data can be determined, so that the interpretability of the multi-perspective fMRI classification model can be improved.

[0115] Based on the same inventive concept, as an implementation of the above-mentioned fMRI classification and analysis method based on multiple perspectives, an embodiment of the present invention further provides an fMRI classification and analysis device based on multiple perspectives. Figure 6 FIG is a structural diagram of a device for fMRI classification analysis based on multiple perspectives according to an embodiment of the present invention, see Figure 3 As shown, the multi-view fMRI classification and analysis device may include:

[0116] Acquisition module 601, for acquiring fMRI sample sets and behavioral data;

[0117] A preprocessing module 602 is used to preprocess the fMRI sample set to obtain a multi-view fMRI sample set;

[0118] A construction module 603 is used to construct a multi-view fMRI classification model and an interpretability analysis module, wherein the fMRI classification model includes a dynamic-static cross attention encoder and a temporal attention encoder, and the interpretability analysis module includes a decision statistics difference algorithm submodule and a decision behavior related algorithm submodule;

[0119] A training module 604 is configured to train the fMRI classification model using a multi-view fMRI sample set to obtain a trained fMRI classification model;

[0120] A classification module 605 is configured to input the medical image to be classified into the trained fMRI classification model so that the trained fMRI classification model outputs a classification result;

[0121] The correlation analysis module 606 is used to input the classification results, the multi-view fMRI sample set and the behavioral data into the interpretability analysis module, so that the interpretability analysis module outputs the multi-view fMRI sample set and the significant brain regions and significant connections related to the disease, as well as the correlation between the decision results of the fMRI classification model and the differences between categories and the correlation with the behavioral data.

[0122] The preprocessing module 602 is specifically used to convert the digital medical imaging and communication standard format into the neuroimaging information technology initiative format to construct an fMRI sample set from a time series perspective, where the fMRI sample set from a time series perspective is composed of an image sequence reflecting the blood oxygen level-dependent effect at multiple time points; delete the first five time points from the multiple time points to obtain first fMRI data; perform time correction and head motion correction on the first fMRI data to obtain corrected fMRI data; perform registration and confounding elimination on the corrected fMRI data in sequence to obtain confounding-eliminated fMRI data; perform bandpass filtering and Gaussian kernel smoothing filtering on the confounding-eliminated fMRI data in sequence to obtain filtered fMRI data, where the filtered fMRI data is data containing multiple brain regions; calculate the Pearson correlation between the multiple brain regions to obtain an fMRI sample set from a static functional connectivity perspective; and use a sliding window method to determine an fMRI sample set from a dynamic functional connectivity perspective on the filtered fMRI data to obtain a multi-perspective fMRI sample set.

[0123] Building module 603, specifically used for the fMRI classification model, also includes a static functional connection embedding layer, a dynamic functional connection embedding layer, a temporal embedding layer, a marker extraction fusion layer and a classification result output layer. The static functional connection embedding layer, the static-dynamic cross attention encoder, the marker extraction fusion layer and the classification result output layer are connected in sequence. The dynamic functional connection embedding layer, the static-dynamic cross attention encoder, the marker extraction fusion layer and the classification result output layer are connected in sequence. The temporal embedding layer, the temporal attention encoder, the marker extraction fusion layer and the classification result output layer are connected in sequence. The static functional connection embedding layer and the dynamic functional connection embedding layer are both spatial row embedding layers.

[0124] Building module 603, the dynamic and static cross attention encoder includes a dynamic and static cross attention layer, a first residual module and a second residual module respectively connected to the static cross attention layer, a first forward propagation layer connected to the first residual module, a second forward propagation layer connected to the second residual module, a third residual module connected to the first forward propagation layer, and a fourth residual module connected to the second forward propagation layer. The dynamic and static cross attention layer includes a first linear layer, a second linear layer, a third linear layer, a fourth linear layer and a dynamic and static multi-head attention mechanism calculation layer connected in sequence.

[0125] Building module 603, the temporal attention encoder includes a multi-head attention mechanism layer, a fifth residual module, a third forward propagation layer and a sixth residual module connected in sequence.

[0126] Constructing module 603, the interpretability analysis module also includes a back-propagation saliency map submodule and a joint masking saliency map submodule, and the decision statistics difference algorithm submodule and the decision behavior related algorithm submodule are both connected to the back-propagation saliency map submodule.

[0127] Construction module 603, the decision statistics difference algorithm submodule includes a two-sample t-test unit and a difference correlation calculation unit connected in sequence, and the decision behavior correlation algorithm submodule includes a measurement correlation calculation unit and a behavior matrix decision correlation unit connected in sequence. The measurement correlation calculation unit is a correlation calculation unit for the fMRI sample set and behavioral data from the perspective of static functional connectivity.

[0128] The correlation analysis module 606 is specifically used to input the multi-view fMRI sample set into the joint masked saliency map submodule, so that the joint masked saliency map submodule masks the brain area information in the multi-view fMRI sample set, outputs the multi-view masked saliency map, and extracts the significant brain areas from the multi-view masked saliency map; inputs the classification results into the back propagation saliency map submodule, so that the back propagation saliency map submodule back propagates the classification results to obtain the multi-view back propagation saliency map, and extracts the significant connections from the static saliency map of the multi-view back propagation saliency map; inputs the fMRI sample set of the first type of static functional connectivity perspective, the fMRI sample set of the second type of static functional connectivity perspective, and the static saliency map into the decision statistical difference algorithm submodule, so that the decision statistical difference algorithm submodule determines the corresponding first mean and first sample variance based on the fMRI sample set of the first type of static functional connectivity perspective, and the fMRI sample set of the second type of static functional connectivity perspective. The first sample set determines the corresponding second mean and second sample variance, and determines the correlation between the static saliency map and the difference between categories in the decision result of the fMRI classification model based on the first mean, the first sample variance, the second mean, the second sample variance and the static saliency map; the fMRI sample set from the static functional connectivity perspective, the behavioral data and the static saliency map are input into the decision behavior related algorithm submodule, so that the decision behavior related algorithm submodule determines the corresponding third mean based on the fMRI sample set from the static functional connectivity perspective, and determines the corresponding fourth mean based on the behavioral data, and determines the correlation between the static saliency map and the behavioral data in the decision result of the fMRI classification model based on the fMRI sample set from the static functional connectivity perspective, the third mean, the behavioral data and the fourth mean. The fMRI sample set from the static functional connectivity perspective in the multi-perspective fMRI sample set includes the first type of fMRI sample set from the static functional connectivity perspective and the second type of fMRI sample set from the static functional connectivity perspective.

[0129] The correlation analysis module 606 inputs the fMRI sample set from the first static functional connectivity perspective, the fMRI sample set from the second static functional connectivity perspective, and the static saliency map into a decision statistics difference algorithm submodule, including: inputting the fMRI sample set from the first static functional connectivity perspective and the fMRI sample set from the second static functional connectivity perspective into a two-sample t-test unit, so that the two-sample t-test unit determines the difference between the fMRI sample set from the first static functional connectivity perspective and the fMRI sample set from the second static functional connectivity perspective based on the first mean, the first sample variance, the second mean, and the second sample variance, and constructs a difference matrix based on the difference; inputting the difference, the mean corresponding to the difference matrix, and the mean corresponding to the static saliency map into a difference correlation calculation unit, so that the difference correlation calculation unit determines the correlation between the static saliency map and the difference between the categories in the decision result of the fMRI classification model based on the difference, the mean corresponding to the difference matrix, and the mean corresponding to the static saliency map.

[0130] It should be noted that the above description of the embodiment of the multi-view fMRI classification and analysis device is similar to the description of the embodiment of the multi-view fMRI classification and analysis method, and has similar beneficial effects as the embodiment of the multi-view fMRI classification and analysis method. For any technical details not disclosed in the embodiment of the multi-view fMRI classification and analysis device of the present invention, please refer to the description of the embodiment of the multi-view fMRI classification and analysis method of the present invention for an understanding.

[0131] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A multi-view fMRI classification analysis method, characterized in that: include: Obtain fMRI sample sets and behavioral data; Preprocessing the fMRI sample set to obtain a multi-view fMRI sample set; Constructing a multi-perspective fMRI classification model and an interpretability analysis module, wherein the fMRI classification model includes a dynamic-static cross attention encoder and a temporal attention encoder, and the interpretability analysis module includes a decision statistics difference algorithm submodule and a decision behavior related algorithm submodule; Training the fMRI classification model using the multi-view fMRI sample set to obtain a trained fMRI classification model; Inputting the medical image to be classified into the trained fMRI classification model so that the trained fMRI classification model outputs a classification result; The classification results, the multi-view fMRI sample set and the behavioral data are input into the interpretability analysis module, so that the interpretability analysis module outputs the significant brain regions and significant connections related to the disease of the multi-view fMRI sample set, as well as the correlation between the decision results of the fMRI classification model and the differences between categories and the correlation with the behavioral data.

2. The multi-view fMRI classification and analysis method according to claim 1, characterized in that: The format of the fMRI sample set is a digital medical imaging and communication standard format. The fMRI sample set is preprocessed to obtain a multi-view fMRI sample set, including: Converting the digital medical imaging and communications standard format into a neuroimaging information technology initiative format to construct an fMRI sample set from a time series perspective, wherein the fMRI sample set from a time series perspective is composed of image sequences reflecting blood oxygen level-dependent effects at multiple time points; deleting first five time points from the plurality of time points to obtain first fMRI data; performing time correction and head motion correction on the first fMRI data to obtain corrected fMRI data; performing registration and confounding elimination on the corrected fMRI data in sequence to obtain confounding-eliminated fMRI data; performing bandpass filtering and Gaussian kernel smoothing filtering on the decongested fMRI data in sequence to obtain filtered fMRI data, wherein the filtered fMRI data is data containing multiple brain regions; Calculating the Pearson correlation between the multiple brain regions to obtain an fMRI sample set from a static functional connectivity perspective; The filtered fMRI data is subjected to a sliding window method to determine an fMRI sample set of a dynamic functional connectivity perspective, so as to obtain the multi-perspective fMRI sample set.

3. The multi-view fMRI classification and analysis method according to claim 1, wherein: The fMRI classification model also includes a static functional connection embedding layer, a dynamic functional connection embedding layer, a temporal embedding layer, a marker extraction fusion layer and a classification result output layer. The static functional connection embedding layer, the dynamic-static cross attention encoder, the marker extraction fusion layer and the classification result output layer are connected in sequence. The dynamic functional connection embedding layer, the dynamic-static cross attention encoder, the marker extraction fusion layer and the classification result output layer are connected in sequence. The temporal embedding layer, the temporal attention encoder, the marker extraction fusion layer and the classification result output layer are connected in sequence. The static functional connection embedding layer and the dynamic functional connection embedding layer are both spatial row embedding layers.

4. The multi-view fMRI classification and analysis method according to claim 1, wherein: The dynamic-static cross-attention encoder includes a dynamic-static cross-attention layer, a first residual module and a second residual module respectively connected to the static cross-attention layer, a first forward propagation layer connected to the first residual module, a second forward propagation layer connected to the second residual module, a third residual module connected to the first forward propagation layer, and a fourth residual module connected to the second forward propagation layer. The dynamic-static cross-attention layer includes a first linear layer, a second linear layer, a third linear layer, a fourth linear layer and a dynamic-static multi-head attention mechanism calculation layer connected in sequence.

5. The multi-view fMRI classification and analysis method according to claim 1, characterized in that: The temporal attention encoder includes a multi-head attention mechanism layer, a fifth residual module, a third forward propagation layer and a sixth residual module connected in sequence.

6. The multi-view fMRI classification and analysis method according to claim 2, characterized in that: The explainability analysis module also includes a back-propagation saliency map submodule and a joint masking saliency map submodule, and the decision statistics difference algorithm submodule and the decision behavior related algorithm submodule are both connected to the back-propagation saliency map submodule.

7. The multi-view fMRI classification and analysis method according to claim 6, characterized in that: The decision statistics difference algorithm submodule includes a two-sample t-test unit and a difference correlation calculation unit connected in sequence, and the decision behavior correlation algorithm submodule includes a measurement correlation calculation unit and a behavior matrix decision correlation unit connected in sequence. The measurement correlation calculation unit is a correlation calculation unit for the fMRI sample set from the static functional connectivity perspective and the behavioral data.

8. The multi-view fMRI classification and analysis method according to claim 7, characterized in that: The fMRI sample set from the static functional connectivity perspective in the multi-view fMRI sample set includes a first type of fMRI sample set from the static functional connectivity perspective and a second type of fMRI sample set from the static functional connectivity perspective. The classification result, the multi-view fMRI sample set, and the behavioral data are input into the interpretability analysis module, so that the interpretability analysis module outputs significant brain regions and significant connections related to the multi-view fMRI sample set and the disease, as well as correlations between the decision results of the fMRI classification model and the differences between categories and the correlations with the behavioral data, including: Inputting the multi-view fMRI sample set into the joint masked saliency map submodule, so that the joint masked saliency map submodule masks brain region information in the multi-view fMRI sample set, outputs a multi-view masked saliency map, and extracts the salient brain region from the multi-view masked saliency map; Inputting the classification result into the back-propagation saliency map submodule, so that the back-propagation saliency map submodule back-propagates the classification result to obtain a multi-view back-propagation saliency map, and extracting the significant connection from the static saliency map of the multi-view back-propagation saliency map; Inputting the first type of fMRI sample set from the static functional connectivity perspective, the second type of fMRI sample set from the static functional connectivity perspective, and the static saliency map into the decision statistics difference algorithm submodule, so that the decision statistics difference algorithm submodule determines a corresponding first mean and a first sample variance based on the first type of fMRI sample set from the static functional connectivity perspective, and determines a corresponding second mean and a second sample variance based on the second type of fMRI sample set from the static functional connectivity perspective, and determines a correlation between the static saliency map and the difference between categories in the decision result of the fMRI classification model based on the first mean, the first sample variance, the second mean, the second sample variance, and the static saliency map; The fMRI sample set from the static functional connectivity perspective, the behavioral data, and the static saliency map are input into the decision-making behavior-related algorithm submodule, so that the decision-making behavior-related algorithm submodule determines a corresponding third mean based on the fMRI sample set from the static functional connectivity perspective, and determines a corresponding fourth mean based on the behavioral data, and determines the correlation between the static saliency map and the behavioral data in the decision result of the fMRI classification model based on the fMRI sample set from the static functional connectivity perspective, the third mean, the behavioral data, and the fourth mean.

9. The multi-view fMRI classification and analysis method according to claim 8, characterized in that: The step of inputting the first type of fMRI sample set from the static functional connectivity perspective, the second type of fMRI sample set from the static functional connectivity perspective, and the static saliency map into the decision statistics difference algorithm submodule comprises: Inputting the first type of fMRI sample set from the static functional connectivity perspective and the second type of fMRI sample set from the static functional connectivity perspective into the two-sample t-test unit, so that the two-sample t-test unit determines the difference between the first type of fMRI sample set from the static functional connectivity perspective and the second type of fMRI sample set from the static functional connectivity perspective based on the first mean, the first sample variance, the second mean, and the second sample variance, and constructing a difference matrix based on the difference; The difference, the mean corresponding to the difference matrix, and the mean corresponding to the static saliency map are input into the difference correlation calculation unit, so that the difference correlation calculation unit determines the correlation between the difference between the static saliency map and the category in the decision result of the fMRI classification model based on the difference, the mean corresponding to the difference matrix, and the mean corresponding to the static saliency map.

10. A multi-view fMRI classification and analysis device, characterized in that: include: Acquisition module, used to acquire fMRI sample sets and behavioral data; A preprocessing module, configured to preprocess the fMRI sample set to obtain a multi-view fMRI sample set; A construction module for constructing a multi-perspective fMRI classification model and an interpretability analysis module, wherein the fMRI classification model includes a dynamic-static cross attention encoder and a temporal attention encoder, and the interpretability analysis module includes a decision statistics difference algorithm submodule and a decision behavior related algorithm submodule; A training module, configured to train the fMRI classification model using the multi-view fMRI sample set to obtain a trained fMRI classification model; A classification module, configured to input the medical image to be classified into the trained fMRI classification model, so that the trained fMRI classification model outputs a classification result; A correlation analysis module is used to input the classification results, the multi-view fMRI sample set and the behavioral data into the interpretability analysis module, so that the interpretability analysis module outputs the multi-view fMRI sample set and the significant brain regions and significant connections related to the disease, as well as the correlation between the decision results of the fMRI classification model and the differences between categories and the correlation with the behavioral data.

Images