Longitudinal nerve image data modeling and feature clustering method based on Transform model

Abstract

The invention discloses a longitudinal neural image data modeling and feature clustering method based on a Transform model, and the method comprises the steps: obtaining the longitudinal neural image data of a subject at three time points, so as to generate a second-order difference brain network; dividing the second-order differential brain network into a plurality of interaction sub-matrixes; coding the interaction sub-matrix to obtain a tested interaction embedding vector; matching the interactive embedding vector of the subject with the memory vector to obtain a new representation vector of the subject and a corresponding score list; performing graph reconstruction by using the new representation vector; and meanwhile, determining a clustering center. According to the method, deep learning and longitudinal nerve image analysis are combined, an efficient and automatic time sequence feature modeling and clustering analysis method is constructed, and the method can be used as an intelligent auxiliary analysis tool in mental disease research; the method has important scientific research and application value and wide popularization prospect in the aspect of supporting individual difference research and connection mode exploration.

Description

Technical Field

[0001] The present invention belongs to the interdisciplinary field of psychiatry, medical image analysis and machine learning, and specifically relates to a longitudinal neuroimaging data modeling and feature clustering method based on the Transformer model, which is suitable for structured representation and pattern mining of brain imaging data at multiple time points. Background Art

[0002] Mental illnesses exhibit a high degree of clinical and biological heterogeneity, posing numerous challenges to their diagnosis and treatment. Existing research has largely focused on grouping individuals based on symptomological characteristics. However, this subjective approach lacks a dynamic characterization of underlying neurobiological processes and fails to delve into the underlying mechanisms and progression of the disease. Therefore, while symptom-based classification facilitates disease analysis and treatment, it often becomes a major bottleneck in basic research and mechanistic exploration of mental illness, hindering the implementation of precision medicine goals.

[0003] On the other hand, previous cross-sectional studies have shown that different patients with mental illness may have unique underlying biological mechanisms, which are not static but change dynamically during the treatment of the disease. More importantly, the developmental trajectories of mental illness in different individuals may also differ. For example, the structural and functional changes in the brain during the course of the disease are closely related to changes in clinical symptoms, which shows that a deep understanding of these changes is crucial to improving treatment outcomes and predicting the course of disease treatment. Therefore, from a neuroimaging perspective, extracting characteristic patterns that reflect the evolution of the disease can not only help understand individual differences, but also provide very critical information for precise diagnosis and treatment.

[0004] In recent years, with the advancement of neuroimaging technology, particularly the widespread application of longitudinal neuroimaging data, researchers have been able to track the evolution of individual brain functional networks over time by analyzing imaging data from different time points. Compared to traditional cross-sectional studies, longitudinal imaging studies offer a more dynamic perspective, revealing the evolutionary trajectory of brain structure and function across the course of various psychiatric disorders. However, despite the richer information provided by longitudinal neuroimaging data, traditional statistical analysis or shallow machine learning methods have limitations in processing high-dimensional and temporally complex neuroimaging data, making it difficult to effectively extract deep temporally dependent structures and latent feature representations. To address this issue, deep learning techniques, particularly time series modeling based on the Transformer model, have demonstrated superior performance in medical image analysis. Due to its powerful sequence modeling capabilities and self-attention mechanism, the Transformer model has been widely used in neuroimaging research. Compared with traditional methods, its advantage lies in its ability to effectively handle information interactions across time points and capture the dynamic evolution of underlying patterns.

[0005] Therefore, how to use the powerful modeling capabilities of Transformer to develop a method with time series modeling and feature induction capabilities, which can identify the potential evolutionary patterns of individuals in the course of the disease from a dynamic dimension, and then provide strong technical support for individual difference research, scientific classification and precision medicine of mental illness is a difficult problem that needs to be solved urgently. Summary of the Invention

[0006] The present invention aims to overcome the shortcomings of existing technologies in multi-time point neuroimaging data modeling and individualized feature extraction, and provides a longitudinal neuroimaging data modeling and feature clustering method based on the Transformer model, which can model and cluster the cross-time point change patterns in neuroimaging data, thereby extracting discriminative connection feature evolution trajectories to support subsequent individual difference analysis and data-driven grouping research.

[0007] To achieve the above objectives, the present invention provides a method for longitudinal neuroimaging data modeling and feature clustering based on a Transformer model, comprising the following steps:

[0008] S1. Obtain longitudinal neuroimaging data from the subjects at three time points and convert them into brain functional connectivity matrices; calculate the second-order difference relationship between brain functional connectivity matrices at adjacent time points to generate a second-order difference brain network;

[0009] S2. Divide the brain functional connectivity matrix into multiple resting-state sub-networks, introduce a Transformer-based multi-head attention mechanism to capture the interaction between each sub-network, and divide the subject's second-order difference brain network into multiple interaction sub-matrices between sub-networks;

[0010] S3. Encode the interaction submatrix using an encoder including a graph convolution operation and an interaction attention operation to obtain an interaction embedding vector of the subject;

[0011] S4. Construct a memory library containing multiple memory vectors, match the subject's interaction embedding vector with the memory vector, and obtain a new representation vector for the subject, as well as a list of scores for matching the subject's interaction embedding vector with each memory vector;

[0012] S5. Introduce a graph decoder and reconstruct the graph using a new representation vector; at the same time, determine the cluster center using the K-means method based on the obtained score list.

[0013] A further preferred technical solution of the present invention is that in step S1, longitudinal neuroimaging data of the subject at three time points are obtained and converted into a brain functional connectivity matrix, and the second-order difference relationship between the brain functional connectivity matrices at adjacent time points is calculated to generate a second-order difference brain network; specifically:

[0014] Obtain longitudinal neuroimaging data of the subjects at three time points and convert them into brain functional connectivity matrices, which are then preprocessed for standardization.

[0015] Based on adjacent time points, the second-order difference relationship of the brain functional connectivity matrix is ​​calculated as follows:

[0016] ;

[0017] in, Represents the brain functional connectivity matrix at time point T;

[0018] The generated second-order difference brain network data is represented as , n represents the sample size, represents the integrated data of the two second-order difference relationships of the s-th subject, is the second-order difference relationship data between the first time point and the second time point, It is the second-order difference relationship data between the second time point and the third time point.

[0019] Preferably, in step S2, the Yeo-7 network atlas is used to divide the brain functional connectivity matrix into seven resting-state subnetworks, namely, the visual network VIS, the somatomotor network SMN, the dorsal attention network DAN, the ventral attention network VAN, the limbic network LIM, the frontoparietal network FPN, and the default mode network DMN;

[0020] The Transformer-based multi-head attention mechanism is introduced to capture the interaction between each sub-network, specifically:

[0021] The fMRI time series of each sub-network is divided into segments of fixed length, each segment is mapped into a feature vector of fixed dimension through linear transformation, and a learnable position encoding is added to preserve spatiotemporal information;

[0022] The attention head is calculated independently for each sub-network, the correlation weights between the sub-networks are calculated, the weighted fusion features are normalized by softmax, and then the outputs of the attention heads are concatenated and integrated through a linear layer;

[0023] Spatial self-attention is applied to the attention-weighted features, and the dimensions are compressed through the fully connected layer to output a representation containing the interaction features between sub-networks.

[0024] Preferably, the encoder in step S3 includes a graph convolution operation , an interactive attention operation , and aggregate functions ; Use the encoder to transform the second-order difference brain network of the p-th subnetwork of the s-th subject Mapping to high-dimensional space , expressed as ; Specifically include:

[0025] The graph convolution operation Defined as ,in Represent the learnable weights of the horizontal filter and the vertical filter respectively, is the weight of the last layer of information aggregation operation of the graph convolution, and the network interaction embedding of each second-order difference brain network of the s-th subject is calculated, which is expressed as:

[0026] ;

[0027] The multi-head attention mechanism is used to focus on the relationship between different network nodes. The attention embedding is obtained through the QKV attention mechanism and is expressed as ;

[0028] Using aggregate functions , embedding the network interaction of the embedded attention mechanism of each second-order difference brain network Aggregate and get the subject's interaction embedding vector , expressed as:

[0029] .

[0030] Preferably, in step S4, a memory library containing multiple memory vectors is constructed, and the subject's interaction embedding vector is matched with the memory vector to obtain a new representation vector of the subject and a score list of matching between the subject's interaction embedding vector and each memory vector; specifically:

[0031] initialization memory vector as a library of brain biomarkers;

[0032] Match the subject interaction embedding vector with the memory vector,

[0033] The interaction embedding vector of each subject is defined as a weighted combination of the memory vectors in the biomarker library. The following matching mechanism is introduced to interact the subject interaction embedding vector with the memory vector:

[0034] ;

[0035] ;

[0036] ;

[0037] in, are learnable weights; It is The memory vector is The weight assigned to each subject represents the The first Pattern scores, each subject was assigned a score list ; It is the new representation vector generated by the weighted combination of memory vectors.

[0038] As a preference, a graphics decoder is introduced in step S5, by Output the reconstructed image.

[0039] Preferably, after image reconstruction, the encoder, memory vector, and reconstructed image are optimized using a comprehensive loss function, which can be expressed as:

[0040] ;

[0041] in, is a contrastive loss used to promote the learning of interaction embedding vectors;

[0042] is a triple loss function used to update the memory vector, expressed as:

[0043] ;

[0044] in, According to the weight The first two memory vectors of the sorted s-th subject will be As an anchor, ensure Compare Store more information from anchors;

[0045] For the reconstruction , the reconstruction loss is used to make it close to the input image, and the reconstruction loss is expressed as:

[0046] ;

[0047] is the orthogonal regularization loss of the memory vector, expressed as:

[0048] .

[0049] On the other hand, the present invention provides a non-transitory computer-readable storage medium having computer instructions stored thereon, which enable a computer to execute the aforementioned longitudinal neuroimaging data modeling and feature clustering method based on the Transformer model.

[0050] Another aspect of the present invention provides an electronic device comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus, and the processor calls the logic instructions in the memory to execute the above-mentioned longitudinal neuroimaging data modeling and feature clustering method based on the Transformer model.

[0051] On the other hand, the present invention provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer executes the above-mentioned longitudinal neuroimaging data modeling and feature clustering method based on the Transformer model.

[0052] Compared with the prior art, the longitudinal neuroimaging data modeling and feature clustering method based on the Transformer model of the present invention has the following beneficial effects:

[0053] (1) Deep modeling of longitudinal dynamic features: This paper introduces the Transformer model to perform deep modeling of neuroimaging data of individuals with mental illness at multiple time points, effectively capturing the temporal evolution of brain functional connectivity. Compared with traditional static analysis methods, this paper breaks through the limitation of ignoring time dimension information and provides a more comprehensive and detailed trend and evolution trajectory of brain function changes. This dynamic modeling approach helps to reveal the underlying neurobiological mechanisms in the disease process and promote a deeper understanding of the relationship between disease development and clinical symptoms.

[0054] (2) Enhanced sensitivity and expressiveness of connection pattern grouping: This method uses the temporal feature embedding extracted by Transformer and combines it with clustering algorithms to perform individual difference analysis, which can accurately identify significant structural differences in dynamic connection features between different patients. Compared with traditional methods, this method shows higher sensitivity and stronger expressiveness in connection pattern recognition, and can effectively distinguish subgroups that show different evolutionary trajectories during the course of the disease, thereby improving the recognition accuracy and interpretability of potential connection evolution patterns.

[0055] (3) Good adaptability and scalability: This method is compatible with multiple longitudinal neuroimaging modalities (such as DTI) and can be widely applied to different psychiatric disease research scenarios. It has strong cross-task transfer capabilities and practical promotion potential. In addition to psychiatric disease research, this method is also applicable to other neurological disease individual feature analysis tasks that require long-term tracking and dynamic modeling.

[0056] (4) Advanced technology and high degree of automation: This paper uses the Transformer deep learning framework to automatically extract key connection features from high-dimensional, complex, and multi-time point neuroimaging data. This method significantly reduces the reliance on manual feature design, improves the efficiency of data processing and the intelligence level of the system, and avoids the manual intervention and subjective bias in traditional methods. In addition, the high degree of automation of this method makes it have good prospects for system implementation and engineering transformation.

[0057] In summary, the present invention combines deep learning with longitudinal neuroimaging analysis to construct an efficient and automated temporal feature modeling and cluster analysis method, which can be used as an intelligent auxiliary analysis tool in the study of mental illness. It has important scientific research application value and broad promotion prospects in supporting individual difference research and connection pattern exploration. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a flowchart of the longitudinal neuroimaging data modeling and feature clustering method based on the Transformer model of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the embodiments described are part of the embodiments of the present invention, not all of the embodiments, and they should not be understood as limitations on the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. In the description of the present invention, it should be understood that the terms used are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0060] The following combination Figure 1 The present invention describes the longitudinal neuroimaging data modeling and feature clustering method based on the Transformer model.

[0061] Example 1: This example provides a method for longitudinal neuroimaging data modeling and feature clustering based on the Transformer model. The method includes two stages, each consisting of multiple ordered steps with clear data flow and logical dependencies between the steps.

[0062] like Figure 1 As shown in Figure 2, the first stage is the encoding stage based on sub-network connections, and the second stage is to build a sub-network interaction pattern library to learn the network.

[0063] The first phase aims to construct a subnetwork connection representation that expresses the dynamic characteristics of brain region interactions from longitudinal neuroimaging data, providing structured input for subsequent cluster analysis. The specific steps include:

[0064] S1. Construction of second-order difference brain network.

[0065] Longitudinal neuroimaging data from the subjects at three time points were obtained and converted into a brain functional connectivity matrix. Functional magnetic resonance imaging (fMRI) technology has been able to map the functional connectivity (FC) between brain regions. This method can be used to convert longitudinal neuroimaging data into a brain functional connectivity matrix.

[0066] The second-order difference relationship between the brain functional connectivity matrices at adjacent time points was then calculated to generate the second-order difference brain network;

[0067] Based on adjacent time points, the second-order difference relationship of the brain functional connectivity matrix is ​​calculated as follows:

[0068] ;

[0069] in, Represents the brain functional connectivity matrix at time point T;

[0070] The generated second-order difference brain network data is represented as , n represents the sample size, represents the integrated data of the two second-order difference relationships of the s-th subject, is the second-order difference relationship data between the first time point and the second time point, The network is designed to highlight areas of significant change in the entire brain network between two consecutive time points.

[0071] S2. Interactive learning of brain sub-networks.

[0072] According to the brain area division template, the Yeo-7 network atlas was used to divide the brain functional connectivity matrix into seven resting-state sub-networks, namely the visual network VIS, the somatomotor network SMN, the dorsal attention network DAN, the ventral attention network VAN, the limbic network LIM, the frontoparietal network FPN and the default mode network DMN.

[0073] In this embodiment, the interaction features between sub-networks are studied, rather than relying on manually designed features, which can more effectively capture the significant features between subjects, thereby achieving more accurate clustering. Compared with traditional methods, this method can automatically learn potential interaction relationships from raw data, enhancing the characterization of the dynamic evolution of complex functional brain networks. When modeling this interaction process through graph theory, learning the association relationships between sub-networks is crucial, and the Transformer-based method has been proven to be a robust method for modeling association relationships. Therefore, a brain network interaction module based on Transformer's multi-head attention is introduced in this embodiment to capture interaction features that are helpful for subsequent analysis.

[0074] In this embodiment, a Transformer-based multi-head attention mechanism is introduced to capture the interaction relationship between sub-networks as follows:

[0075] The fMRI time series of each sub-network is divided into segments of fixed length, each segment is mapped into a feature vector of fixed dimension through linear transformation, and a learnable position encoding is added to preserve spatiotemporal information;

[0076] The attention head is calculated independently for each sub-network, the correlation weights between the sub-networks are calculated, the weighted fusion features are normalized by softmax, and then the outputs of the attention heads are concatenated and integrated through a linear layer;

[0077] Spatial self-attention is applied to the attention-weighted features, and the dimensions are compressed through the fully connected layer to output a representation containing the interaction features between sub-networks.

[0078] S3. Interactive embedding vector generation.

[0079] This embodiment constructs an encoder that includes a graph convolution operation , aims to learn the representation of sub-network connection information; secondly, interactive attention operation , to effectively simulate the interactions of brain subnetworks, as well as aggregation functions , further extracting effective information. Use the encoder to transform the second-order difference brain network of the p-th sub-network of the s-th subject into Mapping to high-dimensional space , expressed as .

[0080] Specifically, the graph convolution operation Defined as ,in Represent the learnable weights of the horizontal filter and the vertical filter respectively, is the weight of the last layer of information aggregation operation of the graph convolution, and the network interaction embedding of each second-order difference brain network of the s-th subject is calculated, which is expressed as:

[0081] ;

[0082] Then, the encoder uses a multi-head attention mechanism to focus on the relationship between different network nodes, effectively simulating the interaction between them. The attention embedding is obtained through the QKV attention mechanism and is expressed as ;

[0083] Next, using the aggregate function , embedding the network interaction of the embedded attention mechanism of each second-order difference brain network Aggregate and get the subject's interaction embedding vector , expressed as:

[0084] .

[0085] The second stage is based on the above embedded representation and realizes dynamic modeling of individual differences and potential pattern discovery by introducing representative brain connection prototypes and clustering mechanisms.

[0086] S4. Learning the brain’s biological pattern library.

[0087] Inspired by memory networks, this example proposes the construction of a brain biomarker library that can store typical evolutionary patterns and facilitate personalized matching with individual subjects.

[0088] initialization memory vector As a library of brain biomarkers, they can be updated during the optimization process. These memory vectors are orthogonal to each other and can memorize different information. Therefore, orthogonal regularization is introduced. .

[0089] The interaction between the biomarker library and different subjects is achieved through a search and matching process. It is assumed that the high-dimensional embedding of each subject can be represented by a weighted combination of memory vectors in the biomarker library, and the specificity between subjects is reflected by assigning different weights to these memory vectors. Specifically, the following matching mechanism is introduced:

[0090] ;

[0091] ;

[0092] ;

[0093] in, are learnable weights; It is The memory vector is The weights assigned to each subject, more specifically, Also represents the The first subject pattern scores, so the pattern scores of the subjects represented by the memory vector can be expressed as a list of combined values ,These individual scores sensitively reflect individual characteristics; It is the new representation vector generated by the weighted combination of memory vectors.

[0094] S5. Structural reconstruction.

[0095] In order to enable the model to focus on more useful meta-path-based interaction information, the input graph is reconstructed using a bank memory vector. Specifically, a graph decoder is introduced, which is implemented by The reconstructed image is output as the reconstructed neural image.

[0096] S6. Optimize target design.

[0097] After image reconstruction, the encoder, memory vector, and reconstructed image are optimized using a comprehensive loss function, which is expressed as:

[0098] ;

[0099] in, is a contrastive loss used to promote the learning of interaction embedding vectors;

[0100] is a triple loss function used to update the memory vector, expressed as:

[0101] ;

[0102] in, According to the weight The first two memory vectors of the sorted s-th subject will be As an anchor, ensure Compare Store more information from anchors;

[0103] For the reconstruction , the reconstruction loss is used to make it close to the input image, and the reconstruction loss is expressed as:

[0104] ;

[0105] is the orthogonal regularization loss of the memory vector, expressed as:

[0106] .

[0107] S7. Score-based interaction pattern clustering.

[0108] By matching the interaction embeddings with the memory vectors, each subject was assigned a list of scores After analyzing these score lists, we found that the scores of the subjects were similar, but there were different groups. Using the K-means method, we can determine the cluster centers.

[0109] Example 2: This embodiment provides a non-transitory computer-readable storage medium having computer instructions stored thereon, which enable a computer to execute a longitudinal neuroimaging data modeling and feature clustering method based on a Transformer model, the method comprising the following steps:

[0110] S1. Obtain longitudinal neuroimaging data from the subjects at three time points and convert them into brain functional connectivity matrices; calculate the second-order difference relationship between brain functional connectivity matrices at adjacent time points to generate a second-order difference brain network;

[0111] S2. Divide the brain functional connectivity matrix into multiple resting-state sub-networks, introduce a Transformer-based multi-head attention mechanism to capture the interaction between each sub-network, and divide the subject's second-order difference brain network into multiple interaction sub-matrices between sub-networks;

[0112] S3. Encode the interaction submatrix using an encoder including a graph convolution operation and an interaction attention operation to obtain an interaction embedding vector of the subject;

[0113] S4. Construct a memory library containing multiple memory vectors, match the subject's interaction embedding vector with the memory vector, and obtain a new representation vector for the subject, as well as a list of scores for matching the subject's interaction embedding vector with each memory vector;

[0114] S5. Introduce a graph decoder and reconstruct the graph using a new representation vector; at the same time, determine the cluster center using the K-means method based on the obtained score list.

[0115] Example 3: This embodiment provides an electronic device, which may include: a processor, a communications interface, a memory, and a communications bus, wherein the processor, the communications interface, and the memory communicate with each other via the communications bus. The processor may call logic instructions in the memory to execute a longitudinal neuroimaging data modeling and feature clustering method based on a Transformer model, the method comprising the following steps:

[0116] S1. Obtain longitudinal neuroimaging data from the subjects at three time points and convert them into brain functional connectivity matrices; calculate the second-order difference relationship between brain functional connectivity matrices at adjacent time points to generate a second-order difference brain network;

[0117] S2. Divide the brain functional connectivity matrix into multiple resting-state sub-networks, introduce a Transformer-based multi-head attention mechanism to capture the interaction between each sub-network, and divide the subject's second-order difference brain network into multiple interaction sub-matrices between sub-networks;

[0118] S3. Encode the interaction submatrix using an encoder including a graph convolution operation and an interaction attention operation to obtain an interaction embedding vector of the subject;

[0119] S4. Construct a memory library containing multiple memory vectors, match the subject's interaction embedding vector with the memory vector, and obtain a new representation vector for the subject, as well as a list of scores for matching the subject's interaction embedding vector with each memory vector;

[0120] S5. Introduce a graph decoder and reconstruct the graph using a new representation vector; at the same time, determine the cluster center using the K-means method based on the obtained score list.

[0121] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0122] Example 4: This embodiment provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform a longitudinal neuroimaging data modeling and feature clustering method based on a Transformer model. The method includes the following steps:

[0123] S1. Obtain longitudinal neuroimaging data from the subjects at three time points and convert them into brain functional connectivity matrices; calculate the second-order difference relationship between brain functional connectivity matrices at adjacent time points to generate a second-order difference brain network;

[0124] S2. Divide the brain functional connectivity matrix into multiple resting-state sub-networks, introduce a Transformer-based multi-head attention mechanism to capture the interaction between each sub-network, and divide the subject's second-order difference brain network into multiple interaction sub-matrices between sub-networks;

[0125] S3. Encode the interaction submatrix using an encoder including a graph convolution operation and an interaction attention operation to obtain an interaction embedding vector of the subject;

[0126] S4. Construct a memory library containing multiple memory vectors, match the subject's interaction embedding vector with the memory vector, and obtain a new representation vector for the subject, as well as a list of scores for matching the subject's interaction embedding vector with each memory vector;

[0127] S5. Introduce a graph decoder and reconstruct the graph using a new representation vector; at the same time, determine the cluster center using the K-means method based on the obtained score list.

[0128] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0129] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A longitudinal neuroimaging data modeling and feature clustering method based on the Transformer model, characterized by: The steps include: S1. Obtain longitudinal neuroimaging data from the subjects at three time points and convert them into brain functional connectivity matrices; calculate the second-order difference relationship between brain functional connectivity matrices at adjacent time points to generate a second-order difference brain network; S2. Divide the brain functional connectivity matrix into multiple resting-state sub-networks, introduce a Transformer-based multi-head attention mechanism to capture the interaction between each sub-network, and divide the subject's second-order difference brain network into multiple interaction sub-matrices between sub-networks; S3. Encode the interaction submatrix using an encoder including a graph convolution operation and an interaction attention operation to obtain an interaction embedding vector of the subject; S4. Construct a memory library containing multiple memory vectors, match the subject's interaction embedding vector with the memory vector, and obtain a new representation vector for the subject, as well as a list of scores for matching the subject's interaction embedding vector with each memory vector; S5. Introduce a graph decoder and reconstruct the graph using a new representation vector; at the same time, determine the cluster center using the K-means method based on the obtained score list.

2. The method for longitudinal neuroimaging data modeling and feature clustering based on the Transformer model according to claim 1, characterized in that: In step S1, longitudinal neuroimaging data of the subjects at three time points are obtained and converted into brain functional connectivity matrices. The second-order difference relationship between the brain functional connectivity matrices at adjacent time points is calculated to generate a second-order difference brain network. Specifically, Obtain longitudinal neuroimaging data of the subjects at three time points and convert them into brain functional connectivity matrices, which are then preprocessed for standardization. Based on adjacent time points, the second-order difference relationship of the brain functional connectivity matrix is ​​calculated as follows: ； in, Represents the brain functional connectivity matrix at time point T; The generated second-order difference brain network data is represented as , n represents the sample size, represents the integrated data of the two second-order difference relationships of the s-th subject, is the second-order difference relationship data between the first time point and the second time point, It is the second-order difference relationship data between the second time point and the third time point.

3. The method for longitudinal neuroimaging data modeling and feature clustering based on the Transformer model according to claim 1, characterized in that: In step S2, the Yeo-7 network atlas was used to divide the brain functional connectivity matrix into seven resting-state subnetworks, namely the visual network VIS, the somatomotor network SMN, the dorsal attention network DAN, the ventral attention network VAN, the limbic network LIM, the frontoparietal network FPN, and the default mode network DMN; The Transformer-based multi-head attention mechanism is introduced to capture the interaction between each sub-network, specifically: The fMRI time series of each sub-network is divided into segments of fixed length, each segment is mapped into a feature vector of fixed dimension through linear transformation, and a learnable position encoding is added to preserve spatiotemporal information; The attention head is calculated independently for each sub-network, the correlation weights between the sub-networks are calculated, the weighted fusion features are normalized by softmax, and then the outputs of the attention heads are concatenated and integrated through a linear layer; Spatial self-attention is applied to the attention-weighted features, and the dimensions are compressed through the fully connected layer to output a representation containing the interaction features between sub-networks.

4. The method for longitudinal neuroimaging data modeling and feature clustering based on the Transformer model according to claim 1, characterized in that: The encoder in step S3 includes a graph convolution operation , an interactive attention operation , and aggregate functions ; Use the encoder to transform the second-order difference brain network of the p-th subnetwork of the s-th subject Mapping to high-dimensional space , expressed as ; Specifically include: The graph convolution operation Defined as ,in Represent the learnable weights of the horizontal filter and the vertical filter respectively, is the weight of the last layer of information aggregation operation of the graph convolution, and the network interaction embedding of each second-order difference brain network of the s-th subject is calculated, which is expressed as: ； The multi-head attention mechanism is used to focus on the relationship between different network nodes. The attention embedding is obtained through the QKV attention mechanism and is expressed as ; Using aggregate functions , embedding the network interaction of the embedded attention mechanism of each second-order difference brain network Aggregate and get the subject's interaction embedding vector , expressed as: 。 5. The method for longitudinal neuroimaging data modeling and feature clustering based on the Transformer model according to claim 4, characterized in that: In step S4, a memory library containing multiple memory vectors is constructed, and the subject's interaction embedding vector is matched with the memory vector to obtain the subject's new representation vector and a score list of the matching between the subject's interaction embedding vector and each memory vector; specifically: initialization memory vector as a library of brain biomarkers; Match the subject interaction embedding vector with the memory vector, The interaction embedding vector of each subject is defined as a weighted combination of the memory vectors in the biomarker library. The following matching mechanism is introduced to interact the subject interaction embedding vector with the memory vector: ； ； ； in, are learnable weights; It is The memory vector is The weight assigned to each subject represents the The first Pattern scores, each subject was assigned a score list ; It is the new representation vector generated by the weighted combination of memory vectors.

6. The method for longitudinal neuroimaging data modeling and feature clustering based on the Transformer model according to claim 5, characterized in that: In step S5, a graphics decoder is introduced. Output the reconstructed image.

7. The method for longitudinal neuroimaging data modeling and feature clustering based on the Transformer model according to claim 6, characterized in that: After image reconstruction, the encoder, memory vector, and reconstructed image are optimized using a comprehensive loss function, which is expressed as: ； in, is a contrastive loss used to promote the learning of interaction embedding vectors; is a triple loss function used to update the memory vector, expressed as: ； in, According to the weight The first two memory vectors of the sorted s-th subject will be As an anchor, ensure Compare Store more information from anchors; For the reconstruction , the reconstruction loss is used to make it close to the input image, and the reconstruction loss is expressed as: ； is the orthogonal regularization loss of the memory vector, expressed as: 。 8. A non-transitory computer-readable storage medium, characterized in that Computer instructions are stored thereon, and the computer instructions enable the computer to execute the longitudinal neuroimaging data modeling and feature clustering method based on the Transformer model as described in any one of claims 1-7.

9. An electronic device, characterized in that: include: A processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus, and the processor calls logic instructions in the memory to execute the longitudinal neuroimaging data modeling and feature clustering method based on the Transformer model as described in any one of claims 1-7.

10. A computer program product, characterized in that The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer executes the longitudinal neuroimaging data modeling and feature clustering method based on the Transformer model according to any one of claims 1 to 7.

Images