Brain network cross-modal generation method based on diffusion model

Through the Huygens-Fresnel spherical diffusion and noise removal method, combined with the generative adversarial network, the problems of low computing efficiency and insufficient sample diversity in cross-modal brain network generation are solved, and efficient and accurate brain network generation is achieved. Multimodal data provides the basis for multimodal disease diagnosis.

CN118942648APending Publication Date: 2024-11-12HUBEI UNIV OF ECONOMICS
View PDF 0 Cites 4 Cited by

Patent Information

Application Number
CN202410956463.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-16
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing technology has low computational efficiency in the generation of cross-modal brain networks, lack of diversity and richness of generated samples, and unstable training of generative adversarial networks, resulting in errors in the model when calculating external mode using the characteristics of the brain network in the modal. As a result, the real data characteristics cannot be fully reflected.

Method used

The cross-modal generation method of brain network based on Huygens-Fresnel spherical diffusion denoising is adopted. The end-to-end cross-border through Huygens spherical diffusion module, group transformation iterative noise denoising module and Fresnel wavefield discrimination module are realized. 模态脑网络生成,结合生成对抗策略和扩散模型,使用5折交叉训练和损失函数约束,提升生成精度和多样性。

Benefits of technology

Improve computing efficiency and generation accuracy, realize the end-to-end generation of target domain brain networks from source domain image data, enhance the quality and diversity of brain network generation, and provide a data foundation for multimodal fusion, suitable for multimodal Diagnostic analysis of Parkinson's disease and other diseases.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942648A_ABST
    Figure CN118942648A_ABST
Patent Text Reader

Abstract

The invention discloses a brain network cross-modal generation method based on a diffusion model. The method comprises the following steps: constructing a brain network cross-modal model based on Huygens-Fresnel spherical diffusion denoising; for each patient, collecting two modes of fMRI (functional magnetic resonance imaging) and DTI (diffusion tensor imaging) to carry out five-fold cross training on the constructed model, using 80% of tested objects for training the model in each fold, using the other 20% of the tested objects for testing, and achieving convergence when a loss function value of model training does not change any more; and after model training is completed, model testing is carried out. According to the method, the target domain brain network can be generated from the source domain image data in an end-to-end manner, the calculation efficiency and the generation precision are improved, and a data basis is provided for multi-modal fusion; multi-modal fusion and analysis are facilitated, the target domain brain network is mapped through the generative model, and a data basis is provided for multi-modal Parkinson's disease diagnosis.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of medical image computing, and in particular to a brain network cross-modal generation method based on a diffusion model. Background Art

[0002] Brain networks are usually divided into structural connectivity matrices (SC) and functional connectivity matrices (FC). SC represents the physical connections between different brain regions through axon fiber bundles; FC reveals the neural circuits of various cognitive functions. Both SC and FC can be explored to detect abnormal connections related to diseases for clinical diagnosis and treatment. In medical image processing and brain science research, different modalities provide brain network information from multiple perspectives. Single-modality imaging data may lack certain structural or functional brain connection information. The fusion of SC and FC can enhance the complementarity of structural and functional connections, greatly improving the ability of disease analysis.

[0003] In clinical research, due to the high cost of nuclear magnetic resonance imaging and other reasons, some necessary modal brain imaging data are often lacking, resulting in problems such as few multimodal data samples, overfitting of models, and low accuracy in predicting cognitive diseases. Through cross-modal data synthesis, multimodal data can be completed, thereby achieving accurate diagnosis of brain diseases. In recent years, methods based on deep learning have achieved certain results in cross-modal brain network synthesis. However, the existing technology does not realize end-to-end cross-modal brain network generation, and there is a problem of low computational efficiency. Due to different parameter settings when using the software, there is a large uncertainty and error in the calculation of functional timing features, which makes the machine learning model produce wrong results when using these modal intra-brain network features to calculate the extra-modal brain network; the existing generative methods also have problems of training instability and mode collapse. The training of generative adversarial networks is usually very difficult and unstable. The game between the generator and the discriminator leads to training instability, and mode collapse or oscillation may occur, that is, the generator falls into a cycle of generating similar samples instead of generating diverse samples. The generator tends to generate only part of the mode in the data distribution, rather than the entire data distribution. This results in the generated samples lacking diversity and richness, and failing to fully reflect the characteristics of real data. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a method for cross-modal generation of brain networks based on a diffusion model.

[0005] Technical solution: The cross-modal generation method of brain networks based on diffusion model described in the present invention comprises the following steps:

[0006] (1) Construct a cross-modal model of brain networks based on Huygens-Fresnel spherical diffusion denoising;

[0007] (2) For each patient, two modalities, fMRI (functional magnetic resonance imaging) and DTI (diffusion tensor imaging), were collected to perform 5-fold cross-training on the constructed model. In each fold, 80% of the subjects were used to train the model, and the remaining 20% ​​were used for testing. When the loss function value of the model training no longer changed, convergence was achieved.

[0008] (3) After model training is completed, model testing is performed.

[0009] Furthermore, the brain network cross-modal model based on Huygens-Fresnel spherical diffusion denoising in step (1) includes three modules, namely, a Huygens spherical diffusion module, a group transform iterative denoising module and a Fresnel wave field discrimination module.

[0010] Furthermore, the Huygens spherical diffusion module includes forward diffusion and reverse diffusion.

[0011] Furthermore, the Huygens sphere forward diffusion is to gradually transform the real structure connection matrix C0 into a Gaussian noise matrix C through a sufficient number of T steps. T , where C0 represents the matrix of size N*N obtained by diffusion tensor imaging preprocessed by the software toolbox, N represents the number of brain regions, and C t is the structural connection matrix at time t. In each step, weights are applied to the Gaussian noise and superimposed on the real structural connection matrix. These weights are predefined as decimals that gradually increase between 0 and 1: β1, β2, β3, ..., β T , the calculation formula between two adjacent steps is:

[0012]

[0013] Among them, ∈′ represents the transpose of ∈, N(0, 0.3) represents Gaussian distribution noise with mean 0 and variance 0.3; after t steps of diffusion, its structural connection matrix C t It can be expressed as:

[0014]

[0015] Among them, ξ t =1-β t , ∈ t It represents the noise matrix sampled from N(0, 0.3) at time t, which obeys Gaussian distribution.

[0016] Furthermore, the reverse diffusion of the Huygens sphere is the structural connection matrix C from time T TThe mapping of the structural connection matrix C0 to time 0 is conditioned by the functional brain image to guide the Fresnel network to learn and remove Gaussian noise, which is illustrated by the reverse diffusion from time t to time t-1. The Fresnel equalization network is represented by the symbol FNL. Assume

[0017] C t-1 '=FNL t (C t , fMRI, E t )

[0018] Among them, the input parameters of the Fresnel equalization network FNL are three, namely the structural connection matrix at time t, C t The size is N*N, the size of the functional brain image is V1*V2*V3*q, and the time embedding vector is E t The size is T*N*N; V1, V2, V3 represent the voxel size in three directions of functional magnetic resonance imaging, q represents the scanning time length of functional magnetic resonance imaging; the output structural connection matrix C t-1 The size is N*N.

[0019] Furthermore, the input of the group transform iterative denoising module is the structural connection matrix C at time t t , functional brain images and temporal embedding vector E t , the output is the mechanism connection matrix C after denoising at time t-1 t-1 ';

[0020] Functional brain images are converted into ROI-based fuzzy time series F1 through the discontinuous node feature extraction layer, with a size of N*q. This network layer multiplies the brain anatomical atlas Atlas, with a size of V1*V2*V3, with the voxel information of each time point of fMRI. The whole brain is divided into N regions, and the feature dimension of each brain region is 1*q. The calculation formula is:

[0021] F1=fMRI⊙Atlas

[0022] The bidirectional time-extended attention mechanism layer further explores the topological connection features of F1, fine-tunes the feature resolution between adjacent brain regions, and outputs the fine brain region feature F2, which is calculated as:

[0023]

[0024] Q=LM(F1), V=LM(F1), K1=LM(F1), K2=LM(F1)

[0025] Among them, LM represents the linear mapping network, K1' is the transpose of K1, and the Softma*(*) function is to perform row vector normalization on the variable *;

[0026] F2 and C t After the topology-aware convolutional network layer and the neural inhibition layer, the fusion feature F3 is output; its calculation formula is:

[0027] F3=ReLU(C t ·F2·W1)

[0028] Among them, ReLU is the activation function and W is the parameter to be learned;

[0029] F3, C t and E t After the topology-aware convolutional network layer and the neuron inhibition layer, the fusion feature F4 is output; the calculation formula is:

[0030] F4=ReLU(C t ·F3·.W2+E t )

[0031] F4 passes through the forward propagation layer and the neuron connection correction layer, and outputs the structural connection matrix C t-1 ', the calculation formula is:

[0032] F5=LM(F4)

[0033] C t-1 '=σ(F5·F5')

[0034] Furthermore, the Fresnel wave field discrimination module sequentially uses a cross convolution layer, a longitudinal convolution layer, a transverse convolution layer and a regression conversion layer to output a scalar, 1 represents true, 0 represents false, and when the scalar value is 0.5, it means that the training of this module is completed. t-1 'Enter the Fresnel wave field discrimination module description, the following is the calculation formula:

[0035] U1=CF(C t-1 ')

[0036] U 21 =VF(U1)

[0037] U 22 =HF(U1)

[0038] O=σ(LM(U 21 ||U 22 +I t ))

[0039] Among them, CF represents the cross convolution layer, which uses two cross convolution kernels of size: U1 represents 1*N and N*1 to extract bidirectional brain connection feature information from the structural connection matrix, VF represents the longitudinal convolution layer, HF represents the horizontal convolution layer, and U 21Indicates that the 1*N convolution kernel extracts features from the feature matrix output by the cross convolution layer, U 22 Indicates that the N*1 convolution kernel extracts features from the feature matrix output by the cross convolution layer; U 21 ||U 22 +I t It means that the regression transformation layer maps the concatenated features to between 0 and 1. t Represents the time embedding vector at time t, with a size of T*2N. σ- is the Sigmoid activation function, which maps the input features to between 0 and 1; o is the output value of the Fresnel wavefield discrimination module. The closer its value is to 1, the more real the input sample is. If its value is closer to 0, the more fake the input sample is.

[0040] Furthermore, step (2) includes collecting fMRI and DTI modalities for each patient, wherein the DTI is used to calculate the real structural connectivity matrix C0 through the software toolbox, and the fMRI and structural connectivity matrix C0 are sent to the model, and the structural connectivity matrix C0 is obtained at each diffusion stage through the Huygens forward diffusion process. t Then, through the Huygens reverse diffusion process, the structural connection matrix C after denoising at each stage is gradually obtained. t ', after T steps, a clean structural connection matrix C0' is obtained.

[0041] Furthermore, the four loss functions in the model training process in step (2) are as follows:

[0042] (2.1) Huygens-Fresnel learning loss, by calculating the difference between the structural connection matrix of the reverse diffusion and the structural connection matrix of the forward diffusion process after denoising at each stage, constrains their distribution consistency. This loss function is defined as:

[0043] L1=∑ t≥1 (D(C t , t)-1) 2 +D(C′ t , t) 2

[0044] L2=∑ t≥1 (D(C′ t , t)-1) 2

[0045] D represents the calculation process of the Fresnel wave field discrimination module, performs back propagation on L1, and updates the parameters of the Fresnel wave field discrimination module; performs back propagation on L2, and updates the parameters of the group transform iterative denoising module;

[0046] (2.2) Neural connection strength contrast loss. In order to impose additional constraints on the group transform iterative denoising module, the neural connection strength contrast loss is introduced by measuring the connection strength difference between C0' after denoising and the true structural connection matrix C0, so that the group transform iterative denoising module remains stable during denoising. This loss function is defined as:

[0047] L3=|C0'-C0 |

[0048] Back-propagate L3 to update the parameters of the group transform iterative denoising module;

[0049] (2.3) Topological quotient space similarity loss. In order to further learn the local and global topological properties of the real structural connection matrix, a topological quotient space similarity loss function is designed. This loss function is defined as:

[0050] L4=PCC(C0',C0)+BC(C0',C0)

[0051] Among them, PCC represents the Pearson correlation coefficient, BC represents the betweenness centrality, L4 is back-propagated, and the parameters of the group transform iterative denoising module are updated.

[0052] Furthermore, the model test in step (3) includes inputting four-dimensional functional magnetic resonance imaging and a Gaussian noise matrix, iterating a denoising network through a group transform, and outputting a structural brain network of the target domain.

[0053] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: the computational method based on Huygens-Fresnel diffusion denoising proposed in the present invention can realize end-to-end generation of target domain brain network from source domain image data, improve computational efficiency and generation accuracy, and provide a data basis for multimodal fusion; combining the advantages of generative adversarial strategy and diffusion model, it can not only realize the efficient generation of cross-modal brain network, but also enhance the quality and diversity of brain network generation, which is conducive to multimodal fusion and analysis; taking source domain images and Gaussian matrices as the input of the model, mapping them to target domain brain networks through generative models, providing a data basis for multimodal Parkinson's disease diagnosis, and can also be applied to the diagnosis, analysis and treatment of other diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flow chart of the present invention;

[0055] Figure 2 This is a diagram of the cross-modal brain network computation framework based on Huygens-Fresnel spherical diffusion denoising;

[0056] Figure 3 This is a schematic diagram of the group transform iterative denoising module;

[0057] Figure 4Schematic diagram of the Fresnel wave field discrimination network. DETAILED DESCRIPTION

[0058] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0059] like Figure 1 As shown, the cross-modal generation method of brain network based on diffusion model of the present invention comprises the following steps:

[0060] (1) Construct a cross-modal model of brain networks based on Huygens-Fresnel spherical diffusion denoising;

[0061] The brain network cross-modal model based on Huygens-Fresnel spherical diffusion denoising is mainly composed of three modules: Huygens spherical diffusion module, group transform iterative denoising module and Fresnel wave field discrimination module. The Huygens spherical diffusion module includes forward diffusion and reverse diffusion, such as Figure 2 As shown in the figure, the group transform iterative denoising module and the Fresnel wave field discrimination module form a Fresnel equalization network. The details are as follows:

[0062] a. Huygens Spherical Diffusion Module:

[0063] The forward diffusion process gradually transforms the real structural connection matrix (C0) into a Gaussian noise matrix (C T ), where C0 represents the diffusion tensor imaging (DTI) matrix of size N*N obtained by preprocessing with the software toolbox, and N represents the number of brain regions. In each step, weights are applied to the Gaussian noise and superimposed on the real structural connectivity matrix. These weights are predefined as decimals that gradually increase from 0 to 1: β1, β2, β3, ..., β T To simplify the process, the calculation formula between two adjacent steps is:

[0064]

[0065] Where ∈′ represents the transpose of ∈, and N(0, 0.3) represents Gaussian distribution noise with mean 0 and variance 0.3. After t steps of diffusion, its structural connection matrix C t It can be expressed as:

[0066]

[0067] Here, t =1-β t , ∈ t The noise matrix sampled from N(0, 0.3) at time t is Gaussian distributed. The reverse diffusion process is the structural connection matrix C at time T. TThe mapping of the structural connection matrix C0 to time 0. In this process, the functional brain image fMRI is used as a condition to guide the Fresnel network to learn and remove Gaussian noise. Here, the reverse diffusion from time t to time t-1 is used as an example to illustrate. The Fresnel equalization network is represented by the symbol FNL. Assume

[0068] C t-1 '=FNL t (C t , fMRI, E t ) (3)

[0069] Here, the input parameters of the Fresnel equalization network FNL are three, namely, the structural connection matrix (C t , size is N*N), functional brain imaging (fMRI, size is V1*V2*V3*q), time embedding vector (E t , size is T*N*N); V1, V2, V3 represent the voxel size in three directions of functional magnetic resonance imaging, q represents the scanning time length of functional magnetic resonance imaging; the output structural connection matrix C t-1 The size is N*N.

[0070] b. Group transform iterative denoising module

[0071] like Figure 3 The figure shows the schematic diagram of the group transform iterative denoising module. The input of this module is the structural connection matrix (C t ), functional brain imaging (fMRI) and temporal embedding vectors (E t ), the output is the mechanism connection matrix C after denoising at time t-1 t-1 '.

[0072] Functional brain image fMRI is converted into ROI-based fuzzy time series (F1, size is N*q) through the discontinuous node feature extraction layer. This network layer multiplies the brain anatomical atlas (Atlas, size is V1*V2*V3) with the voxel information of each time point of fMRI. The whole brain is divided into N regions, and the feature dimension of each brain region is 1*q. The calculation formula is:

[0073] F1=fMRI⊙Atlas (4)

[0074] The bidirectional time-extended attention mechanism layer further explores the topological connection features of F1, fine-tunes the feature resolution between adjacent brain regions, and outputs the fine brain region feature F2, which is calculated as:

[0075]

[0076] Q=LM(F1), V=LM(F1), K1=LM(F1), K2=LM(F1) (6)

[0077] Where LM represents a linear mapping network, and K1' is the transpose of K1. The Soffmax(x) function performs row vector normalization on the variable x.

[0078] F2 and C t After the topology-aware convolutional network layer and the neural inhibition layer, the fusion feature F3 is output. Its calculation formula is:

[0079] F3=ReLU(C t ·F2·W1) (7)

[0080] Among them, ReLU is the activation function and W is the parameter to be learned.

[0081] F3, C t and E t After the topology-aware convolutional network layer and the neuron inhibition layer, the fusion feature F4 is output. Its calculation formula is:

[0082] F4=ReLU(C t ·F3·W2+E t ) (8)

[0083] F4 passes through the forward propagation layer and the neuron connection correction layer, and outputs the structural connection matrix C t-1 '. The calculation formula is:

[0084] F5=LM(F4) (9)

[0085] C t-1 '=σ(F5·F5') (10)

[0086] c. Fresnel wave field identification module

[0087] In order to make the structure connection matrix C output by the group transform iteration denoising module t-1 'As close as possible to the structural connection matrix C in the forward diffusion process t-1 ,like Figure 4 As shown in the figure, this module uses a cross convolution layer, a longitudinal convolution layer, a transverse convolution layer and a regression conversion layer in sequence, and outputs a scalar, 1 for true and 0 for false. When the scalar value is 0.5, it means that the training of this module is completed. The cross convolution layer (denoted by CF) uses two cross convolution kernels (sizes are: 1*N and N*1 respectively) to extract the bidirectional brain connection feature information U1 from the structural connection matrix. The longitudinal convolution layer (denoted by VF) and the transverse convolution layer (denoted by HF) respectively represent the 1*N and N*1 convolution kernels extracting features from the feature matrix output by the cross convolution layer (denoted by U 21 and U22 The regression transformation layer concatenates the features (denoted by U 21 ||U 22 +I t represents) is mapped to between 0 and 1, I t represents the time embedding vector at time t, with a size of Tx2N. Here, C t-1 ' Taking the Fresnel wave field discrimination module as an example, the following is the specific calculation formula:

[0088] U1=CF(C t-1 ') (11)

[0089] U 21 =VF(U1) (12)

[0090] U 22 =HF(U1) (13)

[0091] O=σ(LM(U 21 ||U 22 +I t )) (14)

[0092] (2) Model training

[0093] For each patient, fMRI and DTI were collected. The DTI was used to calculate the real structural connectivity matrix C0. The fMRI and structural connectivity matrix C0 were fed into the model and firstly passed through the Huygens forward diffusion process (T steps) to obtain the structural connectivity matrix C0 at each diffusion stage. t Then, after the Huygens reverse diffusion process (T steps), the structural connection matrix C after denoising at each stage is gradually obtained. t ', after T steps, a clean structural connection matrix C is obtained D '. The four loss functions of this training process are as follows:

[0094] Huygens-Fresnel learning loss:

[0095] By calculating the difference between the structural connection matrix of the reverse diffusion and the structural connection matrix of the forward diffusion process after denoising at each stage, the distribution consistency is constrained. This loss function is defined as:

[0096] L1=∑ t≥1 (D(C t , t)-1) 2 +D(C' t , t) (15)

[0097] L2=∑ t≥1 (D(C′ t , t)-1)2 (16)

[0098] D represents the calculation process of the Fresnel wavefield discrimination module. Back-propagation is performed on L1 to update the parameters of the Fresnel wavefield discrimination module. Back-propagation is performed on L2 to update the parameters of the group transform iterative denoising module.

[0099] Contrastive loss of neural connection strength:

[0100] In order to impose additional constraints on the group transform iterative denoising module, the neural connection strength contrast loss is introduced by measuring the difference in connection strength between C0' after denoising and the true structural connection matrix C0, so that the group transform iterative denoising module remains stable during denoising. This loss function is defined as:

[0101] L3=|C0'-C0| (17)

[0102] Back-propagate L3 and update the parameters of the group transform iterative denoising module.

[0103] Topological quotient space similarity loss:

[0104] In order to further learn the local and global topological properties of the real structural connection matrix, a topological quotient space similarity loss function is designed. This loss function is defined as:

[0105] L4=PCC(C0',C0)+BC(C0',C0) (18)

[0106] Among them, PCC represents the Pearson correlation coefficient and BC represents the betweenness centrality. Back propagation is performed on L4 to update the parameters of the group transform iterative denoising module.

[0107] The L subjects were divided into 5-fold cross-training strategies, with 80% of the subjects used for model training in each fold and the remaining 20% ​​used for testing. When the loss function value of the model training no longer changes, convergence is achieved, and the model has a strong ability to generate cross-modal brain networks.

[0108] (3) After model training is completed, model testing is performed.

[0109] After the diffusion model training is completed, the four-dimensional fMRI and Gaussian noise matrix are input, and the denoising network is iterated through group transformation to output the structural brain network of the target domain. The structural brain network can complete incomplete multimodal data and be used for multimodal fusion and diagnosis and analysis of Parkinson's patients. At the same time, this model can be used for modal data completion and multimodal fusion analysis of other cranial nerve diseases.

[0110] The present invention proposes a cross-modal brain network construction method based on Huygens-Fresnel spherical diffusion denoising. It utilizes the high-quality sample generation advantages of the diffusion model and the efficient sample generation advantages of the generative adversarial strategy to learn the complex mapping relationship between different modalities, and realizes an end-to-end mapping framework from brain images to brain networks, thereby improving the efficiency and quality of cross-modal brain network generation.

[0111] The present invention designs a Huygens sphere forward diffusion module, which uses the symmetric properties of the brain network to gradually add independently distributed Gaussian noise to the triangular area on the brain network, propagates and superimposes along the wavefront, and diffuses into a Gaussian matrix with uniform phase distribution. This module can establish a normalized fuzzy brain network and provide white noise samples for reverse diffusion denoising.

[0112] The present invention designs a Huygens spherical reverse diffusion module, uses brain images as conditions, guides the Fresnel equalization network to erase noise from the fuzzy brain network, establishes a nonlinear Gaussian mapping from the source domain to the target domain, captures the topological connection characteristics of the target domain brain network, and realizes accurate cross-modal brain network generation.

[0113] The present invention designs a group transformation iterative denoising module, which introduces a bidirectional time extension attention mechanism and embeds time prompts at the same time, mines the intrinsic correlation of the cross-modal topological quotient space, decouples irrelevant connection features within the modality in multiple stages, and improves the generation quality of the target domain brain network.

[0114] The present invention designs a Fresnel wavefield discrimination module, which introduces cross convolution, lateral convolution and longitudinal convolution networks to learn the far-field topological consistency and near-field temporal continuity of forward diffusion and reverse diffusion brain networks, thereby improving the generation diversity effect of the target domain brain network.

[0115] The present invention designs a topological quotient space similarity loss function, which further improves the generation effect of cross-modal brain networks and enhances the effectiveness of multimodal fusion by maximizing the global directional consistency of the topological field strength of the brain network and the uniformity of local equipotential surfaces.

Claims

1. A method for cross-modal generation of brain networks based on a diffusion model, characterized in that: The steps include: (1) Construct a cross-modal model of brain networks based on Huygens-Fresnel spherical diffusion denoising; (2) For each patient, two modalities, fMRI (functional magnetic resonance imaging) and DTI (diffusion tensor imaging), were collected to perform 5-fold cross-training on the constructed model. In each fold, 80% of the subjects were used to train the model, and the remaining 20% ​​were used for testing. When the loss function value of the model training no longer changed, convergence was achieved. (3) After model training is completed, model testing is performed.

2. The method for generating cross-modal brain networks based on a diffusion model according to claim 1, characterized in that: The brain network cross-modal model based on Huygens-Fresnel spherical diffusion denoising in step (1) includes three modules, namely, a Huygens spherical diffusion module, a group transform iterative denoising module and a Fresnel wave field discrimination module.

3. The method for generating cross-modal brain networks based on a diffusion model according to claim 2, characterized in that: The Huygens spherical diffusion module includes forward diffusion and reverse diffusion.

4. The method for generating cross-modal brain networks based on a diffusion model according to claim 3, characterized in that: The Huygens sphere forward diffusion is to gradually transform the real structure connection matrix C0 into a Gaussian noise matrix C through a sufficient number of T steps. T , where C0 represents the matrix of size N*N obtained by diffusion tensor imaging preprocessed by the software toolbox, N represents the number of brain regions, and C t is the structural connection matrix at time t. In each step, weights are applied to the Gaussian noise and superimposed on the real structural connection matrix. These weights are predefined as decimals that gradually increase between 0 and 1: β1, β2, β3, ..., β T , the calculation formula between two adjacent steps is: Among them, ∈′ represents the transpose of ∈, N(0, 0.3) represents Gaussian distribution noise with mean 0 and variance 0.3; after t steps of diffusion, its structural connection matrix C t It can be expressed as: Among them, ξ t =1-β t , ∈ t It represents the noise matrix sampled from N(0, 0.3) at time t, which obeys Gaussian distribution.

5. The method for generating cross-modal brain networks based on a diffusion model according to claim 3, characterized in that: The Huygens sphere reverse diffusion is the structural connection matrix C from time T T The mapping of the structural connection matrix C0 to time 0 is conditioned by the functional brain image to guide the Fresnel network to learn and remove Gaussian noise, which is illustrated by the reverse diffusion from time t to time t-1. The Fresnel equalization network is represented by the symbol FNL. Assume C t-1 =FNL t (C t ,fMRI,E t ) Among them, the input parameters of the Fresnel equalization network FNL are three, namely the structural connection matrix at time t, C t The size is N*N, the size of the functional brain image is V1*V2*V3*q, and the time embedding vector is E t The size is T*N*N; V1, V2, V3 represent the voxel size in three directions of functional magnetic resonance imaging, q represents the scanning time length of functional magnetic resonance imaging; the output structural connection matrix C t-1 The size is N*N.

6. The method for generating cross-modal brain networks based on a diffusion model according to claim 2, characterized in that: The input of the group transform iterative denoising module is the structural connection matrix C at time t t , functional brain images and temporal embedding vector E t , the output is the mechanism connection matrix C after denoising at time t-1 t-1 '; Functional brain images are converted into ROI-based fuzzy time series F1 through the discontinuous node feature extraction layer, with a size of N*q. This network layer multiplies the brain anatomical atlas Atlas, with a size of V1*V2*V3, with the voxel information of each time point of fMRI. The whole brain is divided into N regions, and the feature dimension of each brain region is 1*q. The calculation formula is: F1=fMRI ⊙ Atlas The bidirectional time-extended attention mechanism layer further explores the topological connection features of F1, fine-tunes the feature resolution between adjacent brain regions, and outputs the fine brain region feature F2, which is calculated as: Q=LM(F1), V=LM(F1), K1=LM(F1), K2=LM(F1) Among them, LM represents the linear mapping network, K1' is the transpose of K1, and the Softma*(*) function is to perform row vector normalization on the variable *; F2 and C t After the topology-aware convolutional network layer and the neural inhibition layer, the fusion feature F3 is output; its calculation formula is: <h2 style=";text-align:left;direction:ltr">F3 = ReLU(C<h2 style=";text-align:left;direction:ltr"> t <h2 style=";text-align:left;direction:ltr"> F2 W1) Among them, ReLU is the activation function and W is the parameter to be learned; F3, C t and E t After the topology-aware convolutional network layer and the neuron inhibition layer, the fusion feature F4 is output; the calculation formula is: F4=ReLU(C t ·F3·W2+E t ) F4 passes through the forward propagation layer and the neuron connection correction layer, and outputs the structural connection matrix C t-1 ', the calculation formula is: F5=LM(F4) C t-1 '=σ(F5·F5')。 7. The method for generating cross-modal brain networks based on a diffusion model according to claim 2, characterized in that: The Fresnel wave field discrimination module uses a cross convolution layer, a longitudinal convolution layer, a transverse convolution layer and a regression conversion layer in sequence, and outputs a scalar, 1 represents true, 0 represents false, and when the scalar value is 0.5, it means that the training of this module is completed. t-1 'Enter the Fresnel wave field discrimination module description, the following is the calculation formula: U1=CF(C t-1 ’) U 21 =VF(U1) U 22 =HF(U1) O=σ(LM(U 21 ||U 22 +I t )) Among them, CF represents the cross convolution layer, which uses two cross convolution kernels of size: U1 represents 1*N and N*1 to extract bidirectional brain connection feature information from the structural connection matrix, VF represents the longitudinal convolution layer, HF represents the horizontal convolution layer, and U 21 Indicates that the 1*N convolution kernel extracts features from the feature matrix output by the cross convolution layer, U 22 Indicates that the N*1 convolution kernel extracts features from the feature matrix output by the cross convolution layer; U 21 ||U 22 +I t It means that the regression transformation layer maps the concatenated features to between 0 and 1. t Represents the time embedding vector at time t, with a size of T*2N. σ is the Sigmoid activation function, which maps the input features to between 0 and 1; O is the output value of the Fresnel wavefield discrimination module. The closer its value is to 1, the more real the input sample is. If its value is closer to 0, the more fake the input sample is.

8. The method for cross-modal generation of brain networks based on a diffusion model according to claim 1, characterized in that: The step (2) includes collecting fMRI and DTI modalities for each patient, wherein the DTI is used to calculate the real structural connection matrix C0 through the software toolbox, and the fMRI and structural connection matrix C0 are sent to the model, and the structural connection matrix C0 is obtained at each diffusion stage through the Huygens forward diffusion process. t Then, through the Huygens reverse diffusion process, the structural connection matrix C after denoising at each stage is gradually obtained. t ', after T steps, a clean structural connection matrix C0' is obtained.

9. The method for cross-modal generation of brain networks based on diffusion model according to claim 1, characterized in that: The four loss functions in the model training process in step (2) are as follows: (2.1) Huygens-Fresnel learning loss, by calculating the difference between the structural connection matrix of the reverse diffusion and the structural connection matrix of the forward diffusion process after denoising at each stage, constrains their distribution consistency. This loss function is defined as: L2=∑ t≥1 (D(C′ t ,t)-1) 2 D represents the calculation process of the Fresnel wave field discrimination module, performs back propagation on L1, and updates the parameters of the Fresnel wave field discrimination module; performs back propagation on L2, and updates the parameters of the group transform iterative denoising module; (2.2) Neural connection strength contrast loss. In order to impose additional constraints on the group transform iterative denoising module, the neural connection strength contrast loss is introduced by measuring the connection strength difference between C0' after denoising and the true structural connection matrix C0, so that the group transform iterative denoising module remains stable during denoising. This loss function is defined as: L3=|C0'-C0 | Back-propagate L3 to update the parameters of the group transform iterative denoising module; (2.3) Topological quotient space similarity loss. In order to further learn the local and global topological properties of the real structural connection matrix, a topological quotient space similarity loss function is designed. This loss function is defined as: L4=PCC(C0',C0)+BC(C0',C0) Among them, PCC represents the Pearson correlation coefficient, BC represents the betweenness centrality, L4 is back-propagated, and the parameters of the group transform iterative denoising module are updated.

10. The method for generating cross-modal brain networks based on a diffusion model according to claim 2, characterized in that: The model test in step (3) includes inputting four-dimensional functional magnetic resonance imaging and a Gaussian noise matrix, iterating a denoising network through a group transform, and outputting a structural brain network of the target domain.

Citation Information

Cited By

  • Image data analysis method and system for radiology department

    CN120070440A

  • A method and system for radiology image data parsing

    CN120070440B

  • Individual heterogeneity elimination method based on cross-modal fusion and related equipment

    CN120105048A

  • Method and related device for eliminating individual heterogeneity based on cross-modal fusion

    CN120105048B