Registration generating fMRI data system with attention matching features
By designing an adversarial generation fMRI data system with attention matching features, and using neural network structure and feature matching layer to generate high-quality fake fMRI data, the problem of low simulation quality of fMRI data in the prior art and inability to reflect brain mutation characteristics is solved, and more efficient data analysis and identification efficiency is achieved.
Patent Information
- Application Number
- CN202411903407.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-23
- Publication Date
- 2025-04-15
AI Technical Summary
The existing fMRI data simulation technology is of poor quality and cannot effectively reflect the brain mutation characteristics of people with different degrees of cognitive impairment, and cannot truly reflect the complex correlation and noise information between various brain regions of the human brain.
An adversarial generation fMRI data system with attention matching features is designed, including fMRI BOLD signal simulation module, fMRI generator module, fMRI signal acquisition module and fMRI judge module. The random data is processed through the neural network structure to generate a forged fMRI signal, and the feature distance between the real fMRI and the forged fMRI is minimized through the feature matching layer.
The generation of high-quality fake fMRI data can effectively reflect the mutation characteristics of the brains of people with different degrees of cognitive impairment, and significantly improve the recognition efficiency and data analysis value.
Smart Images
Figure CN119833152A_ABST
Abstract
Description
Technical Field
[0001] The present invention specifically relates to an adversarial generation of fMRI data system with attention matching features. Background Art
[0002] The existing simulation methods of fMRI data are single and of varying quality. The main implementation method is to construct brain area signals similar to fMRI data based on random time series signals and the correlation or causality between variables using the Matlab simulation toolbox.
[0003] The main defects of existing fMRI data simulation technology are: using Matlab time series to simulate random variables, the quality is poor, especially it cannot reflect the brain variation characteristics of people with different degrees of cognitive impairment, the correlation or causality between variables is more deliberate, it cannot truly reflect the complex relationship between the various brain regions of the human brain, and it cannot reflect noise information. Therefore, the data analysis value in cognitive neuroscience is not high, and the efficiency of identifying brain working modes is low. Summary of the invention
[0004] The present invention provides an adversarial generation fMRI data system with attention matching features to solve the above-mentioned technical problems, and specifically adopts the following technical solutions:
[0005] A system for adversarially generating fMRI data with attention matching features, including:
[0006] fMRI BOLD signal simulation module, used to generate random data of resting-state fMRI signals;
[0007] An fMRI generator module, configured to generate a forged fMRI signal based on random data generated by the fMRI BOLD signal simulation module;
[0008] fMRI signal acquisition module, used to collect real fMRI signals;
[0009] The fMRI decision module receives the forged fMRI signal and the real fMRI signal, and makes a authenticity judgment on the input information and feeds back the judgment result to the fMRI generator module, and the fMRI generator module adjusts the parameters based on the feedback result.
[0010] Furthermore, the fMRI BOLD signal simulation module uses random pink noise as a simulation of resting-state fMRI to simulate the spontaneous fluctuation of low-frequency BOLD signals. The fMRI BOLD signal simulation module generates random data of resting-state fMRI signals as follows:
[0011]
[0012] Where f is the frequency, e is the exponent, φ(f) is a random phase uniformly distributed in the interval [0,2π], and IFFT stands for inverse Fourier transform.
[0013] Further, after the random data of the resting-state fMRI signal is generated by the fMRI BOLD signal simulation module, m time points are intercepted and input into the fMRI generator module;
[0014] After the signal acquisition module acquires the real fMRI signal, m time points are intercepted and input into the fMRI decision module.
[0015] Furthermore, m is greater than or equal to 50 and less than or equal to 200.
[0016] Furthermore, after acquiring the real fMRI signal, the fMRI signal acquisition module uses the HCP MMP cerebral cortex partitioning method to align the fMRI to the sMRI, and divides the data into 180 left and right brain regions, for a total of 360 brain region data.
[0017] Furthermore, the fMRI generator module processes the random data generated by the fMRI BOLD signal simulation module through a first neural network structure to generate a forged fMRI signal.
[0018] Furthermore, the first neural network structure includes: an input layer, a block embedding layer, n layers of TransformerBlock encoding blocks and an output layer.
[0019] Furthermore, the fMRI decision module determines the authenticity of the input information based on the second neural network structure.
[0020] Furthermore, the second neural network structure includes: an input layer, a category token, a block embedding layer, an n-layer Transformer Block encoding block, a Transformer feature matching layer and an output layer, and the Transformer feature matching layer is used to minimize the feature distance between the real fMRI and the forged fMRI.
[0021] Furthermore, the fMRI decision module outputs the feature distance between the real fMRI signal and the forged fMRI signal through its Transformer feature matching layer, and the fMRI generator module modifies the model weights layer by layer according to the back-propagation algorithm to minimize the feature distance.
[0022] The adversarial generated fMRI data system with attention matching features provided by the present invention can well control the difference between forged fMRI and real fMRI, so that the generator and the judge can avoid blindly and randomly searching for the global optimal solution during the adversarial process, but instead take minimizing the feature matching layer as the overall goal. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0024] Figure 1 It is a schematic diagram of an adversarial generation of fMRI data system with attention matching features of the present invention. DETAILED DESCRIPTION
[0025] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.
[0026] like Figure 1 The present invention shows an adversarial generation fMRI data system with attention matching features. In order to adversarially generate fMRI data of patients with different degrees of cognitive impairment and verify the validity of the generated fMRI data, as well as the common features between the generated fMRI and the real fMRI, the adversarial generation fMRI data system with attention matching features of the present invention includes: an fMRI BOLD signal simulation module, an fMRI generator module, an fMRI signal acquisition module and an fMRI judgement module.
[0027] The fMRI BOLD signal simulation module is used to generate random data of resting-state fMRI signals. The fMRI generator module is used to generate fake fMRI signals based on the random data generated by the fMRI BOLD signal simulation module. The fMRI signal acquisition module is used to acquire real fMRI signals. The fMRI judge module receives fake fMRI signals and real fMRI signals, and the fMRI judge module judges the authenticity of the input information and feeds back the judgment result to the fMRI generator module, which adjusts parameters based on the feedback result.
[0028] In order to more effectively guide the neural network to generate realistic brain fMRI signals of patients with different degrees of cognitive impairment, in the implementation of the present application, the fMRI BOLD signal simulation module uses random pink noise as a simulation of resting-state fMRI to simulate the spontaneous fluctuations of low-frequency BOLD signals. This is different from the simulation of task-state fMRI signals. It does not require additional convolution HRF functions to establish the association between external stimuli and blood oxygen level signals. Instead, pink noise, which is widely present in natural systems and increases in spectral energy density as the frequency decreases, is directly used as a representative of rs-fMRI. The mathematical process of the fMRI BOLD signal simulation module to generate random data of resting-state fMRI signals is shown in the following formula:
[0029]
[0030] Where f is the frequency, e is the exponent, φ(f) is a random phase uniformly distributed in the interval [0,2π], and IFFT stands for inverse Fourier transform. First, a frequency array from 0 to half the sampling rate is generated to remove the zero-frequency DC component to avoid The power spectrum of these frequency components is calculated, and a random phase with the same length as the frequency array is generated. The time domain signal s(t) is obtained by inverse fast Fourier transform as a simulation of resting state fMRI.
[0031] In the implementation of the present application, after the fMRI BOLD signal simulation module generates random data of the resting state fMRI signal, m time points are intercepted and input into the fMRI generator module. After the signal acquisition module acquires the real fMRI signal, m time points are intercepted and input into the fMRI decision module. Preferably, m is greater than or equal to 50 and less than or equal to 200. In the implementation of the present application, m is 100.
[0032] It can be understood that, in order to be consistent with the real fMRI signal, the length of the time domain signal is defined as 100 time points, as is the length of the real fMRI signal.
[0033] In the implementation of the present application, after collecting the real fMRI signal, the fMRI signal acquisition module uses the HCP MMP cerebral cortex partitioning method to align the fMRI to the sMRI, which is divided into 180 left and right brain regions, with a total of 360 brain region data. Therefore, the total number of fMRI signals simulated by this method is 360 groups, corresponding to 360 brain regions, and fMRI simulation data of 100 time points for each brain region. In order to further batch train these data in the neural network, this method sets the batch size to the commonly used 32, so the output data shape of the fMRI signal acquisition module is (32, 100, 360).
[0034] It is understandable that the fMRI signal acquisition module downloads fMRI data from the ADNI database. The fMRI data downloaded from the ADNI database cannot be used directly for model training. The above partitioning method needs to be used to divide the fMRI into 360 brain regions. The preprocessed fMRI data is a 360x 100 size array. This array has the same scale as the forged fMRI data, so the mixing method is very simple, just feed it together to the model.
[0035] In the implementation of the present application, the fMRI generator module processes the random data generated by the fMRI BOLD signal simulation module through the first neural network structure to generate a forged fMRI signal. The fMRI judge module judges the authenticity of the input information based on the second neural network structure.
[0036] In an embodiment of the present application, the first neural network structure and the second neural network structure are both based on the VTFF model. Specifically, the first neural network structure includes: an input layer, a block embedding layer, an n-layer Transformer Block encoding block and an output layer. The second neural network structure includes: an input layer, a category token, a block embedding layer, an n-layer Transformer Block encoding block, a Transformer feature matching layer and an output layer. The main difference between the second neural network structure and the first neural network result is that there is an additional Transformer feature matching layer. The Transformer feature matching layer is used to minimize the feature distance between the real fMRI and the forged fMRI.
[0037] In the implementation of the present application, the VTFF model mainly includes an input layer, a block embedding layer, a stack of 12 layers of TransformerBlock and an output layer. Among them, whether it is real fMRI or fake fMRI, the scale is 360x100. These mixed data are fed to the model input layer, and then passed through the block embedding layer, they are flattened and expanded to a size of 36000, plus a class token. The size of the class token is 360x 1, representing the feature values of 360 brain regions. Then, the 12-layer Transformer structure is used for stacking and model training, and the features of each layer and each brain region are gradually summarized to the classtoken. Finally, in the output layer, softmax is used to achieve multi-classification. In the fMRI decision module, the role of the Transformer feature matching layer is to minimize the feature distance between the real fMRI and the fake fMRI. The smaller the distance, the more the fake fMRI looks like the real fMRI.
[0038] In the implementation of the present application, the judgment result is represented by the characteristic distance of the feature matching layer, that is, the difference between the forged fMRI and the real fMRI. The smaller the distance, the higher the degree of forgery. The fMRI judge module outputs the characteristic distance between the real fMRI signal and the forged fMRI signal through its Transformer feature matching layer. The fMRI generator module can see this result and modify the model weights layer by layer according to the back propagation algorithm to minimize the characteristic distance. This process is automatic and uses the back propagation function of Pytorch, which is a routine operation for neural network model training.
[0039] As mentioned above, feature matching technology is used to use the output of a certain intermediate layer of the fMRI discriminator module as a sample feature representation to measure the feature difference between the real sample and the forged sample, as shown in the following formula:
[0040]
[0041] Among them, TB represents the last layer of Transformer Block as the feature matching layer, E is the expected value, and x real is the real data, p real is the real data distribution, and f is the feature representation of the input data by this layer. In the implementation, the degree of feature matching difference can be used to guide the tuning direction of the loss function of the generator in the network. Therefore, in the iterative process, the updated fMRI generator module Loss is minimized as follows:
[0042]
[0043] Among them, λ is the feature matching coefficient, which is used to evaluate the network loss and the impact of different degrees of Transformer feature matching. Since the discriminator is designed to output higher values for real samples and lower values for generated samples, the goal of the generator is to minimize the loss of generated samples, that is, There's a minus sign in front of it.
[0044] In the fMRI adversarial generation process of the implementation method of the present application, the presence of the attention matching layer can well control the difference between the forged fMRI and the real fMRI, so that the generator and the judge avoid blindly and randomly searching for the global optimal solution in the adversarial process, but take minimizing the feature matching layer as the overall goal. At the same time, according to the brain images of patients with different degrees of cognitive impairment in the real fMRI data, different degrees of forged fMRI data can be trained and generated in the feature matching process. For this reason, the present invention can be used to generate high-quality adversarial fMRI simulation data, and can also simulate fMRI time series of brains with different degrees of cognitive impairment, which brings significant effects for further analyzing and comparing the differences between forged and real fMRI features.
[0045] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any form, and any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.
Claims
1. A system for adversarial generation of fMRI data with attention matching features, characterized in that Include: fMRI BOLD signal simulation module, used to generate random data of resting-state fMRI signals; An fMRI generator module, configured to generate a forged fMRI signal based on random data generated by the fMRI BOLD signal simulation module; fMRI signal acquisition module, used to collect real fMRI signals; The fMRI decision module receives the forged fMRI signal and the real fMRI signal, and makes a authenticity judgment on the input information and feeds back the judgment result to the fMRI generator module, and the fMRI generator module adjusts the parameters based on the feedback result.
2. The adversarial generation of fMRI data with attention matching features according to claim 1, characterized in that The fMRI BOLD signal simulation module uses random pink noise as a simulation of resting-state fMRI to simulate the spontaneous fluctuation of low-frequency BOLD signals. The fMRI BOLD signal simulation module generates random data of resting-state fMRI signals as follows: Where f is the frequency, e is the exponent, φ(f) is a random phase uniformly distributed in the interval [0,2π], and IFFT stands for inverse Fourier transform.
3. The adversarial generation of fMRI data with attention matching features according to claim 1, characterized in that After the random data of the resting-state fMRI signal generated by the fMRI BOLD signal simulation module, m timepoints are intercepted and input into the fMRI generator module; After the signal acquisition module acquires the real fMRI signal, m time points are intercepted and input into the fMRI decision module.
4. The system for adversarially generating fMRI data with attention matching features according to claim 3, characterized in that: m is greater than or equal to 50 and less than or equal to 200.
5. The system for adversarial generation of fMRI data with attention matching features according to claim 1, characterized in that: After acquiring the real fMRI signal, the fMRI signal acquisition module uses the HCP MMP cerebral cortex partitioning method to align the fMRI to the sMRI, and divides the data into 180 left and right brain regions, for a total of 360 brain region data.
6. The system for adversarially generating fMRI data with attention matching features according to claim 1, characterized in that: The fMRI generator module processes the random data generated by the fMRI BOLD signal simulation module through a first neural network structure to generate a forged fMRI signal.
7. The system for adversarially generating fMRI data with attention matching features according to claim 6, characterized in that: The first neural network structure includes: an input layer, a block embedding layer, n layers of Transformer Block encoding blocks and an output layer.
8. The system for adversarially generating fMRI data with attention matching features according to claim 7, characterized in that: The fMRI decision module determines the authenticity of the input information based on the second neural network structure.
9. The system for adversarially generating fMRI data with attention matching features according to claim 8, characterized in that: The second neural network structure includes: an input layer, a category token, a block embedding layer, an n-layer Transformer Block encoding block, a Transformer feature matching layer and an output layer, and the Transformer feature matching layer is used to minimize the feature distance between the real fMRI and the forged fMRI.
10. The system for adversarially generating fMRI data with attention matching features according to claim 9, characterized in that: The fMRI decision module outputs the feature distance between the real fMRI signal and the fake fMRI signal through its Transformer feature matching layer, and the fMRI generator module modifies the model weights layer by layer according to the back propagation algorithm to minimize the feature distance.