A novel fMRI site noise automatic correction algorithm using generative adversarial networks

Abstract

The present application belongs to the field of medical image processing and artificial intelligence, and proposes a new fMRI site noise automatic correction algorithm using a generative adversarial network, including a feature generator, a site noise feature generator, a feature fusion generator and a discriminator; compared with the traditional Fader network, the site label automatic coding module is added, which can automatically project and encode the site label of the data into an image, and separate it from the original image, thereby realizing the automatic correction of the multi-center effect of the fMRI data, avoiding the artificial site noise coding assumption, and more accurately identifying and eliminating the multi-center effect.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of medical image processing and artificial intelligence, and in particular to a novel automatic noise correction algorithm for fMRI sites using generative adversarial networks. Background Technology

[0002] Multicenter fMRI data fusion analysis can address the problem of insufficient data from a single center. By jointly processing multiple sets of fMRI data, it improves data quality, eliminating biases caused by individual data points and significantly enhancing statistical results. However, multicenter data fusion itself is controversial. Factors such as differences in the brand or model of the machines used to acquire MRI images, variations in acquisition parameters, and differences in acquisition environments can all cause discrepancies in the final MRI images, known as site noise. The impact of site noise can even be more significant than the impact of differences of interest. Direct fusion of multicenter data can lead to serious bias in research results and significantly confound the findings.

[0003] Currently, the main models for removing site noise can be divided into three categories: parametric regression methods (Chen H et al. Parsingbrain structural heterogeneity in males with autism spectrum disorder reveals distinct clinical subtypes[J]. Hum Brain Mapp, 2019, 40(2): 628-637.), component analysis methods (Cong F et al. Can back-projection fully resolve polarity indeterminacy of independent component analysis in study of event-related potential?[J]. BiomedSignal Process Control. 2011, 6(4): 422-426.), and deep learning methods (Tian D et al. A deep learning-based multisite neuroimage harmonization framework established with a traveling-subject dataset[J]. NeuroImage, 2022, 257: 119297.). Parametric regression methods primarily designate site noise as an uninteresting variable and then use a general linear model or a variation thereof to regress these uninteresting variables, thereby eliminating the influence of site effects on fMRI data. A representative component analysis method is Independent Component Analysis (ICA). ICA performs blind source signal separation on the spatial information of the data, obtaining statistically independent spatial components. It analyzes the correlation between the subject contribution coefficient of each independent component and the site variable, using the spatial components that are only significantly correlated with site noise as the brain structure and function atlas of site noise, and the corresponding subject coefficients as the specific influence of site noise on each subject. Deep learning methods utilize a pre-defined objective function mechanism to transform the representation of data; simply put, it represents the original domain data using the feature space of the target domain. A relatively mature algorithm of this type is Generative Adversarial Networks (GANs), which can use two networks to learn adversarially, allowing the networks to continuously learn and optimize themselves, thereby achieving feature style transfer transformation from the original domain data to the target domain data. Using generative adversarial networks to eliminate multicenter effects in fMRI data can avoid correction errors caused by human prior assumptions, because both parametric regression methods and data-driven ICA methods require human assumptions about the parametric form of site noise. Summary of the Invention

[0004] To address the multicenter effect problem in raw fMRI data acquired from multiple centers, a novel automatic site noise correction algorithm using generative adversarial networks (GANs) is designed to solve this problem. Compared to traditional Fader networks, this invention incorporates an automatic site label encoding module, which automatically projects and encodes site labels from the data into images and separates them from the original images. This achieves automatic correction of multicenter effects in fMRI data. This invention avoids manual assumptions about site noise encoding and can more accurately identify and eliminate multicenter effects.

[0005] The technical solution of the present invention:

[0006] A novel automatic site noise correction algorithm for fMRI using generative adversarial networks includes three generators and one discriminator D. site The three generators are a feature generator G1, a site noise feature generator G2, and a feature fusion generator G3. The generators and discriminator are mainly constructed from modules U, D, and R. The U module includes 3D deconvolution layers, instance normalization layers, and the LeakyReLu activation function, which is responsible for dimensionality augmentation of the image. The D module includes 3D convolution layers, instance normalization layers, and the LeakyReLu activation function, which is responsible for dimensionality reduction of the image, extracting deeper features without losing information. The R module adopts a residual network architecture, which includes 3D convolutional layers, instance normalization layers, and LeakReLu() activation functions, all with two layers. The feature generator G1 consists of four D modules connected in series and one R module. The site noise feature generator G2 consists of two D modules and two R modules, with the D and R modules alternating. The feature fusion generator G3 consists of one D module, four R modules, three U modules, one 3D convolutional layer, and one tanh layer, with the R and U modules alternating. The discriminator D... site It includes four groups of alternately connected D and R modules, one average pooling layer, and one fully connected layer;

[0007] A novel automatic site noise correction algorithm for fMRI using generative adversarial networks includes the following steps:

[0008] Step 1: Slice the raw fMRI image data according to the time dimension, randomly shuffle the sliced ​​image data, and feed it into the feature generator G1 to generate features without site-specific information.

[0009]

[0010] Where x is a spatial image of a specific frame from the original fMRI image data. It is the result output by the feature generator G1;

[0011] Step 2: The site label numbers corresponding to the sliced ​​image data are sent to the site noise auto-encoding module to generate the final site labels. The site noise auto-encoding module includes a dimension mapping module composed of fully connected layers and a site noise feature generation module G2. The final site labels are mapped to the site noise image features of the corresponding sites.

[0012]

[0013] in, This represents the output after the site label passes through the fully connected layer, where y represents the site label. W1 and b1 represent the weights and biases of the first fully connected layer, W2 and b2 represent the weights and biases of the second fully connected layer, n represents the number of sites, and z represents the product of the spatial dimensions of the fMRI image data; LeakyReLu() represents the activation function with a slope of -0.2 on the negative x-axis; after projection mapping through the fully connected layer, the output is transformed by dimension and then fed into the site noise feature generation module G2 for generation.

[0014]

[0015] Step 3: The results from feature generator G1 and the site noise auto-encoding module are concatenated and fed into feature fusion generator G3 to generate the reconstructed original fMRI image.

[0016]

[0017]

[0018] in, express and The purpose of the fused and reconstructed data is to constrain the direction of model optimization, ensuring that the information expressed in the final generated image does not deviate from the original fMRI image; θ g1 θ g2 and θ g3 represents the model parameters of generators G1, G2, and G3, respectively; m represents the number of samples, and S represents the original feature space of the sample data.

[0019] In step one, the feature generator G1 generates features that do not have site feature information. This was obtained through iterative calculations, specifically:

[0020] The output of feature generator G1 is fed into site noise identifier D. site During classification training, the site noise identifier D site Identify residual features The site feature information is used, and the k-th index is used to index the attributes of the output result. The learned site noise information is used to back-update the feature generator G1, so that the feature generator G1 generates features that do not contain site feature information. The site noise auto-encoding module supplements the site feature information lost by the feature generator G1, and restores and reconstructs the original fMRI image;

[0021]

[0022] in, This represents the probability of correctly classifying a sample, where k represents the index of the classified sample.

[0023] D site The loss function is:

[0024]

[0025] Where, (θ d |θ g1 This indicates that the discriminator D is updated while the parameters of the feature generator G1 remain unchanged. site Model parameters;

[0026] Without losing sample specificity, discriminator D site The loss function for the reverse update feature generators G1, G2, and G3 is:

[0027]

[0028] Where, (θ g1 ,θ g2 ,θ g3 |θ d ) indicates that in the discriminator D site With the model parameters unchanged, update the model parameters of generators G1, G2, and G3; λ rec λ is the weighting coefficient of the reconstructed image. d It is the weighting coefficient for the amount of site feature information eliminated.

[0029] The weighting coefficient λ d In discriminator D site During the training process, it increases linearly from 0 to 0.0001.

[0030] The beneficial effects of this invention are as follows: Using the algorithm of this invention, site noise remaining in fMRI images can be removed while ensuring sample specificity. At the same time, this algorithm can also use the site label auto-encoding module to identify and visualize the noise of different sites, providing image support for subsequent site noise removal work, and also providing more reliable technical support for subsequent scientific research based on fMRI images. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the framework of the novel automatic fMRI site noise correction algorithm using generative adversarial networks according to the present invention.

[0032] Figure 2 This is a schematic diagram showing the specific details of the U module, D module, and R module of the present invention;

[0033] Figure 3 This is a schematic diagram of the generator and discriminator architecture of the present invention.

[0034] In the picture: - Convolution kernel, - Instance normalization layer - Fully connected layer - Site tags. Detailed Implementation

[0035] To further illustrate the advantages of this algorithm, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0036] Figure 1 This is a diagram illustrating the overall framework of a novel automatic fMRI site noise correction algorithm using generative adversarial networks, as described in an embodiment of the present invention. Figure 1 As shown, this embodiment of the invention provides an implementation idea for an automatic site noise correction algorithm for raw fMRI data. The overall implementation idea can be divided into three steps:

[0037] Step 1: Since this algorithm assumes that site noise is time-invariant during fMRI image acquisition, the original fMRI data is sliced ​​along the time dimension, ensuring that the image retains only the spatial dimension information. The sliced ​​data is then randomly shuffled and fed into feature generator G1 to generate features without site-specific information.

[0038] Step 2: The site labels corresponding to the sliced ​​data are fed into the site noise auto-encoding module. This module consists of two parts: the first part is a dimension mapping module composed of fully connected layers, and the second part is a site noise feature generation module G2 composed of 3D convolutional layers. Finally, the site labels will be specifically mapped to the site noise image features of the corresponding site.

[0039] Step 3: The results from feature generator G1 and the site noise auto-encoding module are concatenated and fed into feature fusion generator G3 to generate the reconstructed original fMRI image. This step is the most powerful constraint mechanism. This constraint adjustment can prevent the image generator from generating images that meet the conditions but are incorrect, while also preserving sample specificity as much as possible.

[0040] Feature generator G1 generates features that do not contain site feature information. The result is obtained through iterative calculations; the output of the feature generator G1 is then fed into the site noise identifier D. site In the middle, the discriminator D site Its function is to identify residual features The system uses the learned site noise information to update the feature generator G1, enabling G1 to generate features free from site noise. Meanwhile, the site noise autoencoding module fills in the missing site noise information, thus enabling the reconstruction of the original fMRI image. Due to some instability during the initial training process, the weight coefficients λ of the site noise removal component are adjusted. d The value will start from 0 and increase linearly to 0.0001. The purpose of this is to ensure that the model's primary optimization goal at the beginning is to reconstruct the image rather than remove site noise. This can minimize the risk of model training collapse.

[0041] Figure 2The diagram illustrates the details of the U, D, and R modules. The U module is responsible for dimensionality augmentation of the image, with 3D deconvolution layers used to increase image resolution. The D module reduces the image's height and width dimensions, adhering to the principle of reducing these dimensions while increasing the dimension of the image's feature channels. This is done to improve model speed and extract deeper features without losing information. The R module is the core component of the deep learning framework. This module uses a residual network architecture and consists of two convolutional layers. The residual architecture is primarily used to avoid the gradient vanishing problem. The network architecture is used to build modules. When the module network structure is very deep, the gradient backpropagation will suffer from the vanishing problem, so this structure is essential when building deep modules. Instance normalization layers are used in the modules. The purpose of using these normalization layers is to preserve the differences between time points as much as possible, thereby restoring the image details at each time point as much as possible. Since this algorithm assumes that the site noise is time invariant, the independence between samples should also be ensured as much as possible during training. The LeakReLu() activation function is used in the modules to improve their expressive power, thereby making the training of the modules more stable.

[0042] Figure 3 This is a schematic diagram of the generator and discriminator architecture used in the algorithm. Figure 3 In the middle, the generator and discriminator are built using Figure 2 The modules constructed in this paper, in the feature generator G1, progressively decrease the height and width dimensions of the spatial image while progressively increasing the number of feature channels. Finally, the residual R module in layer 6 ensures model depth, thereby guaranteeing that the extracted features have a higher-level representation. The convolutional module in the site noise autoencoding module performs depth processing on the site noise projection mapped by the fully connected layers, allowing for better fusion with the features extracted by feature generator G1 to reconstruct the original fMRI data. The feature fusion generator G3 is mainly responsible for fusing the site noise-free feature data with the site noise data to reconstruct the original fMRI data. Therefore, the feature fusion generator G3 must have a sufficiently deep network structure to ensure the model's expressive power and feature fusion ability. Simultaneously, the feature fusion generator G3 must also have the function of improving image resolution. Therefore, every few residual R modules, the U module performs an image spatial dimension increase operation to restore the features to an image with specified site noise. The discriminator D... site The main goal is to identify residual features The site noise information is used, and the feature generator G1 is guided to eliminate site noise through reverse updates. During this training process, the discriminator D... siteIt plays a guiding role, directing the feature generator G1, the site noise feature generator G2, and the feature fusion generator G3. The discriminator and the generator complement each other during the training process and play a decisive role in the final model optimization. They are all indispensable.

Claims

1. A novel fMRI site noise automatic correction algorithm using a generative adversarial network, characterized in that, The new fMRI site noise automatic correction algorithm comprises three generators and a discriminator D site The three generators are respectively a feature generator G1, a site noise feature generator G2 and a feature fusion generator G3. The generator and the discriminator are mainly jointly built by a U module, a D module and an R module. The U module comprises a 3D deconvolution layer, an instance normalization layer and a LeakyReLu activation function, is responsible for dimension expansion of an image, the D module comprises a 3D convolution layer, an instance normalization layer and a LeakyReLu activation function, is responsible for dimension reduction of the image, and extracts deeper features on the premise of not losing information quantity, and the R module adopts a residual network architecture, comprises a 3D convolution layer, an instance normalization layer and a LeakReLu() activation function, and is provided with double layers. The feature generator G1 comprises four D modules connected in series and one R module. The site noise feature generator G2 comprises two D modules and two R modules, and the D modules and the R modules are connected alternately. The feature fusion generator G3 comprises one D module, four R modules, three U modules, one 3D convolution layer and one tanh layer, wherein the R modules and the U modules are connected alternately. The discriminator D site comprises four groups of alternately connected D module and R module groups, one average pooling layer and one full connection layer. The steps include the following: Step one: slice the original fMRI image data according to the time dimension, randomly shuffle the sliced image data, and send it into the feature generator G1 to generate features without site feature information where x is a certain frame spatial image of the original fMRI image data, is the result output by the feature generator G1; Step two: send the site label number corresponding to the sliced image data into the site noise automatic coding module to generate the final site label; the site noise automatic coding module includes a dimension mapping module composed of a fully connected layer and a site noise feature generation module G2, and the final site label is mapped into the site noise image feature of the corresponding site wherein, represents the output result of the site label after the full connection layer, y represents the site label, W1 and b1 represent the weight and bias of the first layer full connection layer, W2 and b2 represent the weight and bias of the second layer full connection layer, n represents the number of sites, z represents the product of each dimension of the fMRI image data space dimension, and LeakyReLu() represents an activation function with a slope of -0.2 on the negative half axis of x; after projection mapping through the full connection layer, the output result is sent to the site noise feature generation module G2 part after dimension transformation Step three: the feature generator G1 is spliced with the output of the site noise auto-encoding module to generate the reconstructed original fMRI image by the feature fusion generator G3 wherein, denotes with data after fusion reconstruction; and denote the model parameters of the generators G1, G2 and G3, respectively; m denotes the number of samples, and S denotes the original feature space of the sample data.

2. The novel fMRI site noise auto-correction algorithm using a generative adversarial network of claim 1, wherein, The feature generator G1 in the step one generates features without site feature information is obtained through iterative calculation, specifically: The result output by the feature generator G1 is sent to the site noise identifier D site The site noise identifier D is trained by classification in the middle site The site feature information remaining in the feature The attribute of the output result is indexed by the k-th, and the feature generator G1 is updated in reverse by the learned site noise information, so that the feature generator G1 generates features without site feature information The site noise auto-encoding module supplements the site feature information lost by the feature generator G1, and restores the original fMRI image wherein, represents the classification correct probability of the sample, and k represents the index of the classified sample; D site The loss function for the model of FIG. 1 is: wherein, represents updating the model parameters of the discriminator D site with the feature generator G1 parameters unchanged. Discriminator D site The loss function of the reverse update feature generator G1, G2 and G3 is: wherein, represents in the discriminator D site updating the model parameters of the generators G1, G2 and G3 under the condition that the model parameters are unchanged; λ rec is a weight coefficient of the reconstructed image, λ d is a weight coefficient of the site feature information elimination amount.

3. The novel fMRI site noise auto-correction algorithm using a generative adversarial network according to claim 1 or 2, characterized in that, The weight coefficient λ d In the training process of the discriminator D site is linearly increased from 0 to 0.0001.

Images