A method and device for fMRI data consistency correction based on depth domain adaptation

By employing deep learning domain adaptation techniques and utilizing autoencoder networks and moment matching algorithms, the problem of low consistency in multi-batch fMRI data across multiple days was solved, achieving data consistency correction and improving data quality to support the construction of more efficient visual coding models.

CN118981471BActive Publication Date: 2026-11-17Chinese People's Liberation Army Cyberspace Force Information Engineering University
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410991914.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2026-11-17
Estimated Expiration
2044-07-23

AI Technical Summary

Technical Problem

The lack of consistency in fMRI data collected and processed in multiple batches over a span of days limits the sample size of large-scale fMRI datasets and the application of deep learning in the field of visual coding.

Method used

By employing deep learning domain adaptation techniques, intermediate feature information is extracted through an autoencoder network, and a moment matching algorithm is introduced to reduce the moment spacing between different data domains, thereby achieving domain-independent representation and consistency correction of fMRI data.

Benefits of technology

This improved the overall consistency of fMRI data acquired over multiple days, providing a high-quality data foundation for the subsequent construction of interpretable coding models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118981471B_ABST
    Figure CN118981471B_ABST
Patent Text Reader

Abstract

This invention discloses a method and apparatus for fMRI data consistency correction based on deep domain adaptation. Building upon 7T fMRI preprocessed data provided by the Kay team, it treats data acquired multiple times over a year as data domains from different sources. An autoencoder network is used to input fMRI voxel response data from specific brain regions, extracting key intermediate features. Corrected fMRI voxel response data is then reconstructed based on these intermediate features. During data reconstruction, a moment matching algorithm is introduced to reduce the moment spacing between data from different domains in the autoencoder's intermediate feature space, achieving domain-independent representation of the fMRI data and completing the consistency correction. By treating fMRI data acquired multiple times over a year as data domains from different sources, and innovatively employing deep learning domain adaptation technology, the overall consistency of fMRI data acquired over multiple days is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of fMRI visual function data processing technology, and in particular to a method and apparatus for fMRI data consistency correction based on depth domain adaptation. Background Technology

[0002] fMRI datasets, as observational data of neural activity in the visual cortex of the brain, are fundamental conditions for studying visual information encoding. Their scale and quality directly affect the quality of visual encoding model construction and analysis. For example... Figure 1 As shown, a natural image is presented to the subject, stimulating and activating the visual cortex. fMRI records the BOLD signals associated with neural activity. Through a series of preprocessing operations such as 3D reconstruction, registration, and head movement correction, the response values ​​of each voxel of interest in the 3D space of the visual cortex are obtained. The numerous voxel responses from various visual cortical regions form a voxel response vector, which is the fMRI data corresponding to the image stimulus. In this way, images are continuously presented to the subject from the natural image dataset, thereby constructing the fMRI dataset corresponding to the natural image stimulus dataset. Each sample pair includes an image and the voxel response vector of the visual cortical region of interest.

[0003] During data acquisition, images are presented to subjects in a random order using stimulus presentation software such as E-Prime, according to pre-set presentation times, frequencies, and rest intervals. A single image presentation and data acquisition is typically called a trial. For example, the Gallant team used a frequency of 200 milliseconds to alternate between an image and a gray background for 1 second, followed by a 3-second gray background as a rest interval, for a total of 4 seconds. The continuous presentation of multiple image stimuli usually constitutes a run. Multiple scans of a subject after entering the MRI machine constitute a session. Typically, the same image is presented to the subject multiple times to collect corresponding data, which is then preprocessed to improve the quality of the fMRI data. In Gallant's team's data collection, each participant had 5 sessions. Each session consisted of 5 runs for model training (each showcasing 70 different images, presented twice, for a total of 11 minutes) and 2 runs for model validation (each showcasing 12 different images, presented 13 times, for a total of 12 minutes). Without considering resting-state scanning and visual region localization time, a single session lasted over an hour. The limited time participants had on the MRI equipment made it difficult to complete large-scale fMRI data collection in a short time, requiring participants to conduct multiple experiments over several days. Furthermore, to ensure data stability, the same image was often presented to participants repeatedly. These realities limited the sample size of the fMRI dataset.

[0004] In 2008, Gallant's team released the fMRI dataset stimulated by grayscale natural images, collecting data from subjects' visual regions of interest (ROIs) such as V1, V2, V3, V4, and LOC, with a sample size of 1870. In 2017, Horikawa et al. released the fMRI dataset stimulated by natural images, collecting data from subjects' visual ROIs such as V1, V2, V3, V4, LOC, FFA, and PPA, including 200 classes of natural images, with a sample size of 1250. In 2019, Chang et al. released the BOLD-5000 fMRI dataset, including one primary visual region and four advanced visual regions (PPA, RSC, OPA, and LOC), with a sample size approaching 5000. Previously, these mainstream fMRI datasets had sample sizes in the thousands, significantly smaller than publicly available visual datasets in the field of artificial intelligence (such as ImageNet and Microsoft COCO), limiting the performance of deep networks in visual encoding.

[0005] In December 2021, Kay's research team at the University of Minnesota published the world's first large-scale 7T fMRI visual dataset in the top international journal *Nature Neuroscience*. The team praised it for bridging cognitive neuroscience and artificial intelligence. Regarding the image stimuli used in the experiment, nearly 73,000 natural scene images were selected from the well-annotated Microsoft COCO image dataset. Eight participants were selected, each viewing 9,000 different images. Additionally, different participants viewed 1,000 identical images, for a total of 10,000 images viewed per participant. The entire dataset has a sample size in the tens of thousands, an order of magnitude larger than previous fMRI datasets, and was created by the team over a year. Ultra-high field (7T) functional magnetic resonance imaging was used, and each image was presented to the participants three times, producing a relatively robust response signal. Regarding the experimental design for data collection, each participant underwent 40 sessions. Each session consisted of 12 runs, each run lasting 5 minutes and including over 60 4-second stimulus trials (trails). Each session contained a total of 750 trails. Since each image was repeated 3 times, each participant completed a total of 40 × 750 / 3 = 10,000 image presentations. To ensure data collection quality, participants were required to answer whether they had seen the image before during the viewing process, thus maintaining their attention.

[0006] However, the consistency of the 7T fMRI visual dataset after multiple batches of data acquisition and processing over several days is not high. Summary of the Invention

[0007] This invention addresses the problem of low consistency in fMRI data acquired and processed in multiple batches over multiple days (such as the 7T fMRI visual dataset). It proposes a method and device for fMRI data consistency correction based on deep domain adaptation. By treating data acquired multiple times over multiple days as data domains from different sources, the invention innovatively employs deep learning domain adaptation technology to improve the overall consistency of fMRI data acquired over multiple days.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] This invention proposes a depth-domain-adaptive-based fMRI data consistency correction method, comprising:

[0010] The brain region fMRI voxel response data collected multiple times over a day are input into an autoencoder network to extract intermediate feature information;

[0011] Based on the intermediate feature information extracted by the autoencoder network, the corrected fMRI voxel response data is reconstructed. During the data reconstruction process, a moment matching algorithm is introduced to use the brain region fMRI voxel response data collected multiple times over a day as data domains from different sources. This reduces the moment spacing between different data domains in the intermediate feature space extracted by the autoencoder network, achieving domain-independent representation of the fMRI data and completing the consistency correction of the fMRI data.

[0012] Furthermore, the distribution distance between different data domains is calculated as follows:

[0013]

[0014] Where D i and D j Representing different data fields, MD(D i D j () represents the distribution distance between different data domains, and E represents the encoder. and This represents the intermediate feature information extracted from different data domains by the autoencoder network. k=1 represents the first moment, and k=2 represents the second moment.

[0015] Furthermore, in the moment matching algorithm, the distribution distance within the N batches of acquired and processed fMRI data is comprehensively measured by weighted summation of the pairwise domain distances in multiple domains.

[0016] Furthermore, the moment matching algorithm employs the following optimization objective function:

[0017]

[0018] Where L i Let X represent the reconstruction error loss function.i This represents the preprocessed fMRI sample data, X. i 'represents X i The corresponding reconstructed data, where N represents the total number of data collection batches. In MD(D), D represents the decoder, and D represents the N batches of fMRI voxel response datasets acquired over multiple days. λ represents the weighting coefficient of the moment matching term.

[0019] Another aspect of the present invention proposes an fMRI data consistency correction device based on depth domain adaptation, comprising:

[0020] The feature extraction module is used to input brain region fMRI voxel response data collected multiple times over a day into the autoencoder network to extract intermediate feature information;

[0021] The domain-adaptive moment matching module is used to reconstruct corrected fMRI voxel response data based on intermediate feature information extracted by the autoencoder network. During the data reconstruction process, a moment matching algorithm is introduced to use brain region fMRI voxel response data collected multiple times across days as data domains from different sources. This reduces the moment spacing between different data domains in the intermediate feature space extracted by the autoencoder network, achieving domain-independent representation of fMRI data and completing consistency correction of fMRI data.

[0022] Furthermore, the distribution distance between different data domains is calculated as follows:

[0023]

[0024] Where D i and D j Representing different data fields, MD(D i D j () represents the distribution distance between different data domains, and E represents the encoder. and This represents the intermediate feature information extracted from different data domains by the autoencoder network. k=1 represents the first moment, and k=2 represents the second moment.

[0025] Furthermore, in the moment matching algorithm, the distribution distance within the N batches of acquired and processed fMRI data is comprehensively measured by weighted summation of the pairwise domain distances in multiple domains.

[0026] Furthermore, the moment matching algorithm employs the following optimization objective function:

[0027]

[0028] Where L i Let X represent the reconstruction error loss function. i This represents the preprocessed fMRI sample data, X. i'represents X i The corresponding reconstructed data, where N represents the total number of data collection batches. In MD(D), D represents the decoder, and D represents the N batches of fMRI voxel response datasets acquired over multiple days. λ represents the weighting coefficient of the moment matching term.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] This invention innovatively employs deep learning domain adaptation technology, which treats fMRI data acquired multiple times across days as data domains from different sources, thereby improving the overall consistency of fMRI data acquired across days and providing a high-quality data foundation for the subsequent construction of interpretable coding models. Attached Figure Description

[0031] Figure 1 This is a schematic diagram illustrating the fMRI dataset creation process and database construction in the background technology.

[0032] Figure 2 This is one of the flowcharts of an fMRI data consistency correction method based on depth domain adaptation according to an embodiment of the present invention;

[0033] Figure 3 This is a schematic diagram of the autoencoder network architecture according to an embodiment of the present invention;

[0034] Figure 4 This is a second flowchart of a domain-adaptive-based fMRI data consistency correction method according to an embodiment of the present invention;

[0035] Figure 5 This is a schematic diagram of the structure of an fMRI data consistency correction device based on depth domain adaptation according to an embodiment of the present invention. Detailed Implementation

[0036] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments:

[0037] like Figure 2 As shown, an fMRI data consistency correction method based on depth domain adaptation includes:

[0038] S101: Inputting brain region fMRI voxel response data collected multiple times over several days into an autoencoder network to extract key intermediate feature information:

[0039] Specifically, the input of this invention is (specific) brain region fMRI voxel response data X collected multiple times over several days (specifically, 7T fMRI preprocessed data provided by the Kay team can be used), and the output is reconstructed and corrected fMRI data X'. Since the dimensions are the same and the values ​​are similar, an autoencoder network is used to design a deep domain adaptation technique. For example... Figure 3As shown, the autoencoder network mainly consists of an encoder network and a decoder network. For N batches of fMRI voxel response datasets acquired over multiple days: D = {D1, D2, ..., D...} N}, define X1, X2, ..., X N For N batches of fMRI data samples acquired and preprocessed over multiple days, X-ray data of each sample is extracted using an encoder. i Key intermediate feature Z i '.

[0040] S102: Based on the intermediate feature information extracted by the autoencoder network, reconstruct the corrected fMRI voxel response data. During the data reconstruction process, a moment matching algorithm is introduced, using brain region fMRI voxel response data collected multiple times across different days as data domains from different sources. This reduces the moment spacing between different data domains in the intermediate feature space extracted by the autoencoder network, achieving domain-independent representation of the fMRI data and completing the consistency correction of the fMRI data. This includes:

[0041] fMRI data X was reconstructed using a decoder. i During the training of the autoencoder network, the reconstruction error is required to be as small as possible, that is, the voxel response value before and after reconstruction is not allowed to change too much. The loss function is shown in formula (1).

[0042]

[0043] Where L i Let E represent the reconstruction error loss function, E represent the encoder, and D represent the decoder.

[0044] Specifically, the preprocessed fMRI voxel response data acquired over multiple days consists of domain-dependent and domain-independent features. Domain-dependent features cause inconsistencies in the multi-domain data. During data reconstruction, to reduce the impact of low consistency caused by multiple acquisitions over multiple days (different data domains), the encoder network is required to extract domain-independent features related to the visual cortex activated by image stimuli as much as possible when extracting features from the input data, while suppressing the expression of domain-dependent features in the data. In other words, feature extraction is required to perform domain-independent representation.

[0045] like Figure 4 As shown, through optimization, the encoder will try to remove domain-related feature information from the data during encoding, thereby reducing the amount of domain-related feature information in the reconstructed fMRI data acquired multiple times over a day. This improves the consistency of fMRI data acquired multiple times over a day and lays a good data foundation for the subsequent construction and analysis of interpretable coding models.

[0046] To enable the encoder network to achieve domain-independent representation during the encoding process, the intermediate features of data from different domains must belong to the same domain. First, the domain distance between different features is characterized. Data moments are an effective way to measure the distribution distance between different data. As shown in Equation (2), the first and second moments of the feature distribution are used to measure the distribution distance MD between different domains.

[0047]

[0048] Where k = 1 represents the first moment and k = 2 represents the second moment.

[0049] Furthermore, as shown in Equation (3), the distribution distance within the N batches of acquired and processed fMRI data is comprehensively measured by weighted summation of the pairwise domain distances in multiple domains.

[0050]

[0051] Where C represents the number of combinations, and D represents the N batches of fMRI voxel response datasets collected over multiple days.

[0052] Therefore, in the optimization process of autoencoder network, a domain adaptation regularization term, namely a moment matching regularization term, is introduced to minimize the reconstruction error and the distribution distance between different data domains. Thus, the entire optimization objective function is shown in formula (4).

[0053]

[0054] Where λ represents the weighting coefficient of the moment matching term.

[0055] Based on the above embodiments, such as Figure 5 As shown, the present invention also proposes an fMRI data consistency correction device based on depth domain adaptation, comprising:

[0056] The feature extraction module is used to input brain region fMRI voxel response data collected multiple times over a day into the autoencoder network to extract intermediate feature information;

[0057] The domain-adaptive moment matching module is used to reconstruct corrected fMRI voxel response data based on intermediate feature information extracted by the autoencoder network. During the data reconstruction process, a moment matching algorithm is introduced to use brain region fMRI voxel response data collected multiple times across days as data domains from different sources. This reduces the moment spacing between different data domains in the intermediate feature space extracted by the autoencoder network, achieving domain-independent representation of fMRI data and completing consistency correction of fMRI data.

[0058] Furthermore, the distribution distance between different data domains is calculated as follows:

[0059]

[0060] Where D i and D j Representing different data fields, MD(D i D j () represents the distribution distance between different data domains, and E represents the encoder. and This represents the intermediate feature information extracted from different data domains by the autoencoder network. k=1 represents the first moment, and k=2 represents the second moment.

[0061] Furthermore, in the moment matching algorithm, the distribution distance within the N batches of acquired and processed fMRI data is comprehensively measured by weighted summation of the pairwise domain distances in multiple domains.

[0062] Furthermore, the moment matching algorithm employs the following optimization objective function:

[0063]

[0064] Where L i Let X represent the reconstruction error loss function. i This represents the preprocessed fMRI sample data, X. i 'represents X i The corresponding reconstructed data, where N represents the total number of data collection batches. In MD(D), D represents the decoder, and D represents the N batches of fMRI voxel response datasets acquired over multiple days. λ represents the weighting coefficient of the moment matching term.

[0065] In summary, this invention innovatively employs deep learning domain adaptation technology, treating fMRI data acquired multiple times across days as data domains from different sources. This improves the overall consistency of fMRI data acquired across days and provides a high-quality data foundation for the subsequent construction of interpretable coding models.

[0066] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for fMRI data consistency correction based on depth domain adaptation, characterized in that, include: The brain region fMRI voxel response data collected multiple times over a period of time are input into an autoencoder network to extract intermediate feature information; Based on the intermediate feature information extracted by the autoencoder network, the corrected fMRI voxel response data is reconstructed. During the data reconstruction process, a moment matching algorithm is introduced to use the brain region fMRI voxel response data collected multiple times over a day as data domains from different sources. This reduces the moment spacing between different data domains in the intermediate feature space extracted by the autoencoder network, achieving domain-independent representation of fMRI data and completing the consistency correction of fMRI data. Calculate the distribution distance between different data domains as follows: in and Representing different data fields, E represents the distribution distance between different data domains, and E represents the encoder. and This represents the intermediate feature information extracted from different data domains by the autoencoder network, where k=1 represents the first moment and k=2 represents the second moment. In the moment matching algorithm, the distribution distance within N batches of acquired and processed fMRI data is comprehensively measured by weighted summation of the pairwise domain distances in multiple domains. The moment matching algorithm employs the following optimization objective function: in Indicates reconstruction error. This represents the preprocessed fMRI sample data. express The corresponding reconstructed data, where N represents the total number of data collection batches. In the middle, D represents the decoder. D in Chinese means A batch of fMRI voxel response datasets collected over multiple days. This represents the weighting coefficient of the moment matching term.

2. A depth-domain-adaptive fMRI data consistency correction device, characterized in that, include: The feature extraction module is used to input the brain region fMRI voxel response data collected multiple times over a day into the autoencoder network to extract intermediate feature information; The domain-adaptive moment matching module is used to reconstruct the corrected fMRI voxel response data based on the intermediate feature information extracted by the autoencoder network. During the data reconstruction process, a moment matching algorithm is introduced to use the brain region fMRI voxel response data collected multiple times over a day as data domains from different sources. This reduces the moment spacing between different data domains in the intermediate feature space extracted by the autoencoder network, achieving domain-independent representation of fMRI data and completing the consistency correction of fMRI data. In the domain adaptive moment matching module, the distribution distance between different data domains is calculated in the following manner: in and Representing different data fields, E represents the distribution distance between different data domains, and E represents the encoder. and This represents the intermediate feature information extracted from different data domains by the autoencoder network, where k=1 represents the first moment and k=2 represents the second moment. In the moment matching algorithm, the distribution distance within N batches of acquired and processed fMRI data is comprehensively measured by weighted summation of the pairwise domain distances in multiple domains. The moment matching algorithm employs the following optimization objective function: in Indicates reconstruction error. This represents the preprocessed fMRI sample data. express The corresponding reconstructed data, where N represents the total number of data collection batches. In the middle, D represents the decoder. D in Chinese means A batch of fMRI voxel response datasets collected over multiple days. This represents the weighting coefficient of the moment matching term.

Citation Information

Patent Citations

  • An fMRI visual coding model construction method based on transfer learning

    CN109816630A

  • Controller of legged mobile robot

    US20060173578A1