Personalized Brain Map Generation Method and Apparatus therefor

Abstract

A method and apparatus for generating a personalized brain map are disclosed. The method for generating a personalized brain map comprises: (a) acquiring 4D functional magnetic resonance imaging data of a subject's brain; (b) applying the 4D functional magnetic resonance imaging data to a deep learning model to extract a local feature map and a global feature map for a 3D brain volume, respectively; (c) combining the local feature map and the global feature map with a matching time axis to extract feature values ​​for brain function and then reducing the dimensionality; and (d) generating a brain map of the subject by performing brain segmentation based on the result of applying a k-means clustering algorithm to the dimensionality-reduced feature values ​​for brain function.

Description

Technology Field

[0001] The present invention relates to a method and apparatus for generating a personalized brain map based on functional magnetic resonance imaging. Background Technology

[0003] In the fields of medicine and neuroscience, analyses such as the understanding of brain function and disease diagnosis have been conducted based on brain maps classified by region, reflecting the structural or functional characteristics of the brain.

[0004] In the case of these standard brain maps, they were generated based on structural magnetic resonance imaging (sMRI, hereinafter referred to as sMRI) data or by utilizing functional magnetic resonance imaging (fMRI, hereinafter referred to as fMRI) data.

[0005] Recently, research on deep learning-based brain segmentation technology has been progressing in the direction of limiting to specific regions or utilizing existing known standard brain maps to update them. However, while sMRI data can accurately reflect the structural characteristics of the brain, it has limitations in that it does not sufficiently reflect the functional characteristics. On the other hand, while fMRI data reflects the functional characteristics of the brain, it has limitations in that it does not sufficiently reflect individual characteristics because it generates a single representative brain map.

[0006] delete Prior art literature

[65535] (01) Republic of Korea Public Notice No. 10-2021-0068189 A(2021.06.09) The problem to be solved

[0007] The present invention is intended to provide a method and apparatus for generating a personalized brain map based on functional magnetic resonance imaging.

[0008] Furthermore, the present invention aims to provide a method and apparatus for generating a personalized brain map based on functional magnetic resonance imaging, which can generate a brain map that sufficiently reflects individual brain functions for the entire region rather than a specific region without using other standard brain maps, by utilizing functional magnetic resonance imaging data. means of solving the problem

[0010] According to one aspect of the present invention, a method for generating a personalized brain map based on functional magnetic resonance imaging is provided.

[0011] According to one embodiment of the present invention, a method for generating a personalized brain map may be provided, comprising: (a) acquiring 4D functional magnetic resonance imaging data of a subject's brain; (b) applying the 4D functional magnetic resonance imaging data to a deep learning model to extract a local feature map and a global feature map for a 3D brain volume, respectively; (c) combining the local feature map and the global feature map with a matching time axis to extract feature values ​​for brain function and then reducing the dimensionality; and (d) performing brain segmentation based on the result of applying a k-means clustering algorithm to the dimensionality-reduced feature values ​​for brain function to generate a brain map of the subject.

[0012] The above 4D functional magnetic resonance imaging data is composed of a 3D brain volume and a time axis, and the above 3D brain volume is composed of a sagittal plane, a coronal plane, and an axial plane, and the above deep learning model includes an autoencoder and a transformer, wherein the autoencoder receives the above 4D functional magnetic resonance imaging data and extracts a local feature map for the entire brain for each 3D brain volume, and the transformer receives feature maps for each axis of the sagittal plane, the coronal plane, and the axial plane using the local feature map for the above 3D brain volume, and extracts a global feature map for each axis of the sagittal plane, the coronal plane, and the axial plane, respectively, and the local feature map for the above 3D brain volume can be input to the transformer after being extracted for each axis of the sagittal plane, the coronal plane, and the axial plane.

[0013] In step (c) above, the feature values ​​for the brain function can be dimensionally reduced through principal component analysis.

[0014] The feature maps for each axis of the sagittal plane, coronal plane, and horizontal plane input to the transformer may be composed of a plurality of slices having a series of sequences.

[0016] According to another aspect of the present invention, an apparatus for generating a personalized brain map based on functional magnetic resonance imaging is provided.

[0017] According to one embodiment of the present invention, an apparatus may be provided comprising: a memory for storing at least one instruction; and a processor for executing the instruction stored in the memory, wherein each instruction executed by the processor comprises: (a) a step of acquiring 4D functional magnetic resonance imaging data of a subject's brain; (b) a step of applying the 4D functional magnetic resonance imaging data to a deep learning model to extract a local feature map and a global feature map of a 3D brain volume, respectively; (c) a step of combining the local feature map and the global feature map with a matching time axis to extract feature values ​​for brain function and then reducing the dimensionality; and (d) a step of performing brain segmentation by applying a k-means clustering algorithm to the dimensionality-reduced feature values ​​for brain function to generate a brain map of the subject. Effects of the invention

[0019] By providing a method and apparatus for generating a personalized brain map based on functional magnetic resonance imaging according to one embodiment of the present invention, there is an advantage in that a brain map sufficiently reflecting individual brain functions can be generated for the entire region, rather than a specific region, without using other standard brain maps, by utilizing functional magnetic resonance imaging data. Brief explanation of the drawing

[0021] FIG. 1 is a flowchart illustrating a method for generating a personalized brain map according to an embodiment of the present invention. FIG. 2 is a diagram illustrating fMRI data according to an embodiment of the present invention. FIG. 3 is a diagram illustrating the results of brain map generation according to a conventional method and an embodiment of the present invention. FIG. 4 is a schematic diagram illustrating the internal configuration of a computing device according to one embodiment of the present invention. Specific details for implementing the invention

[0022] As used in this specification, singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "composed" or "comprising" should not be interpreted as necessarily including all of the various components or steps described in the specification, and should be interpreted as meaning that some of the components or steps may be excluded, or that additional components or steps may be included. Furthermore, terms such as "...part," "module," etc., as used in the specification refer to a unit that processes at least one function or operation, which may be implemented in hardware or software, or a combination of hardware and software.

[0023] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0025] FIG. 1 is a flowchart illustrating a method for generating a personalized brain map according to an embodiment of the present invention, FIG. 2 is a diagram illustrating fMRI data according to an embodiment of the present invention, and FIG. 3 is a diagram illustrating the results of generating a brain map according to a conventional method and an embodiment of the present invention.

[0026] In step 110, the computing device (100) receives fMRI data.

[0027] Figure 2 shows an example of fMRI data.

[0028] fMRI data has a 4-dimensional structure with 3D brain volumes and a time axis. As shown in Fig. 2, the 3D brain volumes consist of 3-dimensional volumes in the sagittal, coronal, and axial planes, and each volume is at a point in time along the time axis, and as these volumes are stacked along the time axis, they form a 4-dimensional structure.

[0029] In step 115, the computing device (100) preprocesses the 4-dimensional fMRI data.

[0030] For example, before using 4D fMRI data, the computing device (100) can adjust the difference in scan time of slices due to device characteristics during imaging, adjust the size of a voxel which is one of the basic units of data, or adjust noise caused by the head movement of the subject.

[0031] In step 120, the computing device (100) applies the preprocessed 4-dimensional fMRI data to a convolution-based autoencoder model to extract representative feature maps of the whole brain for each volume.

[0032] That is, the computing device (100) can receive 4-dimensional fMRI data and then extract local features of the entire brain to extract a representative feature map of the entire brain for one volume. Through this, the computing device (100) can extract local features of the brain by observing the relationships within the entire brain at each time.

[0033] For example, 4D fMRI data We will assume that. Here, represents the 3D brain volume at each time point, where H represents the number of horizontal (axial) axes, W represents the number of sagittal axes, and D represents the number of coronal axes.

[0034] A convolution-based autoencoder model includes an encoder that extracts a whole brain representative feature map through convolution operations on each brain volume and a decoder that reconstructs it. The convolution-based autoencoder model can be trained to minimize reconstruction loss using the mean absolute error (MAE) between the 3D brain volume and the 3D brain volume reconstructed by the decoder.

[0035] Mean Absolute Error (MAE) can be calculated as in Equation 1.

[0036]

[0037] Here, represents the 3D brain volume reconstructed by the decoder, and Is It represents, and N represents the number of sample sizes.

[0038] In step 125, the computing device (100) extracts a global feature map for each axis of the brain volume based on the whole brain representative feature map for each volume.

[0039] At this time, the computing device (100) can generate a global feature map for each axis of the brain using a local feature map of the brain extracted from a convolution-based autoencoder model.

[0040] For example, a computing device (100) can generate a global feature map for each axis by applying average pooling of local feature maps of the brain extracted from a convolution-based autoencoder model for each axis (e.g., sagittal, coronal, axial) and then applying them to a transformer encoder.

[0041] Transformers can also have an encoder-decoder structure. Such transformers are configured with a structure capable of identifying the relationships between a series of sequential input data.

[0042] The average pooled feature maps input to the Transformer for each axis are internally composed of multiple slices for each axis, and by considering each slice as having a sequential order and learning the relationships between the slices, globally meaningful feature maps can be extracted.

[0043] The loss function of the transformer can be calculated as shown in Equation 2.

[0044]

[0045] Here, represents the mean pooled feature map, and is reconstructed It represents, where m represents a random masked slice. Equation 2 represents the MSE for the horizontal plane (axial) axis, and the loss function can be calculated in the same way for the remaining axes.

[0046] In step 130, the computing device (100) combines a local feature map and a global feature map for a 3D brain volume with a matching time axis to extract feature values ​​for brain function and then performs dimensionality reduction.

[0047] For example, a computing device (100) can combine a local feature map and a global feature map for a 3D brain volume into feature dimension axes for the entire volume to extract feature values ​​for brain function. Then, the computing device (100) can reduce the dimensionality of the combined feature map through principal component analysis to extract meaningful feature values ​​for brain function that contain both global and local features for each axis of the entire volume.

[0048] In one embodiment of the present invention, it is assumed that dimensionality reduction is performed through principal component analysis, but it is not necessarily limited thereto, and it is obvious that feature dimensions can also be reduced through methods such as projection.

[0049] In step 135, the computing device (100) performs brain segmentation by applying a K-means clustering algorithm to feature values ​​of dimensionally reduced brain functions and then generates a brain map of the subject.

[0050] For example, a computing device (100) can perform feature-based personalized brain segmentation using a K-means clustering method that compares each time axis of a voxel (i.e., the entire volume) and clusters it into K components.

[0051] Figure 3 illustrates the results of brain map generation according to a conventional method and an embodiment of the present invention. As shown in Figure 3, it can be seen that a brain map generation method using K-means clustering with local and global feature maps for a 3D brain volume according to an embodiment of the present invention enables the generation of a more precise brain map.

[0053] FIG. 4 is a schematic diagram illustrating the internal configuration of a computing device according to one embodiment of the present invention.

[0054] Referring to FIG. 4, a computing device (100) according to one embodiment of the present invention comprises a preprocessing unit (410), a feature extraction unit (420), a brain map generation unit (430), a memory (440), and a processor (440).

[0055] The preprocessing unit (410) is a means for preprocessing 4D fMRI data of the subject's brain. The preprocessing unit (410) can perform preprocessing on the 4D fMRI data, such as adjusting the difference in scan time of slices due to device characteristics during imaging, adjusting the size of a voxel which is one of the basic units of data, or adjusting noise caused by the subject's head movement.

[0056] The feature extraction unit (420) is a means for applying preprocessed 4D fMRI data to a deep learning model to extract local feature maps and global feature maps for 3D brain volumes, respectively, and then combining them to extract feature values ​​for brain function.

[0057] That is, deep learning models can include convolution-based autoencoders and transformers.

[0058] Accordingly, preprocessed 4D fMRI data can be applied to an autoencoder to extract local features of the entire brain. That is, the autoencoder can extract feature maps for each 3D brain volume. Therefore, local features of the brain can be extracted by observing the relationships within the entire brain at each time step.

[0059] Furthermore, the Transformer can extract global feature maps based on the relationships between the average-pooled feature maps of each axis of the brain volume (i.e., sagittal, coronal, and axial) using local feature maps for each volume extracted by the autoencoder, and then extract global feature maps based on the relationships between the average-pooled feature maps of each axis that have a sequential order. In other words, the Transformer can extract global feature maps for each of the sagittal, coronal, and axial axes.

[0060] The feature extraction unit (420) can extract feature values ​​for the overall function of the brain by combining a local feature map and a global feature map for brain volume along a matching time axis.

[0061] In this way, the extracted feature values ​​for brain functions can be dimensionally reduced. Known methods such as principal component analysis and projection can be used for dimensionality reduction.

[0062] The brain map generation unit (430) is a means for generating an individual's brain map by performing brain segmentation by applying a K-means clustering algorithm to feature values ​​of brain function.

[0063] The memory (440) stores various instructions for performing a personalized brain map generation method according to one embodiment of the present invention.

[0064] The processor (440) is a means for controlling internal components (e.g., a preprocessing unit (410), a feature extraction unit (420), a brain map generation unit (430), a memory (440), etc.) of a computing device (100) according to one embodiment of the present invention.

[0065] Additionally, the processor (440) may execute instructions stored in memory (440). Instructions executed by the processor (440) may perform each of the steps described with reference to FIG. 1.

[0066] An apparatus and method according to an embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the computer-readable medium may be those specifically designed and configured for the present invention, or they may be those known and available to a person skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0067] The hardware device described above may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.

[0068] The present invention has been described above with reference to its embodiments. Those skilled in the art will understand that the present invention may be implemented in modified forms without departing from the essential characteristics of the invention. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the invention is defined by the claims, not by the foregoing description, and all variations within the scope of the claims should be interpreted as being included in the invention.

Claims

Claim 1 A method for generating a personalized brain map performed on a computing device, comprising: (a) acquiring 4D functional magnetic resonance imaging data of a subject's brain, wherein the 4D functional magnetic resonance imaging data includes a plurality of 3D brain volumes arranged along a time axis; (b) applying the 4D functional magnetic resonance imaging data to a deep learning model to extract a local feature map for a 3D brain volume corresponding to each time index of the time axis, and using the local feature map to extract a global feature map for each axis of the 3D brain volume; (c) generating a combined feature map by combining the local feature map corresponding to each time index of the time axis and the global feature map corresponding to the local feature map as feature dimension axes, and performing dimensionality reduction on the combined feature map to calculate a feature value for brain function; and (d) generating a brain map of the subject by performing brain segmentation based on the result of applying a k-means clustering algorithm to the feature value for brain function. Claim 2 A method for generating a personalized brain map according to claim 1, wherein each axis of the 3D brain volume is composed of a sagittal plane, a coronal plane, and an axial plane, and the deep learning model includes an autoencoder and a transformer, wherein the autoencoder receives the 4D functional magnetic resonance imaging data and extracts a local feature map for the entire brain for each 3D brain volume, and the transformer receives feature maps for each axis of the sagittal plane, the coronal plane, and the axial plane using the local feature map for the 3D brain volume, and extracts a global feature map for each axis of the sagittal plane, the coronal plane, and the axial plane, respectively, and wherein the local feature map for the 3D brain volume is extracted for each axis of the sagittal plane, the coronal plane, and the axial plane, then average-pooled and input to the transformer. Claim 3 A method for generating a personalized brain map according to claim 1, wherein in step (c), the feature values ​​for the brain function are dimensionally reduced through principal component analysis. Claim 4 A method for generating a personalized brain map according to claim 2, wherein the feature map for each axis of the sagittal plane, the coronal plane, and the horizontal plane input to the transformer is composed of a plurality of slices having a series of sequences. Claim 5 A memory storing at least one instruction; and a processor executing the instruction stored in the memory, wherein the instruction executed by the processor comprises: (a) a step of acquiring 4D functional magnetic resonance imaging data of a subject's brain—the 4D functional magnetic resonance imaging data includes a plurality of 3D brain volumes arranged along a time axis; (b) a step of applying the 4D functional magnetic resonance imaging data to a deep learning model to extract a local feature map for a 3D brain volume corresponding to each time index of the time axis, and using the local feature map to extract a global feature map for each axis of the 3D brain volume; (c) a step of generating a combined feature map by combining the local feature map corresponding to each time index of the time axis and the global feature map corresponding to the local feature map as feature dimension axes, and performing dimensionality reduction on the combined feature map to calculate a feature value for brain function; and (d) a device characterized by performing a step of generating a brain map of the subject by performing brain segmentation based on the result of applying a k-means clustering algorithm using feature values ​​for the dimensionality-reduced brain function.

Images