A fine-grained brain age interpretation method and interpretation device

Abstract

The application discloses a kind of based on fine-grained brain age interpretation method and interpretation device, belong to neural image analysis field, method includes the pre-segmentation of neural image and the calculation of fine-grained brain age depth model, wherein the calculation method of fine-grained brain age depth model includes local area feature extraction step, global feature extraction step and information fusion step;By neural image pre-segmentation can output the regional segmentation result sequence of neural image, then input each fine-grained local feature sequence of fine-grained depth brain age model in local area feature extraction, then carry out global feature extraction to obtain the global feature sequence that the context information between features is fused, finally local and global feature are fused, obtain the fine-grained brain age sequence corresponding to each region of brain;Due to the sequence generation of multi-scale information fusion and the brain age of all regions once output, this scheme has the advantages of high interpretability and light calculation, can meet the fine-grained interpretation needs of clinical and medical research.

Description

Technical Field

[0001] This invention belongs to the field of neuroimaging analysis technology, and particularly relates to a fine-grained brain age interpretation method and device. Background Technology

[0002] Brain age (BA), as a novel data-driven biomarker, can predict outputs using machine learning models trained on neuroimaging data. Its interpretation shows promise as a marker for measuring brain developmental trajectories in clinical or medical research applications.

[0003] Most existing brain age prediction models are based on deep learning, and corresponding interpretation schemes therefore focus on attribution analysis methods for deep models. Specifically, they utilize decoupling analysis methods of deep models to perform attribution analysis on individual brain ages in whole-brain images, thereby obtaining the brain development status of fine-grained regions. However, existing brain age interpretation schemes have certain limitations. First, contribution-based calculation schemes can only measure the contribution of fine-grained regions to the calculation of whole-brain age, but this contribution does not directly reflect the developmental status of the corresponding regions, which is often a more common measurement requirement in clinical or medical research. Second, brain age interpretation schemes based on separate modeling of each segmented fine-grained brain region, while able to obtain the brain age of each region and thus directly measure its developmental status, require separate regional brain age model construction for each region. This results in high costs for model training and computation, and there is no global information interaction between brain age models of different regions. This leads to difficulty in controlling costs and a lack of interpretability in the obtained fine-grained brain ages due to the absence of global information considering the influence of other regions.

[0004] Therefore, designing a brain age interpretation scheme that can produce fine-grained brain age that directly explains the developmental status of brain regions, while simultaneously extracting local and global information from neuroimaging and having relatively low training and computational overhead is one of the more important technical issues in this field. Summary of the Invention

[0005] The purpose of this invention is to provide a fine-grained brain age interpretation method and device to improve the interpretability of brain age while reducing computational burden.

[0006] To achieve the above objectives, the technical solution provided by the present invention is as follows:

[0007] First aspect

[0008] This invention provides a method for interpreting brain age based on fine-grained methods, comprising the following steps:

[0009] Pre-segmentation steps: Based on the granularity requirements of the applied brain age, select or construct a partition template, and perform region masking segmentation on the human brain neural images to be studied. That is, manually annotate the voxels of each partition to construct a partition template or select an existing template. Then, extract voxels from the neural images to be studied according to the index values ​​of the annotations of each partition in the template. Extract all the annotated voxels of the corresponding partition index, and use the extracted voxels as the segmentation results of the corresponding regions.

[0010] Local region information extraction steps: Input the pre-segmented region segmentation result sequence into the local information extraction module of the fine-grained brain age depth model, extract local information within each region's voxels, thereby producing local information codes for each region, forming a local information coding sequence;

[0011] Global information extraction steps: Input the local information encoding into the global information extraction module to extract global information between local features of the region, thereby producing a global information encoding sequence;

[0012] Information aggregation steps: Local and global information encodings are used as inputs, local-global multi-scale information is fused, and then the fused encoding sequence is mapped to finally produce the brain age of each fine-grained region, forming a regional brain age sequence.

[0013] In the pre-segmentation step, whole-brain images can be segmented into fine-grained regions for clinical or research needs.

[0014] In the local region information extraction step, local information within each region can be extracted at once from the voxel values ​​of all segmented regions.

[0015] In the local region information extraction step, for each voxel of the fine-grained brain age depth model, the local information extraction module has a corresponding multilayer perceptron network to perform nonlinear feature extraction operations. By performing the above operations on each region, the local information encoding sequence can be obtained.

[0016] Among them, the nonlinear feature extraction operation using a multilayer perceptron network can take into account the contribution of all voxels in the region. The operation rule is to multiply the voxel vector of the corresponding region by the weight vector learned by the perceptron, add the bias term and the regularization term, and then perform nonlinear activation through the activation function to finally calculate the local information encoding of the corresponding region.

[0017] In the global information extraction step, the local information encoding sequence can be further processed to extract global information between features, thereby obtaining a global encoding sequence that takes into account the influence of contextual information between regions. The specific calculation method is as follows: the local information encoding sequence is input into the position encoding module of the transformer to calculate the position encoding of the local information of each region. Then, the local information encoding sequence and the position encoding sequence are vector-added and input into the transformer encoder for global self-attention calculation, thereby obtaining an encoding sequence that contains global information between features.

[0018] The position encoding module adds position information to the input local information encoding sequence, ensuring that it remains sensitive to the order of sequence elements during subsequent self-attention calculations. This guarantees that each element of the final global information encoding sequence corresponds one-to-one with the elements of the input sequence. The specific calculation method is as follows: For the element at position t in the local information encoding sequence, its value is divided by the dimension d of that element, and then the result is a random large number raised to the power of d. The position t is then divided by the power of d to obtain the position encoding of the corresponding element. Repeating this step yields the position encoding of each element in the local information encoding sequence.

[0019] The process involves using a transformer encoder to map a local information encoding sequence embedded with positional encoding into a new encoding sequence that extracts global information about the interactions between local features. The specific calculation method is as follows: The local information encoding sequence embedded with positional encoding is input into a linear mapping layer for linear mapping, and then the result is input into a multi-head self-attention layer. For a single element in the sequence, its value is multiplied by the corresponding learnable k, q, and v weight tensors. Then, the q of the target single element is multiplied by the k and v of each element to obtain the attention value between the target element and each element. All attention values ​​are then summed to obtain the self-attention encoding element that extracts global relevance. Repeating this step yields the global information encoding element corresponding to each local information encoding element, forming a self-attention encoding sequence. Finally, numerical normalization and nonlinear mapping are performed through a normalization layer and a multilayer perceptron layer to obtain the final global information encoding sequence.

[0020] In the information aggregation step, the local information encoding sequence and the global information encoding sequence are fused to obtain the fine-grained brain age of each region corresponding to the initial input, which incorporates multi-scale information. This brain age can be directly used to explain the brain development state of the corresponding local region and provide explanatory insights for the application target. The specific calculation method is as follows: the local information encoding sequence and the global information encoding sequence are added together, and then the result is input into a multilayer perceptron for further nonlinear mapping, finally producing the fine-grained brain age corresponding to each brain region.

[0021] Second aspect

[0022] Corresponding to the above method, the present invention also provides a fine-grained brain age interpretation device, which can automatically segment and extract and fuse local-global information from input whole-brain images, and finally output fine-grained brain age sequences for each region that can be directly used to interpret the fine-grained developmental status of the brain in one go; it includes the following circuits: region segmentation circuit, local information extraction circuit, global information extraction circuit, information fusion and fine-grained brain age generation circuit;

[0023] The region segmentation circuit is used to segment the whole brain image into a sequence of local regions according to granularity requirements.

[0024] The local information extraction circuit is used to extract the local information encoding sequence from the segmentation result sequence;

[0025] The global information extraction circuit is used to generate a corresponding global information encoding sequence based on the local information encoding sequence;

[0026] The information fusion circuit and the fine-grained brain age sequence generation circuit are used to aggregate information according to the local information encoding sequence and the global information encoding sequence respectively, and finally generate a fine-grained brain age sequence through the activation of the fine-grained brain age generation circuit.

[0027] Compared with the prior art, the beneficial effects of the present invention are:

[0028] The technical solution provided in this application, by directly generating regional brain ages, can be directly used to measure the brain development status of fine-grained regions, possessing direct fine-grained interpretability. Since a single computation can directly generate a sequence containing the brain ages of all fine-grained regions, and the computation process encompasses the extraction of local and global information and the fusion of two types of multi-scale information, it can include effective multi-scale features compared to calculating the brain ages of individual local regions, thus obtaining results with higher interpretability and accuracy. Furthermore, the computation process includes weight sharing, therefore, compared to the computational load and training efficiency of existing interpretation schemes that model each fine-grained region separately, its cost is significantly reduced, achieving cost savings. Attached Figure Description

[0029] Figure 1 This is a schematic flowchart of a fine-grained brain age interpretation method provided in an embodiment of this application;

[0030] Figure 2 This is a diagram of the fine-grained brain age depth model architecture provided in the embodiments of this application;

[0031] Figure 3This is a structural diagram of a fine-grained brain age interpretation device provided in an embodiment of this application.

[0032] In the diagram: N (N1, N2): number of stacked structures; l: sequence length; m: feature dimension of the encoded sequence; S100: pre-segmentation step; S110: local region information extraction step; S200: local information extraction module; S120: global information extraction step; S210: global information extraction module; S130: information aggregation step; S220: information fusion module; 300: region segmentation circuit; 310: local information extraction circuit; 320: global information extraction circuit; 330: information fusion circuit; 340: fine-grained brain age generation circuit. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] This application provides a fine-grained brain age interpretation method and device to improve the interpretability of brain age while reducing computational burden.

[0035] The fine-grained brain age interpretation method includes the following steps: pre-segmentation, local information extraction, global information extraction, and information fusion. The pre-segmentation step divides the input whole-brain image data into a sequence of fine-grained sub-region voxel sets according to a given region segmentation template. The local information extraction step performs dimensionality reduction on the fine-grained region voxel sets using a multilayer perceptron nonlinear mapping, which includes linear product mapping, nonlinear activation calculation, and forgetting layer calculation, to output the dimensionality-reduced local information encoding sequence for each region. The global information extraction step takes the local information encoding sequence as input and uses positional encoding calculation and the multi-head self-attention mechanism in the transformer encoder to map the local information encoding sequence into a global information encoding sequence containing global information between regions. The information fusion step performs tensor addition fusion on the local and global information sequences, then performs dimensionality reduction mapping using a multilayer perceptron to obtain a feature sequence that integrates multi-scale information. Finally, activation mapping calculation is performed to obtain the final fine-grained brain age sequence for each region.

[0036] The fine-grained brain age interpretation device includes a region segmentation circuit, a local information extraction circuit, a global information extraction circuit, an information fusion circuit, and a fine-grained brain age generation circuit. The system comprises several components: a region segmentation circuit, utilizing brain partition templates constructed or selected according to application granularity requirements, segments the input whole-brain neural image into voxel sets corresponding to each region; a local information extraction circuit, coupled to the output of the region segmentation circuit, receives the segmented region sequences and performs multilayer perceptron nonlinear mapping calculations to generate dimensionality-reduced local information encoding sequences for each region; a global information extraction circuit, cascaded to the output of the local information extraction circuit, utilizes its internal position encoder and a transformer encoder with a self-attention mechanism to perform self-attention calculations on the local information encoding sequences, thereby obtaining encoding sequences containing global information between regions; the outputs of the local and global information encoding circuits are then connected in parallel before the information fusion circuit, enabling the information fusion circuit to linearly add and fuse the output sequences of both circuits, followed by further multilayer perceptron dimensionality reduction mapping; finally, a fine-grained brain age sequence generation circuit is coupled to the information fusion circuit, performing activation function calculations on the dimensionality-reduced, multi-scale encoded information sequences to output fine-grained brain age sequences corresponding to each brain region as the final output of the device.

[0037] It should be noted that the term "coupled (or connected)" as used in the application documents can refer to any direct or indirect connection means. For example, if the document describes a first device coupled (or connected) to a second device, it should be interpreted as the first device being directly connected to the second device, or the first device being indirectly connected to the second device through other devices or some connection means. Components / members / steps using the same reference numerals in the drawings and embodiments throughout this specification (including the claims) represent the same or similar parts. Components / members / steps using the same reference numerals or the same terms in different embodiments can be referred to mutually. The internal implementation details of the "multilayer perceptron layer" mentioned in this application are assumed to be a cascaded structure of linear layers, activation layers, and forgetting layers. Regarding the training steps of the deep model omitted in this specification, no restrictions are placed on the details of the training and optimization objectives, such as the selection of the learning rate and loss function, which can be flexibly changed according to actual accuracy and training requirements.

[0038] like Figures 1-3 The image shown is an embodiment of the present invention.

[0039] As shown in the pre-segmentation step S100 and circuit 300, there is no restriction on the specific modality of the neuroimaging; any brain imaging modality capable of granular segmentation can be used, such as functional magnetic resonance imaging (fMRI) or structural magnetic resonance imaging (SMRI). In the region segmentation circuit, a corresponding mask tensor T2 is generated for each region label using a preset partition template. Then, the whole-brain neuroimaging to be investigated is input into the segmenter, and multiple mask tensor multiplication calculations are performed between it and the input data tensor T1. The specific number of masking operations depends on the granularity of the brain region segmentation template constructed or selected according to application requirements. This embodiment does not limit the implementation details of the segmentation template. For example, an existing template such as AAL (Anatomical Automatic Labeling, AAL) can be used, which divides the brain into 116 regions. In this case, 116 masking calculations are required for T to obtain a set of voxels for 116 regions, which serves as the fine-grained region voxel set sequence Seq1.

[0040] As shown in the local region information extraction step S110 and the local information extraction module S200, the local information extraction circuit (circuit 310) is coupled to the output of the region segmentation circuit (circuit 300). In the local information circuit, Seq1 is input to the local information extraction module of the fine-grained depth model, that is, input to the local information extraction circuit. This step internally contains a number of feature extractors with the same length l as Seq1. The implementation details of the specific extractors are not limited here. Figure 2 Steps and Figure 3 The embodiment shown in the local information extraction circuit uses a multilayer perceptron architecture as the extractor. That is, each extractor performs nonlinear dimensionality reduction mapping on each region voxel set in Seq1 separately. The extraction results of each extractor are then concatenated into a sequence, which is the local information encoding sequence Seq2-1 corresponding to each region.

[0041] As shown in global information extraction step S120, global information extraction module S210, or circuit 320, in the global information extraction step, it needs to be coupled to the output of local information extraction module (module S200) or circuit 310, input Seq2-1, and then a weight-sharing global feature extractor is used to extract global information. The implementation details of the global feature extractor are not limited here. Figure 2 and 3In the illustrated embodiment, a structure (module S210) and circuit (circuit 320) based on positional encoding and a deep learning transformer encoder are used. This structure mainly includes a positional encoding layer, a normalization layer with residual structures, a multi-head self-attention layer, and a multilayer perceptron layer. The specific algorithms for the four types of mapping hidden layers are not limited here; for example, the positional encoding layer algorithm can use either relative or absolute positional encoding. This step ensures that the information influence between encoded regions is not completely lost, resulting in good robustness and high interpretability of the final result. Furthermore, since this structure can extract global information from the local information encoding of all regions at once, it has weight-sharing characteristics, thus ensuring the lightweight nature of the model and reducing the computational cost of the entire scheme. The results of each global information extraction are concatenated to form the corresponding global information encoding sequence Seq2-2 for each region. The number of transformer encoder stacks N1 is not limited here; a trade-off between accuracy and lightweight requirements can be made based on the application scenario. For example, in application scenarios with high accuracy requirements but less stringent lightweight requirements, N1 should be increased.

[0042] As described in information aggregation steps S130, module S220, and circuit 330, the information aggregation module or circuit needs to be simultaneously coupled to the outputs of both the local (module S200 or circuit 310) and global (module S210 or circuit 320) modules. The information extraction step inputs the local (Seq2-1) and global information encoding sequences (Seq2-2) calculated by the local and global information extraction steps. Then, a certain fusion method is used to perform information fusion calculation on the two multi-scale sequences to obtain a fine-grained brain age sequence with multi-scale information. The specific fusion algorithm is not limited here. For example, in the embodiment module S220 and circuit 330, a simple tensor addition algorithm is used for information fusion. The fused region information sequence also undergoes additional multilayer perceptron sequence dimensionality reduction mapping (the length of the multilayer perceptron sequence is consistent with the number of regions l, while the stacking number N2 is not limited and can still be set according to the requirements of interpretation accuracy and lightweightness in the application scenario). In the embodiment of the circuit device (circuit 330), the information fusion step and the final brain age generation step are designed separately. Therefore, the multi-scale information encoding sequence Seq3 output by the information fusion step needs to undergo further activation circuit calculations to obtain the final fine-grained regional brain age sequence Seq4, which aggregates multi-scale information (the design of the activation circuit is not limited here; for example, softmax can be used for classification activation). This sequence has direct fine-grained interpretability because each element corresponds to the brain age of a preset fine-grained brain region, and its value can be directly used to measure the development status of the corresponding local brain region under application requirements. Furthermore, since each element of the sequence contains local and global multi-scale information, it has excellent interpretability and robustness, and is relatively less affected by local noise. In addition, since a single parallel operation can produce fine-grained brain ages for all regions, it also has lightweight characteristics.

[0043] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for interpreting brain age based on fine-grained granularity, characterized in that, Includes the following steps: Pre-segmentation steps: Based on the granularity requirements of the applied brain age, select or construct a partition template, and perform region masking segmentation on the human brain neural images to be studied. That is, manually annotate the voxels of each partition to construct a partition template or select an existing template. Then, extract voxels from the neural images to be studied according to the index values ​​of the annotations of each partition in the template. Extract all the annotated voxels of the corresponding partition index, and use the extracted voxels as the segmentation results of the corresponding regions. Local region information extraction steps: Input the pre-segmented region segmentation result sequence into the local information extraction module of the fine-grained brain age depth model, extract local information within each region's voxels, thereby producing local information codes for each region, forming a local information coding sequence; Global information extraction steps: Input the local information encoding into the global information extraction module to extract global information between local features of the region, thereby producing a global information encoding sequence; Information aggregation steps: Local and global information encodings are used as inputs, local-global multi-scale information is fused, and then the fused encoding sequence is mapped to finally produce the brain age of each fine-grained region, forming a regional brain age sequence.

2. The method for interpreting brain age based on fine-grained detail according to claim 1, characterized in that, In the pre-segmentation step, whole-brain images can be segmented into fine-grained regions for clinical or research needs.

3. The method for interpreting brain age based on fine-grained detail according to claim 1, characterized in that, In the local region information extraction step, local information within each region can be extracted at once from the voxel values ​​of all segmented regions.

4. The method for interpreting brain age based on fine-grained detail according to claim 3, characterized in that, In the local region information extraction step, for each voxel of the fine-grained brain age depth model, the local information extraction module has a corresponding multilayer perceptron network to perform nonlinear feature extraction operations. By performing the above operations on each region, the local information encoding sequence can be obtained.

5. The method for interpreting brain age based on fine-grained detail according to claim 4, characterized in that, Using a multilayer perceptron network for nonlinear feature extraction can take into account the contribution of all voxels in the region. The algorithm is to multiply the voxel vector of the corresponding region by the weight vector learned by the perceptron, add a bias term and a regularization term, and then perform nonlinear activation through an activation function to finally calculate the local information encoding of the corresponding region.

6. The method for interpreting brain age based on fine-grained detail according to claim 1, characterized in that, In the global information extraction step, the local information encoding sequence can be further processed to extract global information between features, thereby obtaining a global encoding sequence that takes into account the influence of contextual information between regions. The specific calculation method is as follows: the local information encoding sequence is input into the position encoding module of the transformer to calculate the position encoding of the local information of each region. Then, the local information encoding sequence and the position encoding sequence are vector-added and input into the transformer encoder for global self-attention calculation, thereby obtaining an encoding sequence that contains global information between features.

7. The method for interpreting brain age based on fine-grained detail according to claim 6, characterized in that, The positional encoding module adds positional information to the input local information encoding sequence, ensuring that it remains sensitive to the order of sequence elements during subsequent self-attention calculations. This guarantees that each element of the final global information encoding sequence corresponds one-to-one with the elements of the input sequence. The specific calculation method is as follows: For the element at position t in the local information encoding sequence, divide its value by the dimension d of that element, then use it as an exponent of a random large number. Then divide position t by the exponent of that large number to obtain the positional encoding of the corresponding element. Repeat this step to obtain the positional encoding of each element in the local information encoding sequence.

8. The method for interpreting brain age based on fine-grained detail according to claim 6, characterized in that, A transformer encoder is used to map a local information encoding sequence embedded with positional encoding into a new encoding sequence that extracts global information about the interactions between local features. The specific calculation method is as follows: The local information encoding sequence embedded with positional encoding is input into a linear mapping layer for linear mapping, and then the result is input into a multi-head self-attention layer. For a single element in the sequence, its value is multiplied by the corresponding learnable k, q, v weight tensor. Then, the q of the target single element is multiplied by the k and v of each element to obtain the attention value between the target element and each element. Then, all attention values ​​are summed to obtain the self-attention encoding element that has extracted global relevance. Repeating this step, the global information encoding element corresponding to each local information encoding element is obtained, forming a self-attention encoding sequence. Then, numerical normalization and nonlinear mapping are performed through a normalization layer and a multilayer perceptron layer to obtain the final global information encoding sequence.

9. The method for interpreting brain age based on fine-grained detail according to claim 1, characterized in that, In the information aggregation step, the local information encoding sequence and the global information encoding sequence are fused to obtain the fine-grained brain age of each region corresponding to the initial input, which incorporates multi-scale information. This brain age can be directly used to explain the brain development state of the corresponding local region and provide explanatory insights for the application target. The specific calculation method is as follows: the local information encoding sequence and the global information encoding sequence are added together, and then the result is input into a multilayer perceptron for further nonlinear mapping, finally producing the fine-grained brain age corresponding to each brain region.

10. A fine-grained brain age interpretation device, characterized in that, The interpretation device can automatically segment and extract and fuse local-global information from the input whole-brain image, and finally output a fine-grained brain age sequence for each region that can be directly used to interpret the fine-grained developmental status of the brain in one go; it includes the following circuits: region segmentation circuit, local information extraction circuit, global information extraction circuit, information fusion and fine-grained brain age generation circuit; The region segmentation circuit is used to segment the whole brain image into a sequence of local regions according to granularity requirements. The local information extraction circuit is used to extract the local information encoding sequence from the segmentation result sequence; The global information extraction circuit is used to generate a corresponding global information encoding sequence based on the local information encoding sequence; The information fusion circuit and the fine-grained brain age sequence generation circuit are used to aggregate information according to the local information encoding sequence and the global information encoding sequence respectively, and finally generate a fine-grained brain age sequence through the activation of the fine-grained brain age generation circuit.

Images