A brain image analysis method, device and electronic equipment based on distance-aware rotation attention and multi-scale cross complement

Abstract

The application provides a brain image analysis method, device and electronic equipment based on distance perception rotation attention and multi-scale cross complement. The method comprises the following steps: obtaining three-dimensional brain image data of an object to be analyzed and performing standardization processing to obtain brain tissue volume data; performing multi-layer feature extraction on the brain tissue volume data to obtain enhanced fusion features; performing Patch embedding of different scales on the enhanced fusion features to form fine-grained Token sequences and coarse-grained Token sequences; performing rotation attention processing and multi-scale cross complementary processing on the coarse-grained Token sequences and the fine-grained Token sequences respectively; adding the two processing results of the fine-grained Token sequences element by element to obtain fine-grained fusion enhanced Token sequences, and adding the two processing results of the coarse-grained Token sequences element by element to obtain coarse-grained fusion enhanced Token sequences; and obtaining a brain image diagnosis result based on the fine-grained fusion enhanced Token sequences and the coarse-grained fusion enhanced Token sequences.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically to a brain image analysis method, device, and electronic device based on distance-aware rotational attention and multi-scale cross-complementarity. Background Technology

[0002] Alzheimer's disease is a neurodegenerative disease characterized by progressive decline in memory and cognitive function. Structural magnetic resonance imaging (SMRI), with its non-invasive, repeatable, and high spatial resolution features, has been widely used for analyzing brain structural atrophy, assessing the risk of mild cognitive impairment, and studying changes in related brain regions. Traditional methods based on voxel morphology analysis or template measurement rely on manual feature setting, are complex, and are susceptible to noise, individual differences, and registration errors.

[0003] In recent years, deep learning models such as convolutional neural networks (CNNs) and Transformers have been used for 3D brain image analysis. While convolutional models possess local texture extraction capabilities, their fixed local receptive fields limit their ability to model dependencies between distant brain regions. Transformers can model global relationships, but standard self-attention mechanisms typically lack explicit representations of 3D spatial distance, relative position, and the continuity of local brain structures, potentially leading to an imbalance in attention allocation between near and far tokens. Furthermore, brain images often have high voxel dimensions, and lesions or discriminative regions typically constitute a small proportion of the image, making the model susceptible to interference from redundant information in non-critical areas. Summary of the Invention

[0004] In view of this, this application proposes a brain imaging analysis method, device and electronic device based on distance-aware rotational attention and multi-scale cross-complementarity to address the shortcomings of existing brain imaging deep learning models in terms of local structural detail preservation, three-dimensional spatial relative position modeling and cross-scale context fusion.

[0005] Specifically, this application is implemented through the following technical solution:

[0006] According to a first aspect of the embodiments of this specification, a brain imaging analysis method based on distance-aware rotational attention and multi-scale cross-complementarity is provided, the method comprising the following steps:

[0007] Step S1: Obtain three-dimensional brain imaging data of the object to be analyzed, and perform standardization processing on the three-dimensional brain imaging data to obtain brain tissue data;

[0008] Step S2: Perform multi-layer feature extraction on the brain tissue data, and enhance the features extracted in each layer using the spatial morphological information of the brain tissue data to obtain the enhanced fusion feature after the deepest layer enhancement.

[0009] Step S3: Patch embedding is performed on the enhanced fusion features at a first scale and a second scale respectively to form a fine-grained token sequence and a coarse-grained token sequence; the first scale is smaller than the second scale;

[0010] Step S4 involves performing rotational attention processing on the fine-grained token sequence and the coarse-grained token sequence, respectively, which integrates relative position awareness and spatial distance constraints, to obtain a fine-grained spatially enhanced token sequence and a coarse-grained spatially enhanced token sequence. The relative position awareness refers to applying rotational encoding based on the token position index to the query vector and key vector in the attention calculation, enabling the query vector and key vector to carry information about the relative positional differences between different tokens. The spatial distance constraint refers to adjusting the attention score based on the position index distance between each token in the query vector and key vector, so that spatially adjacent tokens receive a higher attention response than distant tokens.

[0011] Step S5: Perform multi-scale cross-complement processing on the fine-grained token sequence and the coarse-grained token sequence respectively to obtain a fine-grained complementary token sequence and a coarse-grained complementary token sequence.

[0012] Step S6: Add the fine-grained complementary token sequence and the fine-grained spatial enhancement token sequence element by element to obtain a fine-grained fusion enhancement token sequence; and add the coarse-grained complementary token sequence and the coarse-grained spatial enhancement token sequence element by element to obtain a coarse-grained fusion enhancement token sequence.

[0013] Step S7: Perform adaptive token aggregation on the fine-grained fusion enhancement token sequence and the coarse-grained fusion enhancement token sequence respectively to obtain the fine-grained global representation corresponding to the fine-grained fusion enhancement token sequence and the coarse-grained global representation corresponding to the coarse-grained fusion enhancement token sequence, so as to obtain the brain imaging diagnosis result based on the fine-grained global representation and the coarse-grained global representation.

[0014] According to a second aspect of the embodiments of this specification, a brain imaging analysis device based on distance-aware rotational attention and multi-scale cross-complementarity is provided, the device comprising:

[0015] The image preprocessing unit is used to acquire three-dimensional brain image data of the object to be analyzed, and to perform standardization processing on the three-dimensional brain image data to obtain brain tissue data.

[0016] The feature extraction unit is used to perform multi-layer feature extraction on the brain tissue data, and to enhance the features extracted at each layer using the spatial morphological information of the brain tissue data, so as to obtain the enhanced fusion feature after the deepest layer enhancement.

[0017] The sequence embedding unit is used to perform patch embedding on the enhanced fusion features at a first scale and a second scale, respectively, to form a fine-grained token sequence and a coarse-grained token sequence; the first scale is smaller than the second scale.

[0018] A rotational attention unit is used to perform rotational attention processing on the fine-grained token sequence and the coarse-grained token sequence respectively, fusing relative position awareness and spatial distance constraints, to obtain a fine-grained spatially enhanced token sequence and a coarse-grained spatially enhanced token sequence. The relative position awareness refers to applying rotational encoding based on the token position index to the query vector and key vector in the attention calculation, so that the query vector and key vector carry information about the relative positional differences between different tokens. The spatial distance constraint refers to adjusting the attention score according to the position index distance between each token in the query vector and the key vector, so that spatially adjacent tokens receive a higher attention response than distant tokens.

[0019] An interactive complementary unit is used to perform multi-scale cross-complement processing on the fine-grained token sequence and the coarse-grained token sequence respectively to obtain a fine-grained complementary token sequence and a coarse-grained complementary token sequence.

[0020] The fusion enhancement unit is configured to add the fine-grained complementary token sequence to the fine-grained spatial enhancement token sequence element by element to obtain a fine-grained fusion enhancement token sequence; and to add the coarse-grained complementary token sequence to the coarse-grained spatial enhancement token sequence element by element to obtain a coarse-grained fusion enhancement token sequence.

[0021] The discriminative diagnostic unit is used to perform adaptive token aggregation on the fine-grained fusion-enhanced token sequence and the coarse-grained fusion-enhanced token sequence respectively to obtain the fine-grained global representation corresponding to the fine-grained fusion-enhanced token sequence and the coarse-grained global representation corresponding to the coarse-grained fusion-enhanced token sequence, so as to obtain the brain imaging diagnostic result based on the fine-grained global representation and the coarse-grained global representation.

[0022] According to a third aspect of the embodiments of this specification, an electronic device is provided, comprising: a processor; and a computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method as described in the first aspect.

[0023] The embodiments of this application have at least the following technical effects:

[0024] First, this application enhances the features extracted from each layer by utilizing the spatial morphological information of the brain tissue data in each layer of feature extraction, thereby preserving the global structural information and fine-grained anatomical information of the original input in the deep three-dimensional features and improving the fidelity of brain image features.

[0025] Second, this application enhances the model's ability to model brain inter-region relationships by enabling the model to explicitly perceive the relative position and local structural continuity in three-dimensional space through distance-aware rotational attention; and improves the ability to capture multi-scale pathological patterns by enabling bidirectional information interaction between coarse-grained global semantics and fine-grained local structures through multi-scale cross-complementarity.

[0026] Third, this application uses an adaptive token aggregation mechanism to highlight discriminative tokens that contribute significantly to the diagnostic results, thereby reducing the interference of redundant regions in high-dimensional brain images on model decision-making. Attached Figure Description

[0027] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Some specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings in an exemplary and non-limiting manner. The same reference numerals in the drawings indicate the same or similar parts or components. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0028] Figure 1 This is a schematic flowchart illustrating an exemplary embodiment of the present application of a brain imaging analysis method based on distance-aware rotational attention and multi-scale cross-complementarity;

[0029] Figure 2 This is a schematic diagram of the overall structure of a brain imaging analysis model shown in an exemplary embodiment of this application;

[0030] Figure 3 This is a schematic diagram of the structure of a gated input sensing residual module according to an exemplary embodiment of this application;

[0031] Figure 4 This is a schematic diagram of the structure of a distance-aware rotating attention module according to an exemplary embodiment of this application;

[0032] Figure 5 This is a schematic diagram of the structure of a multi-scale cross-complementary module shown in an exemplary embodiment of this application;

[0033] Figure 6 This is a schematic diagram illustrating an interpretable backtracking process, as shown in an exemplary embodiment of this application;

[0034] Figure 7 This is a block diagram illustrating an electronic device according to an exemplary embodiment of this application;

[0035] Figure 8 This is a block diagram of a brain imaging analysis device based on distance-aware rotational attention and multi-scale cross-complementarity, as illustrated in an exemplary embodiment of this application. Detailed Implementation

[0036] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0037] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0038] The embodiments described in this specification will now be described in detail.

[0039] This application provides a brain imaging analysis method based on distance-aware rotational attention and multi-scale cross-complementarity. This brain imaging analysis method can be executed by an image analysis system, which can be deployed on an electronic device or in the cloud.

[0040] Figure 1 This is a schematic flowchart illustrating an exemplary embodiment of the present application of a brain imaging analysis method based on distance-aware rotational attention and multi-scale cross-complementarity, as shown in the following example. Figure 1 As shown, the brain imaging analysis method includes the following steps:

[0041] Step S1: Obtain three-dimensional brain imaging data of the object to be analyzed, and perform standardization processing on the three-dimensional brain imaging data to obtain brain tissue data.

[0042] In this embodiment, the three-dimensional brain imaging data can be T1-weighted three-dimensional structural magnetic resonance imaging (MRI) brain images.

[0043] After obtaining the three-dimensional brain imaging data, the original images are first de-skeletalized to remove signals from the skull, fat, and other non-brain tissues. Next, brain tissue segmentation is performed to obtain gray matter, white matter, and cerebrospinal fluid, with gray matter volume data preferred as the input for subsequent models. Then, the gray matter volume data is registered to the MNI standard space to reduce individual differences in brain structure. Finally, invalid regions at the edges of the standard space are cropped or resampled to obtain three-dimensional volume data suitable for neural network processing.

[0044] In one example, the dimensions of the 3D volume data are 112×128×112. Further data augmentation strategies such as random flipping, affine transformation, scaling, rotation, and translation can be employed during the training phase.

[0045] Step S2: Perform multi-layer feature extraction on the brain tissue data, and enhance the features extracted in each layer using the spatial morphological information of the brain tissue data to obtain the enhanced fusion feature after the deepest layer enhancement.

[0046] In the process of multi-layer feature extraction, the original spatial morphological information of brain tissue data is used to dynamically enhance the features extracted from each layer. This allows deep features to obtain high-dimensional semantic abstraction capabilities while still retaining the original brain structural information that can be traced back, thereby alleviating the problem of fine-grained anatomical structure degradation caused by multiple downsampling and convolution operations in deep networks.

[0047] Step S3: Patch embedding is performed on the enhanced fusion features at a first scale and a second scale respectively to form a fine-grained token sequence and a coarse-grained token sequence; the first scale is smaller than the second scale.

[0048] Step S4: Perform rotational attention processing on the fine-grained token sequence and the coarse-grained token sequence respectively, which integrates relative position awareness and spatial distance constraints, to obtain the fine-grained spatially enhanced token sequence and the coarse-grained spatially enhanced token sequence.

[0049] The relative position awareness refers to applying a rotation encoding based on the token position index to the query vector and key vector in the attention calculation, so that the query vector and key vector carry information about the relative position differences between different tokens; the spatial distance constraint refers to adjusting the attention score according to the position index distance of each token in the query vector and key vector, so that spatially adjacent tokens receive a higher attention response than distant tokens.

[0050] This embodiment applies position-index-based rotation encoding to the query vector and key vector, enabling the inner product term of the attention score to carry information about the relative positional differences between different tokens. This overcomes the limitation of standard self-attention mechanisms, which rely solely on feature similarity and cannot explicitly model 3D spatial positions. Simultaneously, a distance decay constraint is applied to the attention score based on the positional index distance between the tokens in the query vector and key vector, ensuring that spatially adjacent tokens receive higher attention responses than distant tokens. The combination of these two approaches allows the system to more accurately focus on spatially adjacent and structurally related brain region features when processing 3D brain images, suppressing noise interference from distant, irrelevant regions and improving the system's modeling accuracy of the continuity and spatial dependencies of local brain structures.

[0051] Step S5: Perform multi-scale cross-complement processing on the fine-grained token sequence and the coarse-grained token sequence respectively to obtain the fine-grained complementary token sequence and the coarse-grained complementary token sequence.

[0052] The multi-scale cross-complementary processing includes extracting global context information from the coarse-grained token sequence to enhance the field of view of the fine-grained token sequence; and extracting local structural detail information from the fine-grained token sequence to enhance the local spatial resolution of the coarse-grained token sequence.

[0053] In this way, fine-grained features supplement the global context view from coarse-grained features, and coarse-grained features supplement the local spatial resolution from fine-grained features. This embodiment overcomes the shortcomings of single-scale feature expression through a bidirectional complementary mechanism, enabling the system to capture long-range semantic dependencies across brain regions and retain information on changes in small lesion areas and fine anatomical structures, thereby improving the system's comprehensive recognition ability of pathological patterns at different scales.

[0054] Step S6: Add the fine-grained complementary token sequence and the fine-grained spatial enhancement token sequence element by element to obtain a fine-grained fusion enhancement token sequence; and add the coarse-grained complementary token sequence and the coarse-grained spatial enhancement token sequence element by element to obtain a coarse-grained fusion enhancement token sequence.

[0055] Step S7: Perform adaptive token aggregation on the fine-grained fusion enhancement token sequence and the coarse-grained fusion enhancement token sequence respectively to obtain the fine-grained global representation corresponding to the fine-grained fusion enhancement token sequence and the coarse-grained global representation corresponding to the coarse-grained fusion enhancement token sequence, so as to obtain the brain imaging diagnosis result based on the fine-grained global representation and the coarse-grained global representation.

[0056] This embodiment obtains the importance weight of each token to the global decision through adaptive token aggregation, and obtains the global representation based on the weighted summation. This makes the discriminative tokens that contribute more to the diagnostic results receive higher responses in the classification decision, while redundant information in the background region or non-critical brain region is effectively suppressed. This can reduce the interference of invalid regions in high-dimensional brain imaging data on the model decision and improve the accuracy and robustness of classification.

[0057] The steps described above regarding the brain imaging analysis method in this embodiment can also be performed by... Figure 2 The brain imaging analysis model shown is executed. For example... Figure 2 As shown, for three-dimensional structural magnetic resonance brain images, standardized brain tissue volume data is obtained through steps such as craniotomy, tissue segmentation, gray matter extraction, standard template registration, and cropping. The brain tissue volume data is then input into the gated input-aware residual (GIR) module of the brain image analysis model to dynamically inject global structural information from the original input during multi-layer three-dimensional convolution and downsampling, thereby mitigating the degradation of fine-grained anatomical structures during deep feature extraction.

[0058] After obtaining the three-dimensional deep features, a dual-scale branch is constructed, in which both the coarse-grained branch and the fine-grained branch have a patch embedding module. The coarse-grained patch embedding module is used to obtain coarse-grained token sequences to express long-range semantic relationships across brain regions, and the fine-grained patch embedding module is used to obtain fine-grained token sequences to preserve local structural details.

[0059] For each branch, a distance-aware rotational attention (DRA) module is designed to introduce rotational position encoding into the query vector and the key vector, enabling attention computation to express relative position information. At the same time, a distance decay bias is constructed based on the three-dimensional spatial distance of the tokens and superimposed on the attention score to enhance the response between locally structure-related tokens and suppress irrelevant long-distance dependencies.

[0060] Furthermore, each branch is designed with a Multi-Scale Cross-Complementary (MCC) module, which uses fine-grained tokens as queries to extract coarse-grained global context, uses coarse-grained tokens as queries to extract fine-grained local structural information, and fuses bidirectional cross-attention outputs with spatial enhancement features to obtain an enhanced token sequence containing local details, global semantics, and spatial relationships.

[0061] Finally, an Adaptive Token Aggregation (ATA) module is designed. This module performs linear response calculation and Softmax normalization on the fused and enhanced token sequences at each granularity to obtain the importance weight of each token to the global decision; then, the token features are weighted and aggregated according to the weights to obtain the global representation at the corresponding granularity and output the diagnostic results.

[0062] In addition, the importance weights of tokens at various granularities can be rearranged into a three-dimensional weighted grid according to the Patch partitioning method, and upsampled to the original brain image space. Combined with brain tissue masks and anatomical brain region templates, voxel-level attention heatmaps, brain region contribution distribution, key region coordinates, and visualization results can be generated.

[0063] The following is a detailed explanation. Figure 2 The structure and function of each module.

[0064] Regarding the GIR module:

[0065] In some embodiments, Figure 1 Step S2 involves inputting the brain tissue data into a feature extraction network, which contains multiple stacked feature extraction layers. The output of the previous feature extraction layer serves as the input of the next feature extraction layer, and the deepest feature extraction layer outputs the final enhanced fusion feature.

[0066] Each feature extraction layer includes a gated feature retention block.

[0067] For each feature extraction layer, the brain tissue data and the original features output by the current feature extraction layer are input into the gated feature retention block. The gated enhanced features output by the gated feature retention block are added element by element to the original features output by the current feature extraction layer to obtain the enhanced fusion features finally output by the current feature extraction layer.

[0068] The gated feature retention block is configured to perform the following steps:

[0069] Spatial size alignment and channel dimension alignment are sequentially performed on the brain tissue data to obtain aligned features with the same spatial resolution and the same number of channels as the original features output by the current feature extraction layer;

[0070] Global average pooling is performed on the alignment features to obtain the average response intensity of each channel, and then gating weights are generated by a fully connected network and a sigmoid activation function.

[0071] The alignment features are weighted according to the channel dimension using the gating weights to obtain the gated enhanced features output by the gated feature retention block.

[0072] Specifically, such as Figure 3 As shown, the GIR module includes a feature extraction network, which is used to extract multi-level semantic features from the input brain tissue data. For each feature extraction layer of the feature extraction network, a large-kernel 3D convolution is first used to expand the receptive field, followed by batch normalization, activation functions, and multi-layer residual convolutions to obtain 3D features at different levels.

[0073] Gated Feature Preservation (GFP) is used to dynamically and adaptively inject the raw spatial morphological information of the initial input into feature maps at different levels. For any feature layer, GFP blocks use brain tissue data. and current layer features For input. First, for... Perform adaptive 3D average pooling (Adaptive_avg_pool3d) to make its spatial dimensions equal to... Consistency is achieved; then, channel alignment is realized through 1×1×1 convolution or linear projection to obtain alignment features. Next, two fully connected layers are used to generate gating weights. The calculation can be expressed as:

[0074] (1)

[0075] in, For the average response intensity, and All of these are trainable parameters. and These are the ReLU function and the Sigmoid activation function, respectively.

[0076] Then, Acting on After obtaining the gating enhancement features, then combine them with... Adding elements one by one yields enhanced fusion features.

[0077] Thus, by adding a gated feature retention block to each feature extraction layer, this embodiment enables the feature extraction network to dynamically retain the original brain structure information during downsampling and deep semantic extraction.

[0078] Regarding the two-scale branch:

[0079] like Figure 2 As shown, after the three-dimensional enhanced fusion features are extracted by the GIR module, the three-dimensional enhanced fusion features are input into two parallel coarse-grained branches and fine-grained branches to construct token sequences with different spatial granularities.

[0080] To capture brain region information at different scales, this embodiment sets two patch sizes: fine-grained and coarse-grained.

[0081] In the fine-grained branch, smaller 3D patches are used to divide the enhanced fusion features. For example, the patch size for the first scale (fine-grained) is set to... Voxels. By dividing the three-dimensional feature into several voxels of size... The small cubes are flattened into one-dimensional vectors and mapped to fixed-dimensional tokens through a linear projection layer, resulting in a fine-grained token sequence. Each token in the sequence corresponds to local structural details within a small neighborhood in the original brain image, such as subtle sulci and gyri changes in the gray cortex or small-scale morphological differences in brain regions.

[0082] In the coarse-grained branch, a larger 3D patch is used to divide the enhanced fusion features. For example, the patch size for the second scale (coarse-grained) is set to... Voxels. The three-dimensional feature is divided into several voxels of size. The large cube blocks are flattened and linearly projected onto a fixed-dimensional token, resulting in a coarse-grained token sequence. Each token in this sequence corresponds to a large range of structural information in the original brain image, capable of expressing long-range semantic relationships and global contextual information across brain regions.

[0083] Thus, in this embodiment, the fine-grained branch uses a smaller first scale for patch embedding, which can preserve more refined local spatial details; while the coarse-grained branch uses a larger second scale for patch embedding, which can effectively compress the sequence length, reduce computational complexity, and expand the receptive field of each token, enabling it to capture a wider range of brain region interaction information.

[0084] In some embodiments, the fine-grained branch and the coarse-grained branch are each provided with a DRA module, and the DRA module is configured to perform the following steps:

[0085] The input token sequence is linearly mapped to generate query vector, key vector, and value vector respectively.

[0086] Based on a preset attenuation base, angular frequency vectors are constructed in each channel dimension, and the angular frequency vectors are copied in the channel dimension to obtain rotation angle vectors;

[0087] For each token's position index, the position index is multiplied by the rotation angle vector to obtain the token's sine rotation position code and cosine rotation position code in each channel dimension;

[0088] For each two adjacent channels forming a two-dimensional vector in the query vector, perform a two-dimensional rotation operation to obtain the rotated query vector; and for each two adjacent channels forming a two-dimensional vector in the key vector, perform a two-dimensional rotation operation to obtain the rotated key vector.

[0089] Attention calculations are performed based on the rotated query vector, the rotated key vector, and the value vector, and a spatially enhanced token sequence of the corresponding granularity is output after feature projection.

[0090] When the query vector and key vector after rotation encoding are used to calculate the inner product, the intersection term in the inner product result contains the sine or cosine difference between the position indices of different tokens, so that the attention calculation explicitly includes the relative position information between tokens.

[0091] In this embodiment, for any branch of the token sequence, a query vector Q, a key vector K, and a value vector V are obtained through linear mapping. To enhance the ability to model relative positions, rotational position encoding is applied to Q and K, so that the position information of each token is embedded in the feature channel in the form of a rotational transformation. The rotational position encoding in this embodiment does not introduce additional trainable parameters and can reflect the relative positional differences in attention similarity calculation.

[0092] Traditional self-attention mechanisms lack sensitivity to the relative positions of tokens and easily overlook spatial structural relationships. To address this, this embodiment introduces Rotary Positional Embedding (RPE) into the representation of Q and K to integrate positional information in an embedded manner, thereby improving structural modeling capabilities.

[0093] Specifically, let the fine-grained token sequence or the coarse-grained token sequence be... Where B represents the batch size, N is the number of tokens, and C is the channel dimension. For example... Figure 4 As shown, Q, K, and V are generated through three sets of linear mappings, as follows:

[0094] (2)

[0095] in, Let h be the learnable linear transformation weight matrix, h be the number of attention heads, and d = C / h be the channel dimension of each attention head. The operation is used to reconstruct the linearly transformed features into a shape suitable for multi-head attention computation.

[0096] To encode relative position information without introducing additional parameters, we introduce a two-dimensional rotational position code for each position and embed it into the Q and K vectors.

[0097] First, construct the angular frequency vector. Take half-channel index Then we have:

[0098] (3)

[0099] Next, Copy once along the channel dimension to accommodate subsequent rotation operations, forming a rotation angular frequency vector of length d:

[0100] (4)

[0101] Let X contain N tokens, and the indices of the tokens be... , Let j be the value of the t-th token; for the channel dimension index j Define the sine and cosine phase codes for each channel dimension:

[0102] (5)

[0103] Then, for each pair of channel dimensions Perform a two-dimensional rotation operation to embed the position code into the feature representation:

[0104] (6)

[0105] Right now:

[0106] (7)

[0107] (8)

[0108] in, and The query vector or the key vector before rotation The first channel and the first Each channel value and These are the corresponding channel values ​​after rotation.

[0109] The above two-dimensional rotation process can be represented as a whole as follows:

[0110] (9)

[0111] in, This function performs a periodic rotation operation on a two-dimensional vector composed of every two elements in the channel dimension, thereby effectively encoding relative position information.

[0112] Finally, the rotation position code is embedded into Q and K through the following transformation:

[0113] (10)

[0114] in, The key vector after rotation. This is the query vector after rotation.

[0115] By using the above embedding method, the model effectively enhances its ability to model the relative positional relationships in the sequence while keeping the original attention mechanism structure and parameter scale unchanged, thereby improving its performance in image and sequence tasks.

[0116] In some embodiments, the DRA module is further configured to perform the following steps:

[0117] Determine the position index distance between each token in the query vector and each token in the key vector;

[0118] For each attention head, a distance decay bias is constructed based on the location index distance. The value of the distance decay bias decreases as the location index distance increases, and the decay rate is different for different attention heads.

[0119] The distance decay bias is superimposed on the inner product of the rotated query vector and the rotated key vector, and after attention calculation and feature projection, a spatially enhanced token sequence of the corresponding granularity is obtained.

[0120] To further enhance the attention mechanism's ability to model local structural information, this embodiment designs a Distance-Aware Decay Bias (DDB) in the DRA module to explicitly model the role of the relative distance between tokens in the attention weights. The DDB mechanism simulates the influence of information transmission gradually decreasing with increasing distance, thereby enhancing the model's focus on local regions, suppressing redundant long-distance dependencies, and improving the accuracy and efficiency of feature learning.

[0121] Specifically, such as Figure 4 As shown, let the index of any two tokens in Q and K in the h-th attention head be . and The position index distance between these two tokens in the sequence is This embodiment designs a decay bias term related to the location index distance for each token pair. The definition is as follows:

[0122] (11)

[0123] in, This represents the preset initial attenuation coefficient, used to control the attenuation intensity; Used to control the decay rate between different attention heads, where H represents the total number of attention heads. This represents the decay rate of each attention head.

[0124] DDB can effectively encourage the model to pay more attention to nearby tokens and suppress attention weights on distant tokens, thereby improving local modeling ability and convergence speed.

[0125] Ultimately, distance-aware rotational attention can be represented as:

[0126] (12)

[0127] In some embodiments, the fine-grained branch and the coarse-grained branch are each provided with an MCC module, and the MCC module is configured to perform the following steps:

[0128] Using the fine-grained token sequence as the query vector and the coarse-grained token sequence as the key and value vectors, cross-attention computation is performed to obtain a fine-grained complementary token sequence that integrates global context semantics.

[0129] Using the coarse-grained token sequence as the query vector and the fine-grained token sequence as the key and value vectors, cross-attention computation is performed to obtain a coarse-grained complementary token sequence that integrates local structural details.

[0130] like Figure 5 As shown, the MCC module is used to implement bidirectional information exchange between coarse-grained and fine-grained tokens. Let the fine-grained token sequence be... The coarse-grained token sequence is Fine-grained branching paths For Query, with For Key and Value, global semantic context is extracted from coarse-grained branch paths using scaled dot product attention to obtain enhanced fine-grained features; the coarse-grained branch paths are... For Query, with Using the key and value, local structural details are extracted from the fine-grained branch paths to obtain enhanced coarse-grained features.

[0131] Taking a fine-grained branch path as an example, this path uses a fine-grained token sequence as... For the query vector, from the coarse-grained token sequence Extract the global semantic context.

[0132] First, for fine-grained token sequences With coarse-grained token sequences Perform a linear mapping to generate a query vector. , key vector Sum value vector :

[0133] (13)

[0134] in, , , It is a learnable linear transformation weight matrix.

[0135] Then, to adapt to the multi-head attention mechanism, the above query vector is... , key vector Sum value vector Mapped to multi-head form:

[0136] (14)

[0137] Where h is the number of attention heads, and d = C / h is the channel dimension of each attention head.

[0138] Finally, cross-attention computation is performed to obtain a fine-grained complementary token sequence that incorporates global context semantics. :

[0139] (15)

[0140] Similarly, in coarse-grained branch paths, with coarse-grained token sequences For the query vector, from the fine-grained token sequence Extract detailed local structural information.

[0141] First, for fine-grained token sequences With coarse-grained token sequences Perform a linear mapping to generate a query vector. , key vector Sum value vector :

[0142] (16)

[0143] in, , , It is a learnable linear transformation weight matrix.

[0144] Then, to adapt to the multi-head attention mechanism, the above query vector is... , key vector Sum value vector Mapped to multi-head form:

[0145] (17)

[0146] Finally, cross-attention computation is performed to obtain a coarse-grained complementary token sequence that incorporates local structural details. :

[0147] (18)

[0148] in, , The pre-defined mapping weight matrix is ​​used to map the output feature dimension to the same dimension as the input feature dimension.

[0149] In obtaining and After that, By adding element-wise the fine-grained spatially enhanced token sequence, a fine-grained fused enhanced token sequence is obtained. ;Will By adding element-wise the coarse-grained spatially enhanced token sequence, we obtain the coarse-grained fused enhanced token sequence. .

[0150] About the ATA module:

[0151] like Figure 6 As shown, the ATA module in this embodiment can highlight the discriminative token before classification.

[0152] Suppose a fine-grained fusion enhancement token sequence output by dual-scale branches. Enhanced Token Sequence with Coarse-grained Fusion Abbreviated as First, it is obtained through linear mapping. response value :

[0153] (19)

[0154] in, This is a linear mapping operator.

[0155] Then the response value Softmax normalization is performed to obtain the token importance weights. :

[0156] (20)

[0157] Subsequently, based on the importance weight of the tokens... right We perform a weighted summation to obtain the global representation. :

[0158] (twenty one)

[0159] Fine-grained global representation of the fine-grained branch output The input fully connected layer yields fine-grained diagnostic results, while the coarse-grained branch outputs a coarse-grained global representation. The input fully connected layer yields a coarse-grained diagnostic result. The coarse-grained diagnostic result is then added to the fine-grained diagnostic result to obtain the final brain imaging diagnostic result. The brain imaging diagnostic result may include, for example, category probability, risk score, or auxiliary discrimination label.

[0160] To further illustrate how to generate coarse-grained and fine-grained brain region contribution distributions based on the token importance weights of the fine-grained and coarse-grained branches, respectively, and achieve cross-scale interpretability output of model decisions.

[0161] Specifically, Figure 1 The method further includes step S8, which calculates the brain region contribution distribution for the fine-grained token importance weights and the coarse-grained token importance weights, respectively.

[0162] The calculation process for the brain region contribution distribution is as follows:

[0163] The importance weights of each granularity token are rearranged according to the Patch partitioning method to obtain a three-dimensional weight grid;

[0164] Upsampling of the three-dimensional weighted grid yields a voxel-level attention heatmap with the same spatial dimensions as the brain tissue data.

[0165] By combining a predefined brain tissue mask to remove non-brain tissue regions in the voxel-level attention heatmap, and by thresholding the remaining brain tissue regions, a set of high-response voxels with attention intensity higher than a preset threshold is extracted.

[0166] The set of high-response voxels is matched voxel by voxel with a predefined anatomical brain region template. The number of high-response voxels and the cumulative attention intensity in each brain region are counted to obtain the brain region contribution distribution.

[0167] Thus, based on the adaptive token aggregation results, this paper further explains how to generate voxel-level attention heatmaps and brain region contribution distributions based on the token importance weights of fine-grained and coarse-grained branches, respectively, to achieve cross-scale interpretable output of model decisions.

[0168] This embodiment first obtains fine-grained token importance weights through an adaptive token aggregation mechanism. Coarse-grained token importance weights Subsequently, for each branch, the brain region contribution distribution calculation described in step S8 is performed independently. Taking the fine-grained branch as an example, the specific sub-steps include:

[0169] Step S8.1, rearrange the three-dimensional weighted grid.

[0170] Fine-grained token importance weights Following the partitioning rules used during Patch embedding in step S3, the grid is rearranged in reverse to form a three-dimensional weighted grid consistent with the Patch grid spatial layout. After this operation, the importance weight of each Token is mapped back to its corresponding three-dimensional spatial position, forming a sparse weight distribution map.

[0171] Step S8.2, voxel-level attention heatmap generation.

[0172] Trilinear interpolation upsampling is performed on the three-dimensional weighted grid to make its spatial dimensions consistent with the brain tissue volume data after standardization in step S1, thereby obtaining a voxel-level attention heatmap.

[0173] The value of each voxel in the voxel-level attention heatmap represents the strength of that location's contribution to the model's decision, realizing the attention distribution mapping from the discrete token space to the continuous voxel space.

[0174] The coarse-grained branch employs the same operation to obtain a voxel-level attention heatmap at a coarse-grained scale. Due to the larger patch size of the coarse-grained branch, its upsampled heatmap exhibits a more macroscopic brain region response pattern, complementing the fine-grained heatmap.

[0175] Step S8.3, brain tissue mask constraint and high response region extraction.

[0176] To eliminate the interference of non-brain tissue regions such as the skull and air on the interpretation results, a brain tissue mask template corresponding to the standard brain space (e.g., MNI 152 space) is used. Voxel-by-voxel constraints are applied to both the voxel-level attention heatmap at the fine-grained scale and the voxel-level attention heatmap at the coarse-grained scale. Only the attention response values ​​of the brain tissue regions within the mask are retained, while the response values ​​of the regions outside the mask are set to zero.

[0177] Subsequently, the attention heatmap after masking is threshold-screened, and voxels with attention intensity higher than a preset threshold (e.g., 0.8) are identified as high-response voxels. High-response voxel sets are extracted at both the fine-grained and coarse-grained scales. Simultaneously, the three-dimensional spatial coordinates of all voxels within each set are calculated, and the smallest hexahedral bounding box that can enclose the set is determined to locate the spatial region of focus for the model.

[0178] Optionally, the centroid coordinates of the high-response voxels can also be calculated as the spatial location output of the key points.

[0179] Step S8.4, Statistical analysis of the contribution of anatomical brain regions.

[0180] The sets of high-response voxels at the fine-grained scale and the sets at the coarse-grained scale are matched one-to-one with predefined anatomical brain region templates, such as brain region templates (Automated Anatomical Labeling, AAL), to determine the brain region to which each high-response voxel belongs.

[0181] Based on this, the number of voxels belonging to the high-response voxel set and the cumulative attention intensity in each brain region were counted respectively, and the brain regions were sorted from high to low according to the cumulative attention intensity to obtain the brain region contribution distribution at the fine-grained scale and the brain region contribution distribution at the coarse-grained scale.

[0182] The brain region contribution distribution can be output as the name of each brain region, standard anatomical coordinates, attention intensity value, and sorting list.

[0183] This embodiment, through the above-described interpretable backtracking process, can map the model's decision-making basis from the feature layer to the spatial layer, and then to the brain region layer, outputting axial, coronal, and sagittal heatmaps as well as key brain regions.

[0184] During the training phase, preprocessed brain image samples and their category labels can be input. Figure 2 The brain imaging analysis model shown is trained under supervised conditions using cross-entropy loss, weighted cross-entropy loss, focus loss, or a combination thereof, and its parameters are updated using the AdamW optimizer. To improve model stability, five-fold cross-validation, data augmentation, weight decay, and learning rate scheduling are employed.

[0185] During the inference phase, after inputting the brain image to be analyzed, the brain image analysis model sequentially completes GIR feature extraction, dual-scale patch embedding, DRA spatial enhancement, MCC cross-scale complementation, ATA aggregation, and brain region regression, and finally outputs diagnostic probability, attention heatmap, key brain region names, key brain region contribution ranking, and spatial coordinates of high-response regions.

[0186] In summary, the brain imaging analysis scheme provided by the above embodiments of this application has at least the following advantages:

[0187] First, the GIR module preserves the global structural information and fine-grained anatomical information of the original input in deep three-dimensional features, thereby improving the fidelity of brain imaging features.

[0188] Second, by combining rotational position encoding and distance decay bias in the DRA module, the model can explicitly perceive the relative position in three-dimensional space and the continuity of local structures, thereby enhancing the ability to model the relationship between brain regions.

[0189] Third, the MCC module enables bidirectional information exchange between coarse-grained global semantics and fine-grained local structures, thereby improving the ability to capture multi-scale pathological patterns.

[0190] Fourth, the ATA module highlights discriminative tokens that contribute significantly to diagnostic results, reducing the interference of redundant regions in high-dimensional brain images on model decision-making. Furthermore, through token weight backtracking, voxel-level heatmap generation, and brain region template matching such as AAL, an interpretation chain is formed from the feature layer to the spatial layer to the brain region layer, improving the transparency of model decision-making and clinical readability.

[0191] Figure 7 This is a schematic diagram of an electronic device illustrated in this specification according to an exemplary embodiment. Please refer to... Figure 7 At the hardware level, the device includes a processor 710, an internal bus 720, a network interface 730, memory 740, a hardware acceleration device 750, and non-volatile memory 760, and may also include other hardware required for its functions. One or more embodiments of this application can be implemented in software, for example, the processor 710 reads the corresponding computer program from the non-volatile memory 760 into the memory 740 and then runs it. Of course, in addition to software implementation, one or more embodiments of this application do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the above processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0192] Figure 8 This is a structural block diagram of a brain imaging analysis device based on distance-aware rotational attention and multi-scale cross-complementarity, as illustrated in an exemplary embodiment of this application. The brain imaging analysis device can be applied to, for example... Figure 7 The electronic device shown implements the technical solution of this application. The brain imaging analysis device includes: an image preprocessing unit 810, a feature extraction unit 820, a sequence embedding unit 830, a rotational attention unit 840, an interaction complementarity unit 850, a fusion enhancement unit 860, and a discriminant diagnosis unit 870, wherein:

[0193] The image preprocessing unit 810 is used to acquire three-dimensional brain image data of the object to be analyzed, and to perform standardization processing on the three-dimensional brain image data to obtain brain tissue data.

[0194] The feature extraction unit 820 is used to perform multi-layer feature extraction on the brain tissue data, and to enhance the features extracted at each layer using the spatial morphological information of the brain tissue data, so as to obtain the enhanced fusion feature after the deepest layer enhancement.

[0195] The sequence embedding unit 830 is used to perform patch embedding on the enhanced fusion features at a first scale and a second scale, respectively, to form a fine-grained token sequence and a coarse-grained token sequence; the first scale is smaller than the second scale.

[0196] The rotation attention unit 840 is used to perform rotation attention processing on the fine-grained token sequence and the coarse-grained token sequence respectively, which integrates relative position awareness and spatial distance constraints, to obtain a fine-grained spatially enhanced token sequence and a coarse-grained spatially enhanced token sequence. The relative position awareness refers to applying rotation encoding based on the token position index to the query vector and key vector in the attention calculation, so that the query vector and key vector carry information about the relative position differences between different tokens. The spatial distance constraint refers to adjusting the attention score according to the position index distance between each token in the query vector and key vector, so that spatially adjacent tokens receive a higher attention response than distant tokens.

[0197] The interactive complementarity unit 850 is used to perform multi-scale cross-complementary processing on the fine-grained token sequence and the coarse-grained token sequence respectively to obtain the fine-grained complementary token sequence and the coarse-grained complementary token sequence.

[0198] The fusion enhancement unit 860 is used to add the fine-grained complementary token sequence to the fine-grained spatial enhancement token sequence element by element to obtain a fine-grained fusion enhancement token sequence; and to add the coarse-grained complementary token sequence to the coarse-grained spatial enhancement token sequence element by element to obtain a coarse-grained fusion enhancement token sequence.

[0199] The discriminant diagnostic unit 870 is used to perform adaptive token aggregation on the fine-grained fusion enhancement token sequence and the coarse-grained fusion enhancement token sequence respectively to obtain the fine-grained global representation corresponding to the fine-grained fusion enhancement token sequence and the coarse-grained global representation corresponding to the coarse-grained fusion enhancement token sequence, so as to obtain the brain imaging diagnostic result based on the fine-grained global representation and the coarse-grained global representation.

[0200] In some embodiments, the feature extraction unit 820 is used to input the brain tissue data into a feature extraction network, which includes multiple stacked feature extraction layers, with the output of the previous feature extraction layer serving as the input of the next feature extraction layer, and the deepest feature extraction layer outputting the final enhanced fusion feature; wherein, each feature extraction layer is further provided with a gated feature retention block;

[0201] For each feature extraction layer, the brain tissue data and the original features output by the current feature extraction layer are input into the gated feature retention block. The gated enhanced features output by the gated feature retention block are added element by element to the original features output by the current feature extraction layer to obtain the enhanced fusion features finally output by the current feature extraction layer.

[0202] The gated feature retention block is configured to perform the following steps:

[0203] Spatial size alignment and channel dimension alignment are sequentially performed on the brain tissue data to obtain aligned features with the same spatial resolution and the same number of channels as the original features output by the current feature extraction layer;

[0204] Global average pooling is performed on the alignment features to obtain the average response intensity of each channel, and then gating weights are generated by a fully connected network and a sigmoid activation function.

[0205] The alignment features are weighted according to the channel dimension using the gating weights to obtain the gated enhanced features output by the gated feature retention block.

[0206] In some embodiments, the fusion enhancement unit 860 is used to obtain fine-grained token importance weights and coarse-grained token importance weights through the adaptive aggregation; the brain imaging analysis device further includes:

[0207] A cross-scale interpretation unit is used to calculate the brain region contribution distribution for both the fine-grained token importance weights and the coarse-grained token importance weights; wherein the brain region contribution distribution calculation process is as follows:

[0208] The importance weights of each granularity token are rearranged according to the Patch partitioning method to obtain a three-dimensional weight grid;

[0209] Upsampling of the three-dimensional weighted grid yields a voxel-level attention heatmap with the same spatial dimensions as the brain tissue data.

[0210] The non-brain tissue regions in the voxel-level attention heatmap are removed by combining a predefined brain tissue mask, and the retained brain tissue regions are threshold-screened to extract a set of high-response voxels with attention intensity higher than a preset threshold. The three-dimensional coordinates of each voxel in the set of high-response voxels and the minimum bounding box covering the set of high-response voxels are calculated.

[0211] The set of high-response voxels is matched voxel by voxel with a predefined anatomical brain region template. The number of high-response voxels and the cumulative attention intensity in each brain region are counted to obtain the brain region contribution distribution.

[0212] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0213] Accordingly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in any of the above embodiments.

[0214] Accordingly, embodiments of this application also provide a computer program product configured to perform the methods described in any of the above embodiments.

[0215] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0216] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0217] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0218] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0219] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0220] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0221] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0222] It should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0223] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A brain imaging analysis method based on distance-aware rotational attention and multi-scale cross-complementarity, characterized in that, The method includes the following steps: Step S1: Obtain three-dimensional brain imaging data of the object to be analyzed, and perform standardization processing on the three-dimensional brain imaging data to obtain brain tissue data; Step S2: Perform multi-layer feature extraction on the brain tissue data, and enhance the features extracted in each layer using the spatial morphological information of the brain tissue data to obtain the enhanced fusion feature after the deepest layer enhancement. Step S3: Perform patch embedding at the first scale and the second scale on the enhanced fusion features respectively to form a fine-grained token sequence and a coarse-grained token sequence. The first scale is smaller than the second scale; Step S4 involves performing rotational attention processing on the fine-grained token sequence and the coarse-grained token sequence, respectively, which integrates relative position awareness and spatial distance constraints, to obtain a fine-grained spatially enhanced token sequence and a coarse-grained spatially enhanced token sequence. The relative position awareness refers to applying rotational encoding based on the token position index to the query vector and key vector in the attention calculation, enabling the query vector and key vector to carry information about the relative positional differences between different tokens. The spatial distance constraint refers to adjusting the attention score based on the position index distance between each token in the query vector and key vector, so that spatially adjacent tokens receive a higher attention response than distant tokens. Step S5: Perform multi-scale cross-complement processing on the fine-grained token sequence and the coarse-grained token sequence respectively to obtain a fine-grained complementary token sequence and a coarse-grained complementary token sequence. Step S6: Add the fine-grained complementary token sequence and the fine-grained spatial enhancement token sequence element by element to obtain a fine-grained fusion enhancement token sequence; and add the coarse-grained complementary token sequence and the coarse-grained spatial enhancement token sequence element by element to obtain a coarse-grained fusion enhancement token sequence. Step S7: Perform adaptive token aggregation on the fine-grained fusion enhancement token sequence and the coarse-grained fusion enhancement token sequence respectively to obtain the fine-grained global representation corresponding to the fine-grained fusion enhancement token sequence and the coarse-grained global representation corresponding to the coarse-grained fusion enhancement token sequence, so as to obtain the brain imaging diagnosis result based on the fine-grained global representation and the coarse-grained global representation.

2. The method according to claim 1, characterized in that, Step S2 includes: The brain tissue data is input into a feature extraction network, which contains multiple stacked feature extraction layers. The output of the previous feature extraction layer is used as the input of the next feature extraction layer, and the deepest feature extraction layer outputs the final enhanced fusion feature. Each feature extraction layer includes a gated feature retention block. For each feature extraction layer, the brain tissue data and the original features output by the current feature extraction layer are input into the gated feature retention block. The gated enhanced features output by the gated feature retention block are added element by element to the original features output by the current feature extraction layer to obtain the enhanced fusion features finally output by the current feature extraction layer. The gated feature retention block is configured to perform the following steps: Spatial size alignment and channel dimension alignment are sequentially performed on the brain tissue data to obtain aligned features with the same spatial resolution and the same number of channels as the original features output by the current feature extraction layer; Global average pooling is performed on the alignment features to obtain the average response intensity of each channel, and then gating weights are generated by a fully connected network and a sigmoid activation function. The alignment features are weighted according to the channel dimension using the gating weights to obtain the gated enhanced features output by the gated feature retention block.

3. The method according to claim 1, characterized in that, The rotational attention processing in step S4 includes: The input token sequence is linearly mapped to generate query vector, key vector, and value vector respectively. Based on a preset attenuation base, angular frequency vectors are constructed in each channel dimension, and the angular frequency vectors are copied in the channel dimension to obtain a rotated angular frequency vector. For each token's position index, the token value at the position index is multiplied by the rotation angular frequency vector to obtain the sine rotation position code and cosine rotation position code of the token in each channel dimension; For each two adjacent channels forming a two-dimensional vector in the query vector, perform a two-dimensional rotation operation to obtain the rotated query vector; and for each two adjacent channels forming a two-dimensional vector in the key vector, perform a two-dimensional rotation operation to obtain the rotated key vector. Attention calculations are performed based on the rotated query vector, the rotated key vector, and the value vector, and a spatially enhanced token sequence of the corresponding granularity is output after feature projection.

4. The method according to claim 3, characterized in that, The two-dimensional rotation operation is achieved through the following matrix operations: ； in, For the input token sequence, the first... The location index of each token. For channel indexing, For the first The token in the 1st Rotational angular frequency vectors on each channel and The query vector or the key vector before rotation The first channel and the first Each channel value and These are the corresponding channel values ​​after rotation.

5. The method according to claim 3, characterized in that, The rotational attention processing in step S4 also includes: Determine the position index distance between each token in the query vector and each token in the key vector; For each attention head, a distance decay bias is constructed based on the location index distance. The value of the distance decay bias decreases as the location index distance increases, and the decay rate is different for different attention heads. The distance decay bias is superimposed on the inner product of the rotated query vector and the rotated key vector, and after attention calculation and feature projection, a spatially enhanced token sequence of the corresponding granularity is obtained.

6. The method according to claim 1, characterized in that, The multi-scale cross-complementary processing in step S5 includes: Using the fine-grained token sequence as the query vector and the coarse-grained token sequence as the key and value vectors, cross-attention computation is performed to obtain a fine-grained complementary token sequence that integrates global context semantics. Using the coarse-grained token sequence as the query vector and the fine-grained token sequence as the key and value vectors, cross-attention computation is performed to obtain a coarse-grained complementary token sequence that integrates local structural details.

7. The method according to any one of claims 1 to 6, characterized in that, Step S7 includes obtaining fine-grained token importance weights and coarse-grained token importance weights through the adaptive token aggregation. The method further includes step S8, which involves calculating the brain region contribution distribution for both the fine-grained token importance weights and the coarse-grained token importance weights; wherein the brain region contribution distribution calculation process is as follows: The importance weights of each granularity token are rearranged according to the Patch partitioning method to obtain a three-dimensional weight grid; Upsampling of the three-dimensional weighted grid yields a voxel-level attention heatmap with the same spatial dimensions as the brain tissue data. The non-brain tissue regions in the voxel-level attention heatmap are removed by combining a predefined brain tissue mask, and the retained brain tissue regions are threshold-screened to extract a set of high-response voxels with attention intensity higher than a preset threshold. The three-dimensional coordinates of each voxel in the set of high-response voxels and the minimum bounding box covering the set of high-response voxels are calculated. The set of high-response voxels is matched voxel by voxel with a predefined anatomical brain region template. The number of high-response voxels and the cumulative attention intensity in each brain region are counted to obtain the brain region contribution distribution.

8. A brain imaging analysis device based on distance-aware rotational attention and multi-scale cross-complementarity, characterized in that, The device includes: The image preprocessing unit is used to acquire three-dimensional brain image data of the object to be analyzed, and to perform standardization processing on the three-dimensional brain image data to obtain brain tissue data. The feature extraction unit is used to perform multi-layer feature extraction on the brain tissue data, and to enhance the features extracted at each layer using the spatial morphological information of the brain tissue data, so as to obtain the enhanced fusion feature after the deepest layer enhancement. The sequence embedding unit is used to perform patch embedding on the enhanced fusion features at a first scale and a second scale, respectively, to form a fine-grained token sequence and a coarse-grained token sequence; the first scale is smaller than the second scale. A rotational attention unit is used to perform rotational attention processing on the fine-grained token sequence and the coarse-grained token sequence respectively, fusing relative position awareness and spatial distance constraints, to obtain a fine-grained spatially enhanced token sequence and a coarse-grained spatially enhanced token sequence. The relative position awareness refers to applying rotational encoding based on the token position index to the query vector and key vector in the attention calculation, so that the query vector and key vector carry information about the relative positional differences between different tokens. The spatial distance constraint refers to adjusting the attention score according to the position index distance between each token in the query vector and key vector, so that spatially adjacent tokens receive a higher attention response than distant tokens. An interactive complementary unit is used to perform multi-scale cross-complement processing on the fine-grained token sequence and the coarse-grained token sequence respectively to obtain a fine-grained complementary token sequence and a coarse-grained complementary token sequence. The fusion enhancement unit is configured to add the fine-grained complementary token sequence to the fine-grained spatial enhancement token sequence element by element to obtain a fine-grained fusion enhancement token sequence; and to add the coarse-grained complementary token sequence to the coarse-grained spatial enhancement token sequence element by element to obtain a coarse-grained fusion enhancement token sequence. The discriminative diagnostic unit is used to perform adaptive token aggregation on the fine-grained fusion-enhanced token sequence and the coarse-grained fusion-enhanced token sequence respectively to obtain the fine-grained global representation corresponding to the fine-grained fusion-enhanced token sequence and the coarse-grained global representation corresponding to the coarse-grained fusion-enhanced token sequence, so as to obtain the brain imaging diagnostic result based on the fine-grained global representation and the coarse-grained global representation.

9. An electronic device, characterized in that, include: processor; as well as A computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method as described in any one of claims 1 to 7.

Images