Attention-guided multi-branch fusion classification method for responsible vessel localization

Abstract

The application discloses a kind of attention guide multi-branch fusion classification methods for responsible blood vessel positioning, it is mainly related to the field of auxiliary judgment of blood vessel obstruction;Including steps: S1, constructing multi-branch fusion network;S2, sample label is obtained by sampler;S3, different sample label pair is sent into corresponding network branch, and after the improved main network of weight sharing, the feature vector of each branch is obtained;S4, classification probability output p c 、 p r Is obtained;S5, segmentation probability graph is obtained;S6, the classification probability p b Of balanced branch is obtained;S7, according to classification probability p c 、 p r 、 p b Classification cumulative loss is constructed The application provides new power for computer-aided physician to judge whether large blood vessel is blocked and specific responsible blocked blood vessel, can effectively solve the problem of class serious imbalance in data sample, and reach satisfactory classification precision.

Description

Technical Field

[0001] This invention relates to the field of auxiliary judgment of vascular occlusion, specifically an attention-guided multi-branch fusion classification method for locating the responsible vessel. Background Technology

[0002] Acute ischemic stroke (AIS) caused by large intracranial vessel occlusion (LVO) has a high rate of disability and mortality. Since the introduction of endovascular treatment (EVT) into clinical practice, the prognosis of AIS patients caused by anterior circulation LVO has improved significantly. However, because the effectiveness of EVT is highly time-dependent and is only available in specialized centers, often requiring inter-hospital transfers, it can delay surgical intervention and increase the likelihood of poor prognosis. Therefore, rapid identification of occluded vessels is crucial for diagnosis and treatment selection.

[0003] Computed tomographic angiography (CTA) is the primary method for detecting occlusion of proximal large intracranial vessels and a prerequisite imaging feature for anterior circulation thrombectomy. To visually represent low-vessel occlusion (LVO), CTA images processed using maximum intensity projection are shown below. Figure 1 As shown, a single head CTA scan may generate hundreds of high-resolution images for review, posing challenges to radiologists and the clinical imaging workflow and potentially delaying patient treatment. Therefore, it is imperative to assist physicians in quickly and accurately identifying the presence of LVO and the location of the responsible vessel on CTA, thus securing the most timely surgical treatment for patients.

[0004] Given the superior performance of deep learning in medical applications, the focus of research on rapidly identifying occluded blood vessels has shifted from pre-hospital LVO prediction scales to deep learning technology. In existing technologies, a study called DeepSymNet uses the left and right hemispheres of the brain as input, determining the presence of LVO by comparing the differences in semantic information after 3D convolution between the two hemispheres; Jia You et al. constructed a multi-level AIS machine learning evaluation model using structured demographic data, clinical data, and NCCT imaging features obtained from a deep learning model, effectively applying it to the LVO binary classification task. However, these studies have not considered the inherent class imbalance problem in medical data and are limited to detecting the presence of LVO. Summary of the Invention

[0005] The purpose of this invention is to solve the problems existing in the prior art and provide an attention-guided multi-branch fusion classification method for locating responsible blood vessels. This method provides new assistance to computer-assisted physicians in judging whether large blood vessels are blocked and the specific responsible blocked blood vessel. It can effectively solve the problem of severe class imbalance in data samples and achieve satisfactory classification accuracy.

[0006] To achieve the above objectives, the present invention employs the following technical solution:

[0007] An attention-guided multi-branch fusion classification method for locating responsible vessels includes the following steps:

[0008] S1. Construct a multi-branch fusion network, which includes a regular learning branch, a reverse learning branch, and a balancing branch. The regular learning branch is used for feature learning of normal individuals, the reverse learning branch is used for feature learning of patients, and the balancing branch is used to dynamically fuse the feature information of the regular learning branch and the reverse learning branch to complete the identification of the blocked responsible blood vessel.

[0009] S2. Obtain sample labels using the sampler. The sample labels for the regular learning branch are: (x c y c The sample labels for the reverse learning branch are: (x r y r The sample labels for the balanced branch are: (x b y b ), where x is the training sample, y is the corresponding label, y∈{1,2,…,L}, L is the number of final classification categories, c represents the regular learning branch sample, r represents the reverse learning branch sample, and b represents the balanced branch sample;

[0010] S3. Different sample label pairs are fed into the corresponding network branches, and after passing through the improved main network with weight sharing, the feature vectors F of each branch are obtained. c F r F b , of which F c For the feature vectors of the regular learning branch, F r Feature vectors of the reverse learning branch, F b Eigenvectors of balanced branches;

[0011] S4. Transfer the feature vector F c After weight-distributed convolutional layers Conv c The obtained feature vector The input is fed into the classification module to obtain the classification probability output p. c , the feature vector F r After weight-distributed convolutional layers Conv r The obtained feature vector The input is fed into the classification module to obtain the classification probability output p. r ;

[0012] S5. The feature vector F obtained from the conventional learning branch... c The data is fed into the attention guidance module, which guides the model's focus, to obtain the segmentation probability map. The feature vector F obtained from the back learning branch r The data is fed into the attention guidance module, which guides the model's focus, to obtain the segmentation probability map.

[0013] S6, Feature vector F b Simultaneously passing through the convolutional layer Conv c Conv r After passing through the convolutional layer Conv c Obtain the feature vector After convolutional layer Conv r Obtain the feature vector Then The output of the adaptive fusion module after balancing branches The classification probability p of the balanced branch is obtained by feeding it into the classifier. b ;

[0014] S7. Based on the classification probability p c p r p b To construct the cumulative loss of the classification

[0015]

[0016]

[0017] Among them, y c y represents the true classification label corresponding to the regular learning branch. r For the true classification label corresponding to the reverse learning branch, y b To balance the true classification labels corresponding to the branches, α is used to shift the focus of model learning, and N represents the current epoch. max This represents the total number of training epochs.

[0018] According to the segmentation probability map Constructing the auxiliary guidance loss function Used to guide the model's focus, the final loss function of the entire network can be described as:

[0019]

[0020] Where a and b are the loss weights.

[0021] Preferably, in step S2: the regular learning branch uses a uniform sampler that preserves the original data distribution, the reverse learning branch uses a reverse sampler, and the balancing branch uses a class-equalized sampler that assigns equal sampling probabilities to all categories.

[0022] Preferably, in step S3, the improved main network includes a branch-level aggregation module. This module aggregates two blocks as a group into a node, and then connects the blocks and nodes on the branches in a tree structure, as expressed by the formula:

[0023]

[0024]

[0025] Among them, O n (x) represents the output of the nth node, and N(x) is the node function. and Let B(x) represent the outputs of the two blocks under the nth node, and let B(x) represent the convolution blocks.

[0026] Preferably, in step S3, the improved main network includes a semantic feature enhancement module, which can be expressed in formulaic language as follows:

[0027]

[0028] Among them, and These represent low-level feature maps X and X, respectively. l At position i and the highest level feature map X h The eigenvalue at position j. This is determined by the transformation function f q (*), f k (*), f v (*) The function value obtained by transformation, F sim F is a function for measuring similarity. mul Let F be the weighted aggregation function. norm The function is the canonical softmax. Then it means The output value at position i after feature interaction transformation;

[0029] After completing all transformations, all new features need to be unified to X. h The size is determined, and finally, the final feature output is obtained through 1×1 convolution.

[0030] Preferably, in step S5, note that in the guidance module, the inputs of the regular learning branch and the reverse learning branch are processed through a shared backbone to obtain the high-level semantic feature vector F. c With F r Then, through continuous upsampling, stitching, and convolution operations, the segmentation probability map is finally obtained.

[0031] Assume the segmentation of the true label is y seg Then the loss function of this auxiliary guidance module It can be represented as:

[0032]

[0033] Where BCE(·) is the binary cross-entropy loss function. These represent the true segmentation labels of the input images for the regular branch and the reverse learning branch, respectively.

[0034] Preferably, step S6 includes the following steps:

[0035] S61. Obtain the weights of the two branches in spatial location; first, use two convolutional layers (conv) with a kernel size of 1×1. sq1 and conv sq2 Features and The compression process halves the number of channels, and then the channels are concatenated to obtain the final product. Next, pass it through ex. sq The function is used to extract the spatial mapping values ​​of two input features:

[0036]

[0037] Where V w ∈R 2×H×W It represents the spatial importance of the pixel values ​​at each point of the two input features; V w Pass it into the Softmax function to obtain and The final weight in spatial location and

[0038]

[0039]

[0040] in, V represents w The pixel value at position (i, j) in the k-th channel. and These respectively represent the features and Pixel weights at spatial location (i, j).

[0041] S62. Obtain the weights of the two branches at the channel position; first, combine the two input features... and The fused features are obtained by adding the elements at the corresponding positions of the corresponding channels. Then, the channel global information f for a specific response is obtained through a global average pooling layer. gap Finally, two separate extraction functions are used to extract the channel mappings of the two branches respectively:

[0042]

[0043] in, For the fully connected operation function (j = 1, 2), W ex-j Let F be its weight parameter. ex-j The values ​​at the corresponding channel positions are also subjected to a Softmax operation to obtain their respective relative weights:

[0044]

[0045]

[0046] Where m represents the current channel as the m-th channel, m∈{1,2...C}, and This represents the set of channel weights for the two branches.

[0047] The final output of the adaptive weight allocation module For a comprehensive weighted average of spatial location and channel response and The result of adding corresponding elements:

[0048]

[0049]

[0050]

[0051] Preferably, in step S7: a is 1 and b is 0.5.

[0052] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0053] 1. This invention provides new assistance to computer-aided physicians in determining whether large blood vessels are blocked and which vessel is responsible for the blockage. It can effectively solve the problem of severe class imbalance in data samples and achieve satisfactory classification accuracy.

[0054] 2. This invention guides the network model's attentional tendency through an attention guidance module, enabling the model to provide more reasonable visual explanations. It can embed relevant prior knowledge into the model, guiding the model's attentional tendency towards the responsible blood vessel area, making the final decision-making behavior more reasonable. This helps deepen doctors' understanding of the deep model decision-making process of this invention and enhances the interpretability of the model's decision-making results.

[0055] 3. This invention uses a conventional learning branch and a reverse learning branch to extract feature information from the majority and minority classes, respectively, to mitigate the impact of class imbalance during training. In addition, a new balancing branch is added to dynamically and adaptively fuse the feature information from the first two branches, obtaining more effective high-level semantic information for better classification performance.

[0056] 4. This invention addresses the difficulty in detecting occluded responsible vessels due to differences in physiological structure by using two modules: branch hierarchical aggregation and semantic feature enhancement. The former retains richer spatial location information by aggregating feature information from different depths of each branch, while the latter obtains more robust semantic information by actively searching and fusing the most valuable features on the feature maps of each branch. Attached Figure Description

[0057] Figure 1 This is a diagram of the overall framework of the multi-branch fusion network of the present invention;

[0058] Figure 2 This is a schematic diagram of the branch hierarchy aggregation module of the present invention;

[0059] Figure 3 This is a schematic diagram of the semantic feature enhancement module of the present invention;

[0060] Figure 4 This is a schematic diagram of the adaptive fusion module of the present invention;

[0061] Figure 5 This is a schematic diagram of the attention guidance module of the present invention;

[0062] Figure 6 This is a schematic diagram of the dataset construction process during the experiment of this invention;

[0063] Figure 7 It is a sample image obtained by projecting CTA images with maximum density;

[0064] Figure 8 This is a visual visualization of the model's decision results. Detailed Implementation

[0065] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined in this application.

[0066] Example 1: As shown in the attached document Figure 1-8 As shown, the present invention describes an attention-guided multi-branch fusion classification method for locating responsible blood vessels, which is essentially a classification network.

[0067] Appendix Figure 7 These are sample images obtained by projecting CTA images to maximum density. The major blood vessels in this application are the bilateral internal carotid arteries and the M1 segment of the bilateral middle cerebral arteries. Figure 7 The left image shows a normal person, where four blood vessels prone to blockage are clearly visible. A is the right middle cerebral artery, B is the right internal carotid artery, C is the left middle cerebral artery, and D is the left internal carotid artery. The left and right positions in the image are reversed compared to reality. The right image shows one of the blood vessels blocked (circled in red). It can be seen that when a blood vessel is blocked, it produces a visual sensation of "disappearance".

[0068] As attached Figure 1 As shown, the multi-branch fusion network of this invention includes three branches: a regular learning branch (Branch C), a reverse learning branch (Branch R), and a balancing branch (Branch B). These branches are used for feature learning in normal individuals, feature learning in patients, and adaptive fusion of features from the first two branches to complete the final classification. The input sample label pairs (x...) of the three branches... c y c ), (x r y r ), (x b y b The samples are obtained by different sampling methods, where x is the training sample, y is the corresponding label, y∈{1,2,…,L}, and L is the number of final classification categories.

[0069] First, sample label pairs from different sampling methods are fed into the corresponding network branches. After passing through the improved main network with weight sharing, the feature vectors F of each branch are obtained. c F r F b For the feature vector F of the regular learning branch (Branch C) and the backsampling branch (Branch R) c F r It not only needs to be processed through a Conv convolutional layer with non-shared weights c Conv rThe obtained feature vector The input is fed into the classification module to obtain the classification probability output p. c p r It also needs to be fed into the attention guidance module, which guides the model's focus, to obtain the segmentation probability map. For the balanced branch (Branch B), only the classification probability output, F, needs to be calculated. b First, it will simultaneously pass through the convolutional layer Conv. c Conv r Obtain the feature vector Then the output of the adaptive fusion module is... The classification probability p of the balanced branch is obtained by feeding it into the classifier. b Finally, based on the classification probability p... c p r p b To construct the cumulative loss of the classification

[0070]

[0071]

[0072] Where, y c y r y b Here, α represents the true classification labels corresponding to the three branches, α is used to shift the focus of model learning, and N represents the current epoch. max This represents the total number of training epochs. The segmentation probability graph... This is then used to construct the auxiliary guidance loss function. Used to guide the model's focus, the final loss function of the entire network can be described as:

[0073]

[0074] Where a and b are the loss weights, with a set to 1 and b set to 0.5.

[0075] Different sampling strategies are adopted for different branches in this invention. For the conventional learning branch (Branch C), which focuses on learning the feature representation of intracranial large blood vessels in normal individuals, a uniform sampler that preserves the original data distribution is used. This ensures that each sample is selected with an equal probability in each sampling period, ultimately extracting more normal samples (because there are far more normal samples than patient samples in the dataset), which is beneficial for learning the feature representation of normal samples. The reverse learning branch (Branch R) uses a reverse sampler, which makes the sampling probability of each class proportional to the inverse of its sample size. That is, the more samples in a class, the lower the sampling probability of that class. The advantage of this is that this branch will pay more attention to patient samples with less data and learn more feature information about the occurrence of vascular occlusion. For the balanced branch (Branch B), this invention uses a class equalization sampler that assigns equal sampling probabilities to all classes. This branch is added to dynamically and adaptively fuse the feature information of the first two branches to obtain more effective classification feature information. Furthermore, this invention also uses the classification results of this branch as the basis for saving the training network model. This not only effectively balances the learning focus among training samples, but also solves the problem of inconsistency between the training and testing phases.

[0076] Due to the unique physiological structure, the length and direction of the major intracranial blood vessels vary from person to person. Therefore, compared to binary classification for LVO (Large Occlusion), accurately identifying the responsible vessel for occlusion is a more challenging multi-class classification task, requiring the backbone network to extract richer and more effective feature information. HR-Net not only preserves high-quality spatial information but also extracts robust contextual semantic information through continuous cross-resolution information exchange, making it well-suited for this multi-class classification task. Therefore, this invention selects HR-Net as its basic backbone network. To reduce the risk of model overfitting, the HR-Net used in this invention has only three stages. To achieve better classification results, this invention improves the original network by designing two unit modules: Branch Hierarchical Aggregation (BHDA) and Semantic Feature Enhancement (SFE). The former aggregates feature information at different depths in each branch and stage to preserve richer spatial information, while the latter actively searches and fuses the most valuable information from the feature maps of each branch to obtain more robust semantic features. Through these two modules, the improved network can extract more effective classification information, thus effectively addressing the difficulty in classifying the responsible vessel for occlusion caused by differences in physiological structure.

[0077] To achieve precise localization of the responsible occluded vessel, the network needs to retain higher-quality spatial feature information. The Branch Hierarchical Aggregation Module (BHDA) excels at this, combining shallower and deeper feature information from each stage within each branch. This allows each branch to retain richer spatial feature information at a relatively low cost, thus contributing to the network's final accurate localization. Its specific structure is as follows: Figure 2 As shown, BHDA aggregates two blocks (each block consists of two 3×3 convolutions) into a node, and then connects the blocks and nodes on the branches in a tree structure. It doesn't simply aggregate the blocks and nodes upwards along the tree; this invention also uses the output of the aggregated node as the input to the next group of blocks to feed back into the main branch, thus better preserving spatial information. This can be expressed as:

[0078]

[0079]

[0080] Among them, O n (x) represents the output of the nth node, and N(x) is the node function. and Let B(x) represent the outputs of the two blocks at the nth node, and let B(x) represent the convolutional blocks. Compared to the outputs obtained from consecutive convolution operations, the final output of BHDA will have richer feature information at different levels and depths.

[0081] To ensure the network's performance in correctly classifying occluded vessels, this invention employs a Semantic Feature Enhancement (SFE) module. This module actively searches and fuses the most valuable features from various branch feature maps to obtain more robust semantic information, thereby improving the network's final performance in classifying occluded vessels. The SFE module is essentially a combination of self-attention and cross-attention, with the specific structure as follows: Figure 3 As shown, this module uses low-level feature map X l For Q, query the highest-level feature map X, which is the key K and value V. h This leads to the acquisition of a new feature map X after active interaction. F Its relationship with X l Same size (when X) l =X h (At this point, it becomes a self-attention mechanism). Through this top-down, non-local interaction, valuable feature information in low-level regions can be enhanced and integrated into high-level feature maps, which is beneficial for improving the network's ability to extract higher-level semantic information. This can be expressed as:

[0082]

[0083] in, and These represent low-level feature maps X and X, respectively. l At position i and the highest level feature map X h The eigenvalue at position j; This is determined by the transformation function f q (*), f k (*), f v (*) The function value obtained by transformation. F sim The function for measuring similarity (for matrix multiplication), F mul F is the weighted aggregation function (for matrix multiplication). norm The function is the canonical softmax. Then it means The output value at position i after feature interaction transformation. After completing all transformations, all new features need to be unified to X. h The size is determined, and finally, the final feature output is obtained through 1×1 convolution.

[0084] The conventional learning branch and the reverse learning branch focus on the vascular features of normal individuals and patients, respectively. Using either branch for prediction will result in bias. The adaptive fusion (AFM) module, however, dynamically and adaptively fuses the feature information from the first two branches, allowing it to effectively balance the learning focus of vascular features between normal individuals and patients, making it suitable for the final classification branch. The specific structure of this AFM module is as follows: Figure 4 As shown, and The characteristic F of the balanced branch are respectively represented. b After passing through the Conv convolutional layers unique to the other two branches c Conv r The resulting new feature vectors, after entering the module, undergo weight allocation in spatial location and channel response along different paths, and then adaptively complete the final feature fusion based on their respective weights. To obtain the weights in spatial location, two convolutional layers with a kernel size of 1×1 are first used. sq1 and conv sq2 Features and The compression process halves the number of channels, and then the channels are concatenated to obtain the final product. Next, pass it through ex. sq The function is used to extract the spatial mapping values ​​of two input features:

[0085]

[0086] Among them, V w ∈R 2×H×W It represents the spatial importance of the pixel values ​​of each point in two input features. Furthermore, in order to capture the spatial dependency between the two features, this application uses V... w Pass it into the Softmax function to obtain and The final weight in spatial location and

[0087]

[0088]

[0089] in, V represents w The pixel value at position (i, j) in the k-th channel. and These respectively represent the features and Pixel weights at spatial location (i, j). Weights at channel locations are obtained by first processing the two input features... and The fused features are obtained by adding the elements at the corresponding positions of the corresponding channels. Then, the channel global information f for a specific response is obtained through a global average pooling layer. gap Finally, fully connected operations are used to compress the features. This application employs two separate extraction functions to extract the channel mappings of the two branches respectively:

[0090]

[0091] in, For the fully connected operation function (j = 1, 2), W ex-j The weighting parameters are defined as follows. To balance the influence of the two input features on the final output along the channel dimension, this application uses F... ex-j The values ​​at the corresponding channel positions are also subjected to a Softmax operation to obtain their respective relative weights:

[0092]

[0093]

[0094] Where m represents the current channel, m∈{1,2...C}, and This represents the set of channel weights for the two branches. The final output of the adaptive weight allocation module For a comprehensive weighted average of spatial location and channel response and The fusion result of adding elements at corresponding positions.

[0095]

[0096]

[0097]

[0098] In the medical field, traditional deep learning models often lack consideration for prior knowledge, resulting in poor interpretability of the results. To address this, this application provides an auxiliary attention guidance module that directs the model to focus more on the responsible blood vessel region, thereby providing a more reasonable explanation for the chosen decision. Its specific structure is as follows: Figure 5 As shown, the inputs of the regular learning (Branch C) and the reverse learning branch (Branch R) are processed through a shared backbone to obtain the high-level semantic feature vector F. c With F r Then, through continuous upsampling, stitching, and convolution operations, the segmentation probability map is finally obtained. Assume the segmentation of the true label is y seg Then the loss function of this auxiliary guidance module It can be represented as:

[0099]

[0100] Where BCE(·) is the binary cross-entropy loss function, These represent the true segmentation labels of the input images for the regular branch and the reverse learning branch, respectively.

[0101] Example 2: Experimental Section

[0102] 1. Data Source

[0103] The original data used in this experiment was provided by the First Affiliated Hospital of Chongqing Medical University. After data cleaning and patient information anonymization, a total of 375 samples were obtained, including 254 normal control group cases and 121 patients (47 cases of right middle cerebral artery occlusion, 23 cases of right internal carotid artery occlusion, 35 cases of left middle cerebral artery occlusion, and 16 cases of right internal carotid artery occlusion). The final dataset construction process used in this application is attached. Figure 6As shown, firstly, both CTA and non-contrast computed tomography (NCCT) images are used to remove redundant information from the skull and other structures that might interfere with the blood vessels. Then, maximum intensity projection processing is used to obtain an intracranial vascular map. Finally, three experienced clinicians complete case-level classification labels and pixel-level annotation of the responsible blood vessels. To better evaluate the classification performance of this application on responsible blood vessels, this embodiment uses sensitivity, precision, accuracy (ACC), and Macro-F1 as evaluation metrics. Sensitivity and precision are used to determine the model's performance in each classification, while ACC and Macro-F1 evaluate the model's overall performance.

[0104] 2. Implementation details

[0105] This experiment was conducted using the PyTorch framework on a GeForce GTX 2070 SUPER with 8GB of RAM. The input image size was 256×256, the batch size was 16, ADAM was selected as the network optimizer, momentum was set to 0.9, and the initial learning rate was 0.0001. The learning rate was increased using a warm-up strategy in the first 10 training epochs, and then decreased at a rate of 0.1 in the 80th and 120th epochs, for a total of 150 epochs. To augment the case data, offline data augmentation was used, performing a horizontal flipping operation only on patients and changing their classification and pixel labels. This augmented data was only used during the training phase. Due to the significant difference in sample sizes between different classes, resulting in severe data imbalance, the training and testing data for healthy individuals and patients were divided into an 8:2 and 6:4 ratio. During the training phase, the three branches received images with different sampling strategies as input, according to the attached... Figure 1 The training process is completed in the middle, and all modules will work. However, during the inference phase, only the balancing branch works normally, and the rest of the parts and various hyperparameters will not have any impact on the results.

[0106] 3. Network backbone selection

[0107] Table 1. Comparison Experiment of Backbone Network Selection

[0108] Resnet18 Res2Net18 Resnext18 Densenet121 MobileNetV3 ShuffienetV2 HRNet Parameters 13.96M 13.84M 15.44M 6.96M 6.04M 6.36M 6.39M ACC 58.82 62.74 53.92 60.78 54.90 55.88 61.76 F1 30.17 44.04 25.22 39.74 29.55 23.53 42.30

[0109] First, we will explain why HR-Net was chosen as the backbone for this experiment. Considering the limited number of samples in our medical dataset, we selected a typical classification network with fewer parameters for the initial experiment (single branch, without any adjustment strategy to the dataset). The results are shown in Table 1. Although HR-Net only achieved the second-best ACC of 61.76%, which is 0.98 percentage points lower than the ACC of 62.74% achieved by Res2Net-18, HR-Net has only half the number of parameters as Res2Net-18. Taking all factors into consideration, this embodiment selected HR-Net as the backbone for subsequent experiments.

[0110] 4. To address the severe class imbalance problem in medical data, and to demonstrate the effectiveness of the network proposed in this application, this embodiment performs a performance evaluation and comparison with other state-of-the-art methods for handling class imbalance problems on a private LVO dataset.

[0111] CDB: This method proposes a loss function that dynamically assigns weights to each sample based on the difficulty of the class to which the sample belongs.

[0112] LDAM-DRW: It proposes a theoretically principled label distribution, which achieves edge loss by minimizing the edge-based generalization bound.

[0113] CB-Focal: This method is designed with a weighted scheme, using the number of valid samples for each class to rebalance the loss function.

[0114] BBN: This paper selects different data sampling strategies for different learning tasks to achieve better performance.

[0115] Table 2. Comparison of Comparison Methods

[0116]

[0117] As shown in Table 2 (the bold red line represents the optimal result, and the bold blue line represents the second-best result), without the auxiliary attention guidance module, compared to other algorithms, the network in this application, while not achieving the best results in sensitivity and accuracy for each category, still achieved most of the second-best results. Furthermore, in terms of overall metrics, the model in this application achieved the best ACC of 79.41% and the best F1-MA of 71.53%, representing improvements of 10.78% and 11.68% respectively compared to the baseline. After adding the attention guidance module, the model's performance further improved, with ACC increasing by 2.94 percentage points to 82.35% and F1-MA increasing by 6.38 percentage points to 77.91%, achieving the best and second-best results for the vast majority of metrics. This indicates that the model method proposed in this application has a more significant advantage in solving the classification dataset of large vessel occlusion with severe data imbalance.

[0118] To make the decisions made by the deep model more convincing, this embodiment uses class activation graph visualization to explain the model's decision-making behavior. Furthermore, a comparison of the actual results before and after adding the auxiliary guidance module further illustrates that this module not only improves the model's classification accuracy but also enhances its interpretability. Figure 8 As shown, each row of images visualizes the performance of all algorithms within the same category. The first column displays the original image, with the responsible vessel area of ​​the occlusion circled in red. From the visualized heatmap, although the compared algorithms and those without the attention guidance module (columns 2-7) focus on areas near the responsible vessel area, these areas are too broad and lack significant differentiation across different vessel occlusion situations, making them unconvincing. However, with the addition of the auxiliary attention guidance module (last column), the model can accurately locate the occlusion area for occluded samples, and for normal samples, the model's focus area is more focused. This demonstrates the effectiveness of the auxiliary attention guidance module.

[0119] 5. Ablation test

[0120] Table 3. Ablation Experiment

[0121]

[0122] US Uniform Sampler

[0123] Ablation experiments were conducted to evaluate the importance of each module of the network to the final result, as detailed in Table 3. It can be observed that the model's prediction performance improves with the addition of different sampling strategy branches, increasing the ACC from 61.76% to 73.53%. This is because different branches focus on learning different information, and combining them yields richer feature information, thus improving the final classification performance. Furthermore, Table 3 also demonstrates the effectiveness of the improved network main module, adaptive fusion module, and auxiliary attention guidance module. Whether used individually or in combination, these modules improve the final classification performance. Notably, the gain achieved by using the auxiliary attention guidance module alone is comparable to that achieved by using the other two modules simultaneously, further highlighting the rationale for including this module. Ultimately, the best performance is achieved by combining all strategies, reaching an optimal ACC of 82.35%.

Claims

1. An attention-guided multi-branch fusion classification method for locating responsible vessels. Its characteristics include the following steps: S1. Construct a multi-branch fusion network, which includes a regular learning branch, a reverse learning branch, and a balancing branch. The regular learning branch is used for feature learning of normal people, the reverse learning branch is used for feature learning of patients, and the balancing branch is used to dynamically fuse the feature information of the regular learning branch and the reverse learning branch to complete the identification of the blocked responsible blood vessel. S2. Obtain sample labels using the sampler. The sample labels for the regular learning branch are: (x c y c The sample labels for the reverse learning branch are: (x r ,y r The sample labels for the balanced branch are: (x b ,y b ), where x is the training sample, y is the corresponding label, y∈{1,2,…,L}, L is the number of final classification categories, c represents the regular learning branch sample, r represents the reverse learning branch sample, and b represents the balanced branch sample; S3. Different sample label pairs are fed into the corresponding network branches, and after passing through the improved main network with weight sharing, the feature vectors F of each branch are obtained. c F r F b , of which F c For the feature vectors of the regular learning branch, F r Feature vectors of the reverse learning branch, F b Eigenvectors of balanced branches; S4. Transfer the feature vector F c After weight-distributed convolutional layers Conv c The obtained feature vector The input is fed into the classification module to obtain the classification probability output p. c , the feature vector F r After weight-distributed convolutional layers Conv r The obtained feature vector The input is fed into the classification module to obtain the classification probability output p. r ; S5. The feature vector F obtained from the conventional learning branch... c The data is fed into the attention guidance module, which guides the model's focus, to obtain the segmentation probability map. The feature vector F obtained from the back learning branch r The data is fed into the attention guidance module, which guides the model's focus, to obtain the segmentation probability map. S6, Feature vector F b Simultaneously passing through the convolutional layer Conv c Conv r After passing through the convolutional layer Conv c Obtain the feature vector After convolutional layer Conv r Obtain the feature vector Then The output of the adaptive fusion module after balancing branches The classification probability p of the balanced branch is obtained by feeding it into the classifier. b ; S7. Based on the classification probability p c p r p b To construct the cumulative loss of the classification Among them, y c y represents the true classification label corresponding to the regular learning branch. r For the true classification label corresponding to the reverse learning branch, y b To balance the true classification labels corresponding to the branches, α is used to shift the focus of model learning, and N represents the current epoch. max This represents the total number of training epochs. According to the segmentation probability map Constructing the auxiliary guidance loss function Used to guide the model's focus, the final loss function of the entire network can be described as: Where a and b are the loss weights.

2. The attention-guided multi-branch fusion classification method for locating responsible vessels according to claim 1, characterized in that, In step S2: the regular learning branch uses a uniform sampler that preserves the original data distribution, the reverse learning branch uses a reverse sampler, and the balancing branch uses a class-equalized sampler that assigns equal sampling probabilities to all categories.

3. The attention-guided multi-branch fusion classification method for locating responsible vessels according to claim 1, characterized in that, In step S3, the improved main network includes a branch-level aggregation module. This module aggregates two blocks into a single node, and then connects the blocks and nodes on the branches in a tree structure, as expressed by the formula: Among them, O n (x) represents the output of the nth node, and N(x) is the node function. and Let B(x) represent the outputs of the two blocks under the nth node, and let B(x) represent the convolution blocks.

4. The attention-guided multi-branch fusion classification method for locating responsible vessels according to claim 1, characterized in that, In step S3, the improved main network includes a semantic feature enhancement module, which can be expressed in formulaic language as follows: Among them, and These represent low-level feature maps X and X, respectively. l At position i and the highest level feature map X h The eigenvalue at position j. This is determined by the transformation function f q (*), f k (*), f v (*) The function value F obtained by transformation sim F is a function for measuring similarity. mul Let F be the weighted aggregation function. norm The function is softmax; Then it means The output value at position i after feature interaction transformation; After completing all transformations, all new features need to be unified to X. h The size is determined, and finally, the final feature output is obtained through 1×1 convolution.

5. The attention-guided multi-branch fusion classification method for locating responsible vessels according to claim 1, characterized in that, In step S5, note that in the guidance module, the inputs of the regular learning branch and the reverse learning branch are processed through the shared backbone to obtain the high-level semantic feature vector F. c With F r Then, through continuous upsampling, stitching, and convolution operations, the segmentation probability map is finally obtained. Assume the segmentation of the true label is y seg Then the auxiliary guidance loss function It can be represented as: Where BCE(·) is the binary cross-entropy loss function. These represent the true segmentation labels of the input images for the regular branch and the reverse learning branch, respectively.

6. The attention-guided multi-branch fusion classification method for locating responsible vessels according to claim 1, characterized in that, Step S6 includes the following steps: S61. Obtain the weights of the two branches in spatial location; first, use two convolutional layers (conv) with a kernel size of 1×1. sq1 and conv sq2 Features and The compression process halves the number of channels, and then the channels are concatenated to obtain the final product. Next, pass it through ex. sq The function is used to extract the spatial mapping values ​​of two input features: Where V w ∈R 2×H×W It represents the spatial importance of the pixel values ​​at each point of the two input features; V w Pass it into the Softmax function to obtain and The final weight in spatial location and in, V represents w The pixel value at position (i, j) in the k-th channel. and These respectively represent the features and Pixel weights at spatial location (i, j); S62. Obtain the weights of the two branches at the channel position; first, combine the two input features... and The fused features are obtained by adding the elements at the corresponding positions of the corresponding channels. Then, the channel global information f for a specific response is obtained through a global average pooling layer. gap Finally, two separate extraction functions are used to extract the channel mappings of the two branches respectively: in, For the fully connected operation function (j = 1, 2), W ex-j Its weight parameters; then F ex-j The values ​​at the corresponding channel positions are also subjected to a Softmax operation to obtain their respective relative weights: Where m represents the current channel as the m-th channel, m∈{1,2...C}, and This represents the set of channel weights for the two branches. The final output of the adaptive weight allocation module For a comprehensive weighted average of spatial location and channel response and The result of adding corresponding elements:

7. The attention-guided multi-branch fusion classification method for locating responsible vessels according to claim 1, characterized in that, In step S7: a is 1, b is 0.5.

Images