Cerebrovascular lesion feature generation method based on multi-instance learning and anatomical structure positioning

Abstract

The invention discloses a cerebrovascular lesion feature generation method based on multi-instance learning and anatomical structure positioning, and the method comprises the steps: obtaining a brain image which comprises a brain CT image and an MRI image; the brain image is preprocessed; positioning and extracting the preprocessed brain image through an anatomical structure positioning network to obtain a cerebrovascular anatomical structure image; wherein the anatomical structure positioning network adopts a deep learning model; performing deep semantic coding on the cerebrovascular anatomical structure image to obtain a high-dimensional feature vector, performing multi-instance weighted fusion on the high-dimensional feature vector through an attention mechanism to obtain a fusion feature, converting the fusion feature to obtain an intermediate feature, and performing label classification and severity regression calculation according to the intermediate feature to obtain a target feature; according to the method, the accuracy and the stability of cerebrovascular lesion feature recognition in CT and MRI images can be remarkably improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a cerebral vascular lesion feature generation method based on multi-instance learning and anatomical structure positioning. BACKGROUND

[0002] With the rapid development of computer vision and artificial intelligence technology, the automatic auxiliary analysis of cerebral vascular lesions based on CT and MRI images has become a research hotspot in the field of medical image processing. The existing technology mainly includes two categories: one is the full-image classification or segmentation method based on convolutional neural network (CNN), which learns features and identifies lesions through deep network on the whole CT or MRI cerebral vascular image; the other is to split the multi-layer CT or multi-sequence MRI image into several key layers, and then independently detect the vascular structure and lesions of each layer image, and finally obtain the global recognition conclusion through simple voting or feature splicing. In addition, some researches also introduce the multi-instance learning (MIL) idea, which regards the whole image sequence as a "bag", and each slice or each blood vessel region in the bag as an "instance", and through the maximum response or attention weighting strategy, the multi-instance features are fused to improve the sensitivity to small vascular lesions. However, the above methods still have the following shortcomings:

[0003] 1. Inaccurate vascular structure positioning - traditional end-to-end CNN or simple layer-level segmentation often ignores the complexity and prior information of cerebral vascular anatomy, making it difficult for the model to distinguish the lesion features of different vascular regions in the image;

[0004] 2. Low efficiency of multi-instance fusion - simply using layer-level maximum response or average pooling cannot adaptively allocate the importance of different instances, and is easily disturbed by noise layers or non-lesion regions, reducing the recognition accuracy;

[0005] 3. Lack of end-to-end unified framework - existing researches are independent in vascular structure positioning, feature extraction and multi-instance fusion, lack of overall optimization, difficult to fully utilize the complementary information between modules, and lack of effective robustness processing for image quality fluctuations (such as artifacts, noise, uneven contrast).

[0006] The main reason for the above problems is that, on the one hand, the cerebral vascular anatomy is complex and variable, and there are differences in blood vessel running and morphology between different patients. For the corresponding image, simple convolution or layer-by-layer detection is difficult to capture the global cross-vessel dependence in the image. On the other hand, the multi-instance information fusion lacks a refined attention mechanism and fails to consider the semantic correlation of the local and global blood vessel lesions in the image. In addition, the preprocessing of CT reconstruction artifacts, MRI magnetic susceptibility artifacts and noise is insufficient, which makes the model performance fluctuate greatly in complex clinical environment. In summary, the cerebral vascular lesion features in CT and MRI images cannot be accurately identified and detected. SUMMARY

[0007] To solve the above technical problems, the present application provides a cerebral vascular lesion feature generation method based on multi-instance learning and anatomical structure positioning to solve the problems existing in the prior art.

[0008] To achieve the above purpose, the present application provides a cerebral vascular lesion feature generation method based on multi-instance learning and anatomical structure positioning, comprising:

[0009] obtaining a brain image, wherein the brain image includes a brain CT image and an MRI image; preprocessing the brain image; positioning and extracting the preprocessed brain image through an anatomical structure positioning network to obtain a cerebral vascular anatomical structure image; wherein the anatomical structure positioning network adopts a deep learning model;

[0010] depth semantic encoding is performed on the cerebral vascular anatomical structure image to obtain a high-dimensional feature vector, multi-instance weighted fusion is performed on the high-dimensional feature vector through an attention mechanism to obtain a fusion feature, the fusion feature is converted to obtain an intermediate feature, and label classification and severity regression calculation are performed according to the intermediate feature to obtain a cerebral vascular lesion label type and severity.

[0011] Optionally, the preprocessing includes denoising and enhancement, geometric correction and alignment, color standardization, and size normalization and cropping.

[0012] Optionally, the process of positioning and extracting the preprocessed brain image includes:

[0013] The preprocessed brain image is positioned and extracted through an anatomical structure positioning network, wherein the anatomical structure positioning network includes a backbone network, a feature pyramid network, a path aggregation network and an identification head connected in turn;

[0014] The preprocessed brain image is processed through the backbone network to obtain feature maps of different levels; wherein the backbone network adopts a CSPDarknet network;

[0015] The feature pyramid network and the path aggregation network are used for fusing the feature maps of different levels to obtain a multi-scale fused feature set;

[0016] The multi-scale fused feature set is labeled and classified by the recognition head to obtain a cerebral vascular anatomical structure image and a corresponding anatomical structure label.

[0017] Optionally, the loss function of the anatomical structure positioning network comprises a classification loss and a regression loss, and the output result of the anatomical structure positioning network is constrained by a non-maximum suppression method.

[0018] Optionally, the process of performing deep semantic encoding on the cerebral vascular anatomical structure image comprises:

[0019] Each of the cerebral vascular anatomical structure images is divided into image blocks of a fixed size, and a linear transformation is performed on the image blocks to obtain a vector sequence of each image block;

[0020] The vector sequence of each image block and the corresponding position code are taken as input data of a Transformer structure, in each layer of the Transformer structure, attention calculation is performed on the input data by a multi-head attention mechanism, and corresponding multi-head parallel calculation results are obtained according to the attention calculation results, the multi-head parallel calculation results are spliced and mapped to obtain mapping data of the current layer;

[0021] For all Transformer structures, the output result of the previous layer of the Transformer structure is combined with the mapping data of the current layer and is normalized, and the normalized data and the normalized data processed by the feedforward network are normalized again to obtain the output data of the current layer of the Transformer structure;

[0022] The output data of all Transformer structures are stacked and a global feature vector is extracted to obtain a high-dimensional feature vector of the anatomical structure.

[0023] Optionally, the process of performing multi-instance weighted fusion on the high-dimensional feature vector comprises:

[0024] The attention weight is obtained by linear mapping according to the high-dimensional feature vectors of a plurality of cerebral vascular anatomical structure images, and the high-dimensional feature vectors of all cerebral vascular anatomical structure images are weighted fused according to the attention weight to obtain a fused feature.

[0025] Optionally, the process of obtaining the attention weight comprises:

[0026]

[0027] wherein, hi represents the feature vector of the i-th instance, W, b represents the learnable parameters of the attention network; w represents the attention network output layer weight; u i represents the attention feature of instance i in the hidden space; a i represents the normalized attention weight of instance i; e i represents the attention score; wherein the instance is a high-dimensional feature vector of the cerebral vascular anatomical structure image.

[0028] Optionally, the process of obtaining the intermediate feature comprises:

[0029] The fusion feature is processed through a fully connected layer and an activation function to obtain an intermediate feature.

[0030] Optionally, the process of label classification and severity regression calculation according to the intermediate feature comprises:

[0031] The intermediate feature is mapped and classified through a Sigmoid function to obtain the label type of the cerebral vascular lesion

[0032] The intermediate feature is linearly regressed to obtain the severity.

[0033] In another aspect, the present application provides a cerebral vascular lesion feature generation system based on multi-instance learning and anatomical structure positioning, which is used to execute the above-mentioned method.

[0034] Compared with the prior art, the present application has the following advantages and technical effects:

[0035] 1. The cerebral vascular anatomical structure positioning network is introduced, which accurately clips the internal carotid artery system, vertebral basilar artery system, Willis circle, deep perforator vessels and other key vascular regions in the image, and has good effect on feature positioning and segmentation in the image.

[0036] 2. The VisionTransformer (ViT) is used to encode the deep features of the vascular structure region in the positioned image, realize unified modeling of fine-grained and global dependence, and improve the multi-aspect feature extraction of the image.

[0037] 3. A multi-instance attention fusion mechanism is designed to adaptively assign the weight of each slice or each vascular instance in the image, eliminate the interference of artifacts, strengthen the lesion features expressed by the image, and reduce external interference and improve the feature highlighting ability of the image.

[0038] 4. An end-to-end pipeline is constructed to integrate and optimize from CT / MRI image preprocessing to pathological feature generator, improve the robustness to artifacts, noise and contrast changes, and realize accurate identification and representation of various pathological morphological labels of cerebral vessels in the image.

[0039] In summary, the scheme designed in the application can significantly improve the accuracy and stability of the recognition of cerebral vascular lesion features in CT and MRI images, provide relevant auxiliary information for early clinical diagnosis, and provide strong information support for auxiliary decision-making. BRIEF DESCRIPTION OF DRAWINGS

[0040] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of this application, and their

[0041] Fig. 1 An image data processing structure schematic diagram of an embodiment of the application;

[0042] Fig. 2 An anatomical structure positioning network structure schematic diagram of an embodiment of the application;

[0043] Fig. 3 A feature extraction network structure schematic diagram of an embodiment of the application. DETAILED DESCRIPTION

[0044] It should be noted that the embodiments in the application and the features in the embodiments can be combined with each other without conflict. The application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0045] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a group of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0046] The application provides a cerebral vascular lesion feature generation method based on multi-instance learning and anatomical structure positioning, which mainly solves the problem that the lesion features of cerebral blood vessels in CT and MRI images cannot be accurately recognized and detected. It should be noted that the technical scheme of the application mainly provides a method for recognizing and extracting complex features in CT and MRI images, and outputs the label of the feature region and the corresponding feature in CT and MRI images. The application can significantly improve the accuracy and stability of the recognition of cerebral vascular lesion features in CT and MRI images, provide relevant auxiliary information for early clinical diagnosis, and provide strong information support for auxiliary decision-making.

[0047] The technical scheme of the application is as follows:

[0048] The method of the application takes brain CT and MRI multi-frame images as input, first carries out noise suppression, geometric correction and color standardization through an image preprocessing module to ensure data quality, then automatically identifies key regions of brain vascular anatomical structure by using an anatomical structure positioning network, then carries out deep semantic coding on each local structure region by means of a Vision Transformer (ViT) feature extraction network to generate corresponding high-dimensional feature vectors, then integrates all structure features into an integrated global lesion representation according to the attention weight of each instance, and finally outputs pathological features or multi-label prediction results of each class by full connection mapping and activation function based on the fusion features, so as to realize accurate positioning and feature description of cerebral vascular lesions.

[0049] As shown in Figs. 1-3 , the above technical solutions are described in detail as follows:

[0050] The cerebral vascular lesion feature generation method based on multi-instance learning and anatomical structure positioning provided by the application has the following data processing procedures of different data processing modules:

[0051] 1. Image preprocessing module:

[0052] In this module, the image is preprocessed by the related preprocessing method to improve the image quality, and the preprocessing method includes denoising and enhancement, geometric correction and alignment, color standardization, and size normalization and cropping, and the specific content includes:

[0053] (1) Denoising and enhancement:

[0054] First, the original endoscopic image I is subjected to noise suppression and contrast enhancement. On the one hand, a Gaussian filter or a bilateral filter model is used:

[0055]

[0056] where I dn denotes the denoised image, denotes the bilateral filter space and intensity kernel respectively, I raw denotes the original brain CT+MRI image, x and y denote the coordinates of the pixels in different directions, and u and v denote the coordinate offset of the neighboring pixels relative to the center pixel.

[0057] Imaging noise is removed; on the other hand, adaptive histogram equalization (CLAHE) is used to enhance the local contrast of each channel to obtain an enhanced image I enh , so that the subsequent anatomical structure is easier to locate.

[0058] (2) Geometric correction and alignment:

[0059] Due to the perspective distortion or rotation offset of different endoscopic frames, the homography matrix is calculated by using the pre-calibrated camera intrinsic and extrinsic parameters by affine or perspective transformation

[0060] I geo (x,y)=I enh (T -1 [x,y,1] T );

[0061] where I geo denotes the geometrically corrected and aligned image, I enh denotes the contrast enhanced image, denotes the camera homography or affine transformation matrix, x, y denote the coordinates of pixels in different directions, respectively, the image is aligned to the standard view angle, ensuring the consistency and comparability of the input of the subsequent anatomical localization network.

[0062] (3) Color normalization:

[0063] In order to eliminate different illuminations and endoscopic device color bias, channel-wise zero mean unit variance normalization is adopted:

[0064]

[0065] where I denotes the channel normalized image, I geo denotes the geometrically corrected and aligned image, μ c and σ c are the mean and standard deviation of the training set or reference whiteboard image in channel c, respectively, so as to unify the color distribution.

[0066] (4) Size normalization and cropping:

[0067] Finally, the normalized image I norm is scaled to a fixed size (H′, W′) by bilinear interpolation or nearest neighbor interpolation, and cropped according to the center or predefined ROI position:

[0068] I out =Crop(Resize(I norm ,H′,W′));

[0069] where I out denotes the output image after preprocessing, I norm denotes the channel normalized image, H′, W′ denotes the fixed height and width of the neural network input, Crop(·) denotes the cropping process, Resize(·) denotes the interpolation process, and the network input I Ensure that the input size of each subsequent submodule is consistent.

[0070] 2. Anatomical structure positioning network:

[0071] The anatomical structure positioning network mainly includes feature extraction, feature fusion, and recognition head modules, which are as follows:

[0072] (1) BackBone feature extraction module:

[0073] This module receives the original CT+MRI RGB image of the brain (H is the height of the feature, and W is the width of the feature map), extracts bottom feature maps of different resolutions and receptive fields through multi-layer convolution and residual connection (the BackBone of the present application adopts the CSPDarknet network). Specifically, the output feature map at the lth stage is:

[0074] Cl = BackBone l (I), l = 3, 4, 5;

[0075] where C l represents the output feature map of the BackBone at the lth layer, I represents the input RGB image, and l represents the layer number label in the BackBone;

[0076] where C3, C4, and C5 are different in spatial size and channel number, and they focus on image details, medium semantics, and deep semantic information, respectively, laying the foundation for subsequent multi-scale fusion.

[0077] (2) Feature pyramid fusion module (FPN / PAN):

[0078] Different levels of feature maps have different focuses in resolution and abstraction. This module first uses a "top-down" pyramid (FPN) structure, reduces C5 to the same channel number as C4 through 1x1 convolution and up-sampling, and then adds it pixel by pixel with C4; similarly, up-sampling fusion to C3 to generate the multi-scale feature map P l after FPN fusion. Subsequently, the optional PAN path from the bottom up enhances the semantic expression of shallow features, obtaining a multi-scale fusion feature set F l ={F3, F4, F5}. These fusion maps not only retain high-level semantic information but also have good spatial positioning ability, suitable for subsequent detection.

[0079] (3) Recognition head - regression branch:

[0080] On each fusion feature set F l , learn to predict the relative offset for each preset anchor or grid (i, j) to describe the true value box (x, y, w, h) relative to the anchor box (xa , y a , w a , h a ) and size difference:

[0081]

[0082] where (x, y), w, h represent the center coordinates, width and height of the ground truth box, (x a , y a ), w a , h a represent the center coordinates, width and height of the anchor box, t x , t y , t w , t h represent the relative offset of the ground truth box to the anchor box in terms of center coordinates, width and height.

[0083] Network learns the relative offset , the predicted bounding box coordinates can be recovered from the anchor box. This branch focuses on refining the spatial location and scale of each type of anatomical structure. By bounding the predicted box, images of different anatomical structures can be obtained.

[0084] (4) Head-classification branch:

[0085] In parallel with the regression branch, the classification branch outputs the confidence of each grid and each structure class j = 1…8, indicating whether the position contains the corresponding anatomical structure. This branch uses the standard binary cross-entropy loss for training to distinguish between background and each class of target, so as to ensure that the network can correctly identify the structure type and suppress false positives.

[0086] (5) Loss calculation and optimization:

[0087] The overall loss of the network is composed of two parts: classification loss and regression loss:

[0088]

[0089] where represents the confidence of the i-th grid being predicted as structure j, y i,j represents the label, whether the grid i belongs to the anatomical structure j (1 or 0), CIoU represents the complete loU loss function, λ represents the regression loss weight hyperparameter, N cls , N reg represent the normalization factors of classification and regression loss, b i,j , represent the predicted bounding box coordinates and the anchor box matched to the real annotation box coordinates corresponding to the i-th grid position and the j-th anchor box.

[0090] where the classification branch measures the accuracy of the confidence prediction, and the regression branch measures the overlap quality between the predicted bounding box and the ground truth bounding box using CloU (Complete IoU), the center distance and the aspect ratio alignment. The hyperparameter λ controls the balance between the two. Through joint optimization, the model can accurately locate each structure and maintain high classification confidence.

[0091] (6) Non-maximum suppression (NMS) post-processing:

[0092] Finally, all candidate boxes for each class j are sorted by confidence, and non-maximum suppression (NMS) is applied to remove highly overlapping low-confidence predictions, leaving a set of high-quality structure localization results {(b j,k ,p j,k ) | p j,k > τ}. Through this post-processing, eight clear and non-interfering anatomical structure detection boxes are obtained for the subsequent lesion feature extraction module.

[0093] 3. Feature extraction network:

[0094] The feature extraction network aims to model the deep semantics of the located anatomical structure region image and obtain high-dimensional representations at the structure level. The present application uses VisionTransformer (ViT) as the core backbone network to convert the structure region image into an embedding representation with global dependency modeling capability. Compared with traditional convolutional networks, ViT has stronger adaptability and expression ability when processing CT+MRI images of the brain with irregular spaces, fuzzy anatomical boundaries, and diverse lesion distributions.

[0095] For each anatomical structure in the image, the subsequent related content is processed in detail:

[0096] (1) Image division and linear embedding:

[0097] Each anatomical structure image is divided into N non-overlapping small blocks (Patches) of size P x P, where P represents the patch size of image division (e.g., 16), and N represents the total number of patches;

[0098] and embedded into a vector sequence of length D through linear transformation:

[0099]

[0100] where represents the embedding vector of the i-th patch, patch i represents the RGB image X k of the k-th anatomical structure region after division, W edenotes a linear projection matrix, flatten denotes a flatten function, i denotes a patch index, and N denotes the total number of patches.

[0101] Position encoding is added and the classification token [CLS] to form the Transformer input sequence:

[0102]

[0103] where Z0denotes the Transformer input (including the position encoding and the [CLS]), x [CLS] denotes the [CLS] embedding vector, denotes the embedding vector of the i-th patch, E pos denotes the position encoding matrix.

[0104] (2) Multi-head self-attention mechanism (MSA):

[0105] The ViT models the relationship between different patches through the multi-head self-attention mechanism. The attention calculation of each layer of the Transformer is as follows:

[0106]

[0107] where the query vector, the key vector, and the value vector are Q = ZW Q , K = ZW K , and V = ZW respectively, Z denotes the MSA input, d = D / h is the dimension of each attention head, and D denotes the Transformer embedding dimension.

[0108] After parallel computation of multiple heads, concatenation and mapping are performed to obtain:

[0109] MSA(Z) = Concat(head1, …, head h )W O ;

[0110] where head h denotes the calculation result of the h-th attention head, MSA(Z) denotes the output result of the final multi-head self-attention mechanism, Z denotes the MSA input, Concat denotes the concatenation operation, and W O denotes the output projection matrix used for linear mapping.

[0111] (3) Feedforward network and residual connection:

[0112] Each layer of the Transformer also includes a feedforward sublayer, which is composed of two fully connected networks and uses the GELU activation function:

[0113] MLP(x) = GELU(xW1 + b1)W2 + b2;

[0114] where x denotes the input of the feed-forward sub-layer, GELU denotes the GELU activation function, W1 and W2 denote the corresponding weights, and b1 and b2 denote the corresponding bias.

[0115] The output of each layer is as follows:

[0116] Z′ l = LayerNorm(Z l-1 + MSA(Z l-1 )), Z l = LayerNorm(Z′ l + MLP(Z′ l ));

[0117] where Z l denotes the intermediate feature after the attention layer, and Z′ l denotes the final layer output after the feed-forward network.

[0118] (4) Global feature extraction and output:

[0119] After L-layer Transformer stacking, the global feature vector is extracted from the [CLS] position as the final representation of the kth anatomical structure:

[0120]

[0121] where Z L denotes the output of the last layer of the Transformer, and z k denotes the global feature vector of the kth structure ([CLS] token).

[0122] This structure-level vector will be used as the input of the multi-instance attention fusion module to capture the pathological features of key anatomical structures in the brain blood vessels in the image.

[0123] 4. Multi-instance attention fusion module:

[0124] The multi-instance attention fusion module aims to aggregate the instance feature vectors from different anatomical structures and multiple images into a global lesion representation through a learnable attention mechanism, providing comprehensive information for the subsequent pathological feature generator. The core idea is inspired by MILattentionpooling, which assigns different weights to a number of instance features according to their importance to the final task, and then sums them up to obtain the fused feature.

[0125] (1) Input instance features:

[0126] The multi-instance attention fusion module first receives N instance feature vectors from the feature extraction network (ViT) Each vector represents a global representation of an anatomical region or a frame of image. These instances can come from different anatomical locations of the same patient, multiple frames of images under different perspectives, or multiple slices of different cases, thus diverse in quantity and spatial distribution. With a unified dimensional representation, the subsequent attention network can evaluate the importance of each instance to the final fused feature in the same feature space.

[0127] (2) Attention scoring sub-network:

[0128] To measure the relative contribution of each instance, a lightweight attention scoring sub-network is introduced. First, the feature h i is linearly mapped and passed through a hyperbolic tangent activation to obtain the hidden representation. Then the vector is calculated to score. All scores are normalized (softmax) to obtain the attention weight α i to reflect the importance of the instance in the overall pattern learning:

[0129]

[0130] where h i represents the feature vector of the i-th instance, with dimension represents the learnable parameters of the attention network; represents the output layer weight of the attention network; u i represents the attention feature of instance i in the hidden space, with dimension α i represents the normalized attention weight of instance i; e i represents the attention score (unnormalized), scalar;

[0131] (3) Weighted fusion:

[0132] After obtaining the normalized weight α i , the multi-instance fusion is realized by weighted summation:

[0133]

[0134] where z represents the global feature vector obtained by weighted fusion, with dimension N represents the total number of instances, i represents the instance label, α i represents the attention weight of the i-th instance, and ∑ i α i = 1,

[0135] This process is equivalent to a weighted average of all instance features according to their attention scores, which not only retains the key information of high-weight instances, but also suppresses noise or irrelevant instances to some extent, thereby generating a more discriminative global representation.

[0136] (4) Output fusion features:

[0137] Finally, the fused feature vector z converges all instance lesion clues in the multi-instance setting, which can be directly used as the input of the subsequent "pathological feature generator" module. Since the dimension of z is the same as that of a single instance h i , it seamlessly connects the subsequent network structure and can carry rich global context and attention weighted information, providing a reliable basis for pathological feature generation.

[0138] 5. Pathological feature generator:

[0139] The specific data processing process of the case feature generator is as follows:

[0140] (1) Fusion feature input:

[0141] The pathological feature generator module takes the global vector output by the multi-instance attention fusion:

[0142]

[0143] where z represents the global feature vector output by the multi-instance attention fusion, D represents the dimension of the fusion feature.

[0144] As the only input, this vector has converged important lesion information of all anatomical structures and multiple images. Here D is the dimension of the fusion feature, which directly determines the input size of the subsequent mapping network.

[0145] (2) Feature mapping subnetwork:

[0146] To convert high-dimensional fusion features into hidden representations suitable for pathological prediction, first pass through a fully connected layer combined with ReLU activation:

[0147]

[0148] where h represents the hidden layer activation output, the feature vector after ReLU, and respectively represent the hidden layer mapping matrix and bias, which are learnable parameters, H is the hidden layer dimension, and z represents the global feature vector output by the multi-instance attention fusion. ReLU activation not only enhances the non-linear representation ability of the network, but also suppresses negative noise to form intermediate features for pathological prediction.

[0149] (3) Pathological classification head:

[0150] For multi-class pathological labels or tags (e.g. inflammation, polyp, hemorrhage, etc. for K classes), a multi-label sigmoid classification head is designed:

[0151] p = σ(W c h + b c ), p ∈ [0, 1] K ;

[0152] where p represents the pathological multi-label probability output, h represents the hidden layer activation output, ranging from [0, 1], W

[0153] (4) Severity regression head:

[0154] For each identified pathology, a linear regression head is designed to evaluate its severity or grade:

[0155]

[0156] where s represents the pathological severity regression output, h represents the hidden layer activation output, W

[0157] (6) Output pathological features:

[0158] Finally, the classification probability p and the regression result s are concatenated or output in parallel to obtain the complete pathological feature vector

[0159]

[0160] It contains both pathological type and severity information, providing rich data information basis for clinical auxiliary diagnosis or subsequent automatic report generation.

[0161] To clearly illustrate the type of anatomical structure region in the image, the following explanation is made, the anatomical structure includes 1. Anterior circulation vascular system: anterior cerebral artery (ACA), middle cerebral artery (MCA), internal carotid artery, anterior communicating artery, posterior communicating artery; 2. Posterior circulation vascular system: vertebral artery, basilar artery, posterior cerebral artery (PCA), superior cerebellar artery (SCA), anterior inferior cerebellar artery (AICA), posterior inferior cerebellar artery (PICA); 3. Willis ring structure: Willis ring; 4. Deep perforator vessels: striate artery, thalamic perforator artery, pontine perforator artery; 5. Venous system: large cerebral vein, superior sagittal sinus / transverse sinus and sigmoid sinus, straight sinus, cortical vein.

[0162] For the above technical solutions, the present application provides the following key points:

[0163] 1. Cerebral vascular anatomical structure positioning network:

[0164] A special lightweight CNN / Transformer hybrid architecture is introduced, which automatically identifies and crops multiple vascular anatomical regions in the image, including the internal carotid artery system, the vertebral basilar artery system, the Willis ring, the deep perforator vessels, and the venous sinus, based on spatial attention and vascular anatomical prior.

[0165] A multi-scale feature pyramid fusion (FPN) is used to improve the positioning accuracy of different caliber blood vessel structures in the image.

[0166] 2. Multi-instance learning framework:

[0167] A set of CT / MRI multi-slice or multi-sequence images is regarded as an instance "Bag", and each slice or each vascular region is regarded as an "Instance", realizing Bag-Instance level unified modeling.

[0168] Based on the learnable attention pooling, dynamic weights are given to each instance to adaptively highlight high lesion information layers and suppress artifacts or irrelevant layers.

[0169] 3. Vision Transformer (ViT) feature extraction:

[0170] Patch Embedding + multi-layer Transformer encoder is used for each blood vessel structure region after positioning, which takes into account local details (Patch level blood vessel morphology) and global dependence (inter-vessel correlation), to generate high-dimensional semantic vectors.

[0171] Three-dimensional position encoding and residual connection are integrated into the Transformer to ensure gradient stability and model interpretability.

[0172] 4. Multi-instance attention fusion module:

[0173] The double-layer attention scoring sub-network is designed, the first layer calculates the structure-level weight according to the difference between vascular structures, and the second layer calculates the layer-level weight according to the change between layers, so as to realize double-weighted fusion across vascular regions and layers.

[0174] The final output is a fusion feature vector, which has stronger vascular lesion discrimination and robustness.

[0175] 5. End-to-end pipeline and robust preprocessing:

[0176] The complete integration, from CT reconstruction optimization, MRI denoising processing, window width and window level standardization, geometric registration correction, contrast enhancement to size normalization, forms an integrated preprocessing flow, which improves the input consistency and anti-artifact ability of the downstream network.

[0177] Batch CT and MRI sequence input is supported, and real-time processing capability is provided.

[0178] 6. Cerebral vascular pathology feature generator:

[0179] Based on the fusion feature vector, a two-layer fully connected + activation mapping is used to output multi-label vascular pathology prediction probability; multiple selection of lesion types (infarction, hemorrhage, aneurysm, stenosis, AVM, etc.), severity assessment and visualization heat map generation are supported.

[0180] The above is only the preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A cerebral vascular lesion feature generation method based on multi-instance learning and anatomical structure positioning, characterized in that, The method comprises the following steps: obtaining a brain image, wherein the brain image comprises a brain CT image and an MRI image; preprocessing the brain image; locating and extracting the preprocessed brain image through an anatomical structure positioning network to obtain a brain blood vessel anatomical structure image; wherein the anatomical structure positioning network adopts a deep learning model; performing deep semantic coding on the brain blood vessel anatomical structure image to obtain a high-dimensional feature vector, performing multi-instance weighted fusion on the high-dimensional feature vector through an attention mechanism to obtain a fusion feature, converting the fusion feature to obtain an intermediate feature, and performing label classification and severity regression calculation according to the intermediate feature to obtain a brain blood vessel lesion label type and severity.

2. The method of claim 1, wherein the preprocessing comprises denoising and enhancement, geometric correction and alignment, color standardization, and size normalization and cropping.

3. The method of claim 1, wherein the process of locating and extracting the preprocessed brain image comprises: locating and extracting the preprocessed brain image through an anatomical structure positioning network, wherein the anatomical structure positioning network comprises a backbone network, a feature pyramid network, a path aggregation network, and a recognition head connected in sequence; processing the preprocessed brain image through the backbone network to obtain feature maps of different levels; wherein the backbone network adopts a CSPDarknet network; fusing the feature maps of different levels through the feature pyramid network and the path aggregation network to obtain a multi-scale fusion feature set; performing bounding box calibration and classification prediction on the multi-scale fusion feature set through the recognition head respectively to obtain a brain blood vessel anatomical structure image and a corresponding anatomical structure label.

4. The method of claim 1, wherein the loss function of the anatomical structure positioning network comprises a classification loss and a regression loss, and the output result of the anatomical structure positioning network is constrained by a non-maximum suppression method.

5. The method of claim 1, wherein the process of performing deep semantic coding on the brain blood vessel anatomical structure image comprises: dividing each of the brain blood vessel anatomical structure images into fixed-size image blocks, performing linear transformation on the image blocks to obtain a vector sequence of each image block; taking the vector sequence of each image block and the corresponding position code as input data of a Transformer structure, in each layer of the Transformer structure, performing attention calculation on the input data through a multi-head attention mechanism, obtaining corresponding multi-head parallel calculation results according to the attention calculation results, and splicing and mapping the multi-head parallel calculation results to obtain mapping data of the current layer; for all Transformer structures, combining and normalizing the output result of the previous layer of the Transformer structure with the mapping data of the current layer, and normalizing the normalized data and the normalized data processed by a feedforward network again to obtain the output data of the current layer of the Transformer structure; ​ ​ ​ ​ Stacking all the output data of the Transformer structure and extracting a global feature vector to obtain a high-dimensional feature vector of the anatomical structure.

6. The method of claim 1, wherein, The process of multi-instance weighted fusion of the high-dimensional feature vector comprises: Obtaining and linearly mapping the high-dimensional feature vectors of a plurality of cerebral vascular anatomical structure images to obtain attention weights, and weighting the high-dimensional feature vectors of all the cerebral vascular anatomical structure images according to the attention weights to obtain fused features.

7. The method of claim 6, wherein, The process of obtaining the attention weights comprises: wherein h i represents the feature vector of the i-th instance, W, b represents the learnable parameters of the attention network; w represents the attention network output layer weight; u i represents the attention feature of instance i in the hidden space; α i represents the normalized attention weight of instance i; e i represents the attention score; wherein the instance is a high-dimensional feature vector of the brain blood vessel anatomical structure image.

8. The method of claim 1, wherein, The process of obtaining the intermediate features comprises: Processing the fused features through a fully connected layer and an activation function to obtain intermediate features.

9. The method of claim 1, wherein, The process of label classification and severity regression calculation according to the intermediate features comprises: Mapping the intermediate features and classifying them through a Sigmoid function to obtain a cerebral vascular lesion label type Linearly regressing the intermediate features to obtain a severity.

10. A cerebral vascular lesion feature generation system based on multi-instance learning and anatomical structure positioning, characterized in that, A device for performing the method of any one of claims 1-9.

Images