Ischemic stroke onset time prediction model based on dwi and flair images

Abstract

The application discloses an ischemic stroke onset time prediction model based on DWI and FLAIR images, and the model comprises the following algorithm modules: 1) a domain self-adaption algorithm module based on multi-layered adversarial learning; 2) a Key Slice Selection algorithm module; and 3) a Prior Guided Feature Enhancement algorithm module. The application adopts a domain self-adaption strategy based on multi-layered adversarial learning for the domain offset problem, aligns the distribution of a source domain and a target domain on an output space through adversarial learning of a cross-domain segmentation network and a domain discriminator, and improves the cross-domain and generalization capabilities of the cross-domain segmentation network. The ischemic stroke resolution capability of an encoder is reinforced through a mismatch segmentation loss, so that the distinguishability of the features of the ischemic stroke lesion area is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image analysis, in particular to an ischemic stroke onset time prediction model based on DWI and FLAIR images. BACKGROUND

[0002] Stroke is one of the leading causes of death worldwide, among which acute ischemic stroke (AIS) is the main type of onset. At present, intravenous thrombolysis rt-TA is the main means for treating AIS, but this treatment has a strict time window (4.5 hours), and if the time window of 4.5 hours is exceeded, the use of this therapy will increase the risk of intracranial hemorrhage in patients. Therefore, the judgment of whether the AIS patient's stroke onset time (TSS) is within 4.5 hours is the key to clinical treatment.

[0003] Research has found that when stroke occurs, ischemic tissue can be immediately observed on the DWI image, while the corresponding area of the FLAIR image needs to be observed for another 3-4 hours to observe ischemic tissue (as shown in FIG. 1). This mismatch pattern is called DWI-FLAIR mismatch, which is defined as the appearance of high signal on the DWI image indicating the absence of high signal in the corresponding area of the FLAIR image. Therefore, DWI-FLAIR mismatch is used to judge whether the stroke onset time of the patient is within 4.5 hours. However, in actual diagnosis, the human eye is difficult to detect the micro features in the DWI-FLAIR image and the evaluation process relies on the subjective experience of the doctor, so the accuracy of the artificial DWI-FLAIR mismatch in clinically evaluating the onset time of AIS patients is not high. Figure 1

[0004] ​With the development of computer vision technology, image analysis methods based on machine learning and deep learning have become the frontier technology of medical image analysis. Because machine vision has the advantages of strong color and gray scale resolution, large light sensing range, high resolution, etc., it can realize lesion recognition at a microsecond level in a complex environment, and can solve the problems of subjectivity, inefficiency and high misjudgment rate of human eye reading. At present, the TSS classification algorithm based on machine learning uses various methods to obtain ROI mask to locate the stroke area, and then extracts the ROI features with the help of medical image tools, and manually selects some features for modeling. Such methods depend on the image knowledge and experience of researchers, and the feature selection process is difficult, which requires a lot of time and effort. In addition, the structure of the machine learning model is relatively simple, which is difficult to fully capture the deep features of the image, and the TSS classification algorithm based on machine learning has the problems of being easily disturbed by abnormal data and poor robustness. With the development of deep learning technology, deep learning has the advantages of large number of parameters and strong robustness, which not only avoids the limitation of manually designing feature models in machine learning, but also can better capture the deep features and semantic information in medical images, and improve the accuracy and reliability of medical image analysis. At present, deep learning algorithms have surpassed traditional machine learning algorithms in some challenging medical image tasks such as brain tumor segmentation and lung vessel segmentation.

[0005] However, the algorithm for solving the TSS classification problem by using deep learning needs to design the network and optimize the parameters for the following problems. In order to locate the stroke area, a segmentation network can be used to automatically segment the stroke ROI mask. The data set collected in actual diagnosis and treatment (target domain) often lacks stroke annotation required for segmentation network training, so in order to save the time of manual annotation data, other data sets with stroke annotation (source domain) can be used to train the segmentation network. However, due to the different scanning imaging conditions of each medical institution, there are distribution differences (domain shift) between different data sets, which makes the well-trained segmentation network unable to be applied across domains. In addition, part of the stroke is distributed in the form of points along the brain groove or brain surface, and the corresponding stroke ROI area is small. In the process of CNN downsampling, the ROI features are severely lost, which cannot provide effective feature information for TSS classification. Moreover, in the FLAIR image, the contrast of stroke lesions and other tissues is low, and similar to other high-signal forms such as brain tumors and cerebrospinal fluid, it is difficult for the deep network to clearly distinguish the boundary and shape of the stroke lesions. SUMMARY

[0006] The present application aims to at least partially overcome the above technical problems and / or other potential problems in the prior art: to provide an ischemic stroke onset time prediction model based on DWI and FLAIR images.

[0007] The technical solutions of the present application are as follows: an ischemic stroke onset time prediction model based on DWI and FLAIR images, the model comprising the following algorithm modules:

[0008] 1) a domain adaptation algorithm module based on multi-layer adversarial learning, which adopts a domain adaptation algorithm based on multi-layer adversarial learning, encourages the cross-domain segmentation network to deceive the domain discriminator, and generates segmentation results similar to the source domain distribution in the target domain, reduces the distribution difference between the source domain and the target domain in the output space to solve the domain shift problem between the two domains;

[0009] 2) a Key Slice Selection algorithm module, which selects key slices containing rich ROI features from a set of MRI image slices of a patient, reduces the feature loss caused by CNN downsampling, and provides reliable judgment basis for TSS classification;

[0010] 3) a Prior Guided Feature Enhancement algorithm module, which constructs a mismatch segmentation loss L ms , strengthens the discrimination ability of the Encoder to the stroke features, to solve the confusion problem of stroke lesions and other tissue features;

[0011] 4) an Attention Augment Joint Predication algorithm module, which refines the representation of stroke features in the ROI feature vector using LesionAttention to improve the performance of TSS classification.

[0012] The beneficial effects of the present application are: the present application adopts a domain adaptation strategy based on multi-layer adversarial learning to solve the domain shift problem, aligns the distributions of the source domain and the target domain in the output space through adversarial learning of the cross-domain segmentation network and the domain discriminator, and improves the cross-domain and generalization ability of the cross-domain segmentation network. The present application uses a key slice selection module to solve the ROI feature loss problem in the downsampling process, filters out the slices whose features are easily lost in the downsampling process, and selects the slices that can provide rich features to participate in the classification task. The present application proposes a prior guided feature enhancement module to solve the confusion problem between stroke lesions and other tissue features, and uses a mismatch segmentation loss to strengthen the training of the Encoder stroke discrimination ability to improve the distinguishability of the stroke lesion area features. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 DWI-FLAIR mismatch and DWI-FLAIR matching example graphs in the background art.

[0014] Figure 2 Model schematic diagram of the present application.

[0015] Figure 3 Field adaptation training algorithm schematic diagram.

[0016] Figure 4 Prior mismatch ROI pseudo-label construction process schematic diagram. DETAILED DESCRIPTION

[0017] The application will be further described in detail below with specific examples, but the application is not limited to the following specific examples.

[0018] EMBODIMENT

[0019] As shown in the embodiment, a stroke onset time prediction model based on DWI and FLAIR images is provided, and the model comprises the following algorithm modules: Figure 2

[0020] 1) A field adaptation algorithm module based on multi-layer adversarial learning, which adopts a field adaptation algorithm based on multi-layer adversarial learning, encourages the cross-domain segmentation network to deceive the domain discriminator, and generates segmentation results similar to the source domain distribution in the target domain, reduces the distribution difference between the source domain and the target domain in the output space to solve the domain offset problem between the two domains;

[0021] 2) A Key Slice Selection algorithm module, which selects key slices containing rich ROI features from a group of MRI image slices of a patient, reduces the feature loss caused by CNN downsampling, and provides reliable judgment basis for TSS classification;

[0022] 3) A Prior Guided Feature Enhancement algorithm module, which constructs a mismatch segmentation loss L ms to strengthen the discrimination ability of the Encoder to the stroke features, so as to solve the confusion problem of stroke lesions and other tissue features;

[0023] 4) An Attention Augment Joint Predication algorithm module, which refines the representation of stroke features in the ROI feature vector by using Lesion Attention, so as to improve the performance of TSS classification.

[0024] Field adaptation based on multi-layer adversarial learning

[0025] ​Due to different imaging conditions such as imaging devices, scanning protocols, etc., the DWI images of different data sets (such as the ISLES 2022 data set and the ADT data set) have distribution differences, thereby causing domain bias problems. The present application considers reducing the distribution differences of the source domain and the target domain in the output (segmentation) space to solve the domain bias problems existing between the domains. The present application adopts a domain self-adaptive algorithm of adversarial learning of a domain discriminator and a cross-domain segmentation network, and the cross-domain segmentation network generates a segmentation result with a similar distribution to the source domain in the target domain, and aligns the distributions of the source domain and the target domain in the output space.

[0026] The adaptive algorithm includes a cross-domain segmentation network G and a domain discriminator D. The present application inputs the segmentation results of the source domain and the target domain into the domain discriminator D respectively to distinguish whether the segmentation results belong to the source domain or the target domain. The present application uses a binary cross-entropy loss L d The supervised domain discriminator has the following formula:

[0027]

[0028] Wherein, is the segmentation result output by the cross-domain segmentation network G for the image I, wherein C is the number of categories. When P is a segmentation result from the target domain, z=0; when P is a segmentation result from the source domain, z=1. is the discrimination result output by the discriminator, wherein H'=H / S, W'=W / S, S is a down-sampling multiple, and H and W are the height and width of the DWI image respectively.

[0029] When training the cross-domain segmentation network, the present application uses a cross-entropy loss to supervise the segmentation result of the source domain:

[0030]

[0031] Wherein I s and Y s are the source domain picture and its segmentation label respectively. P s =G(I s ) is the segmentation result of the cross-domain segmentation network G for the source domain picture I s .

[0032] The present application also uses an adversarial loss to encourage the cross-domain segmentation network G to deceive the domain discriminator, so that the distribution of the segmentation result P t generated in the target domain is similar to that of the source domain segmentation result P s . The formula of the adversarial loss is as follows:

[0033]

[0034] Wherein I t is the target domain picture, and P t =G(I t) is a cross-domain segmentation network G for target domain image I t The segmentation result.

[0035] To mitigate the vanishing gradient problem, this invention employs a multi-layer adversarial learning strategy. For example... Figure 3 As shown, in addition to the output layer features, lower-layer features (the penultimate layer features) are also input into the domain adaptation module for adversarial learning. This allows for more complete extraction of semantic information from the lower-layer features, reducing the impact of semantic bias. The loss of the cross-domain segmentation network during multi-layer adversarial learning is:

[0036]

[0037] in and This is a hyperparameter for the proportional weighting of the segmentation loss and the adversarial loss.

[0038] Key slice selection module

[0039] In MRI images, the same stroke lesion area appears in multiple consecutive slices. Within each slice, the size of the stroke region of interest (ROI) varies. Compared to smaller ROIs, larger ROIs are less prone to feature loss during downsampling, providing more complete and robust feature information.

[0040] To address this, the present invention designs a key slice selection module that, based on the total area of ​​the ROI, filters out key slices that can provide rich features for the classification task. For example... Figure 2 As shown, the cross-domain segmentation network G processes each DWI image I of a patient in the target domain. t Perform stroke segmentation and assign the segmentation result P t =G(I t After binarization, it is used as a ROI mask. The ROI mask is defined as M = {M} (i，j)}(1≤i≤H,1≤j≤W), where: M (i，j) =[max(P t (i，j，1) )>τ]

[0041] If P t If the stroke prediction confidence of a certain point is greater than τ, then the prediction category of that point is considered to be stroke. In this invention, the value of τ is set to 0.5.

[0042] Then, calculate the total area of ​​the ROI for each ROI mask M, using the following formula:

[0043]

[0044] According to the formula R(I) tThe total area of the ROI of each slice is calculated, and the slice with the largest total area of the ROI is selected as the key slice to participate in the subsequent classification task.

[0045] Prior guided feature enhancement

[0046] In the FLAIR image, the contrast of the stroke lesion area and other tissues is low, and it is easy to be interfered by other high signals (such as brain tumors and cerebrospinal fluid), which makes it difficult for the Encoder to distinguish the stroke area from other tissue areas. In view of this, the present application designs a mismatch segmentation loss to guide the Encoder to strengthen the training of the boundary and morphology of the stroke area, and enhances the feature extraction ability of the Encoder to the stroke area. The present application uses a cross-domain segmentation network to mark the stroke high signal area in the DWI image as an ROI mask. On this basis, the present application can obtain the time of onset according to the prior knowledge of DWI-FLAIR mismatch combined with the TSS classification label, and construct a prior mismatch pseudo label (Prior Mismatch PseudoLabel) (as shown in Figure 4 The construction scheme is as follows:

[0047] (1) When the time of onset is less than or equal to 4.5 hours, the label of the ROI area is set to 0, indicating that the corresponding area of FLAIR should not exist stroke high signal representation;

[0048] (2) When the time of onset is greater than 4.5 hours, the label of the ROI area is set to 1, indicating that the corresponding area of FLAIR should exist stroke high signal representation.

[0049] Definition formula of prior mismatch pseudo label Wherein

[0050]

[0051] Y cls is the label of the time of ischemic stroke of the patient. When the time of onset of the patient is less than or equal to 4.5 hours, Y cls = 0; when the time of onset of the patient is greater than 4.5 hours, Y cls = 1.

[0052] The present application constructs a mismatch segmentation loss to strengthen the ability of the Encoder to distinguish the stroke area from other tissue areas, which is beneficial to extract the fusion ROI features with distinguishing characteristics. The present application takes the segmentation loss in the ROI area as the mismatch segmentation loss, and its formula is as follows:

[0053]

[0054] where P f = G(I f ) is the segmentation result of the FLAIR image I f by the cross-domain segmentation network G.

[0055] Weighted joint prediction

[0056] The stroke hyperintensity of the same patient appears in multiple FLAIR slices, and the significance of the stroke hyperintensity of each FLAIR slice is different. The information amount of the stroke contained in the ROI feature vectors of different slices is different. The stroke feature of some slices is not obvious and is submerged in other features, which causes the classifier to be unable to fully utilize the stroke feature for classification prediction. Therefore, a single slice cannot make a clear classification prediction, and it is necessary to refine the representation of the stroke feature in the ROI feature vector and integrate the features of multiple slices to realize reliable TSS classification prediction.

[0057] To solve the above problems, the present application proposes a weighted joint prediction classifier, the structure of which is shown in Figure 2 The feature extraction module extracts key slice features and pools them into a set of ROI feature vectors V = {V i}(1≤i≤α). In order to make the full connection layer classifier fully utilize the stroke feature in the classification process, the present application proposes Lesion Attention to weight the ROI feature vectors and refine the representation of the stroke feature in the ROI feature vector. The value of Lesion Attention represents the proportion of the information amount of the stroke contained in the ROI feature vector, which can be defined as

[0058]

[0059] where, represents the full connection layer weight, and L represents the dimension of the ROI feature vector.

[0060] The present application uses the TSS classification label to supervise , and the loss formula is as follows:

[0061]

[0062] When the onset time of the patient is less than or equal to 4.5 hours, Y cls = 0; otherwise, Y cls = 1.

[0063] Multiply Lesion Attention A l and the feature vector of the ROI in sequence to enhance the representation of the stroke feature in the ROI feature vector, and the enhanced ROI feature vector is defined as:

[0064]

[0065] Finally, the vectors in are concatenated and input into a fully connected classifier to jointly predict the time-to-onset, which is supervised by the TSS classification labels:

[0066] where N is the number of patient samples.

[0067] are the parameters of the fully connected classifier. Evaluation index

[0068] The present application selects accuracy (Accuracy, ACC), precision (Precision, PR), F1 score (F1-score, F1) and AUC to evaluate the classification performance of the network. The accuracy is the proportion of correctly classified samples to the total number of samples.

[0069]

[0070] The precision is the proportion of samples whose actual class is positive in the samples correctly identified as positive by the classifier.

[0071]

[0072] The F1 score considers the precision and recall comprehensively, and is the harmonic mean of the accuracy and the recall.

[0073]

[0074] The AUC is an index for evaluating the performance of a binary classification model, which represents the area size under the ROC curve.

[0075]

[0076] Comparative experiment

[0077] As shown in Table 1, the classification accuracy (81.1%), precision (76.7%), F1 score (86.8%) and AUC (75%) of the TSS classification network proposed by the present application reached the best level in the comparative experiment.

[0078]

[0079]

[0080] Table 1 Comparison of classification performance on ADT dataset

[0081] ​The above merely illustrates the embodiments of the present application, and does not constitute any limitation on the protection scope of the present application. Any technical scheme formed by equivalent exchange or equivalent replacement falls within the protection scope of the present application.

Claims

1. A predictive model for the onset time of ischemic stroke based on DWI and FLAIR images, characterized in that, The model includes the following algorithm modules: 1) Domain Adaptive Algorithm Module Based on Multi-Layer Adversarial Learning: This module adopts a domain adaptive algorithm based on multi-layer adversarial learning, which encourages cross-domain segmentation networks to deceive the domain discriminator and generate segmentation results in the target domain that are similar in distribution to the source domain. This reduces the distribution difference between the source and target domains in the output space to solve the domain offset problem between the two domains. 2) Key Slice Selection Algorithm Module: This module selects key slices containing rich ROI features from a set of MRI image slices of a patient, reducing feature loss caused by CNN downsampling and providing a reliable basis for TSS classification; the cross-domain segmentation network G processes each DWI image I of a patient in the target domain. t Perform stroke segmentation and assign the segmentation result P t =G(I t After binarization, it is used as an ROI mask; 3) Prior Guided Feature Enhancement algorithm module, which constructs a mismatch segmentation loss L ms This enhances the encoder's ability to distinguish stroke features in order to address the problem of confusion between stroke lesions and other tissue features. Based on prior knowledge of DWI-FLAIR mismatch and combined with TSS classification labels, the onset time can be determined, and a prior mismatch pseudo-label is constructed. The segmentation loss within the ROI region is then used as the mismatch segmentation loss, and the formula is as follows: Among them, P f =G(I f ) is a cross-domain segmentation network G for FLAIR image I f The segmentation results, H and W are the height and width of the DWI image, respectively, Y m This is a priori mismatch pseudo-label; 4) The Attention Augment Joint Predication algorithm module utilizes Lesion Attention to refine the representation of stroke features in the ROI feature vector, thereby improving the performance of TSS classification. The Attention Augment Joint Predication algorithm module extracts key FLAIR slice features and pools them into a set of ROI feature vectors V = {V...} i }, 1≤i≤α; Lesion Attention is used to weight the ROI feature vectors to refine the representation of stroke features in the ROI feature vectors; the value of Lesion Attention represents the proportion of stroke information contained in the ROI feature vector, which is defined as in, represents the weights of the fully connected layer, and L represents the dimension of the ROI feature vector; Lesion Attention A l By multiplying the feature vectors of the ROI sequentially, the representation of stroke features in the ROI feature vectors is enhanced. The enhanced ROI feature vectors are defined as follows: Finally The vectors in the dataset are concatenated and then fed into a fully connected classifier to jointly predict the onset time, with TSS classification labels used for supervision. Where N is the number of patient samples. Y represents the parameters of the fully connected classifier. cls This is a label indicating the onset time of the patient's ischemic stroke.

2. The ischemic stroke onset time prediction model based on DWI and FLAIR images according to claim 1, characterized in that, The domain adaptation algorithm module based on multi-layer adversarial learning includes a cross-domain segmentation network G and a domain discriminator D. The segmentation results of the source and target domains are respectively input into the domain discriminator D to distinguish whether the segmentation result belongs to the source domain or the target domain. Specifically, a binary cross-entropy loss L is used. d The supervised domain discriminant is formulated as follows: in, Z is the segmentation result output by the cross-domain segmentation network G on image I, where C is the number of categories; when P is the segmentation result from the target domain, z = 0; when P is the segmentation result from the source domain, z = 1. It is the discrimination result output by the discriminator, where H′=H / S, W′=W / S, S is the downsampling factor, and H and W are the height and width of the DWI image, respectively.

Images