Ais lesion prediction method for ctp under attention mechanism and feature fusion

Abstract

The application discloses an AIS lesion prediction method for feature fusion of CTP under an attention mechanism, and mainly relates to the technical field of medical image segmentation and computer-aided medical treatment. The method comprises the following steps: S1, collecting a certain number of multi-modal CTP and MRI images of AIS patients; S2, obtaining a training set; S3, constructing an AIS lesion prediction model for feature fusion of CTP under an attention mechanism; S4, inputting the training set into the AIS lesion prediction model for feature fusion of CTP under the attention mechanism to perform training, so as to obtain a trained AIS lesion prediction model for feature fusion of CTP under the attention mechanism; and S5, sending a multi-modal CTP image of acute ischemic cerebral stroke to be segmented and predicted into the trained prediction model to perform segmentation; and the method can solve the problem that the existing CTP-based AIS disease prediction has poor automatic segmentation effect and cannot meet the actual medical requirements.

Description

Technical Field

[0001] This invention relates to the fields of medical image segmentation and computer-aided medical technology, specifically to an AIS lesion prediction method that uses attention mechanism to fuse features of CTP. Background Technology

[0002] Stroke is one of the leading causes of death and disability worldwide. In terms of incidence, stroke can be divided into two types: ischemic stroke and hemorrhagic stroke. The former has a very high incidence rate, accounting for approximately 75% to 85% of stroke cases, and is extremely dangerous and fatal. Its onset is often due to small vessel occlusion caused by cardioembolism or thromboembolism, or blockage of cerebral arteries. These blockages prevent local brain tissue from receiving sufficient blood and oxygen, ultimately leading to necrosis of the affected area due to hypoxia. In terms of the onset period, stroke can be divided into three main phases: acute, subacute, and chronic. Acute ischemic stroke (AIS), due to its rapid onset, short effective treatment time, and severe condition, is the focus of our research.

[0003] In clinical medicine, doctors first need to assess the patient's condition and then determine a treatment plan based on the assessment. The faster the assessment process and the more accurate the results, the more appropriate the treatment plan can be selected, resulting in less trauma to the patient, more repaired brain tissue, and a lower risk and severity of complications and sequelae. Therefore, accurate and rapid assessment methods have significant clinical importance.

[0004] Currently, the most widely accepted and accurate approach is magnetic resonance imaging (MRI). However, this method requires several hours of MRI scans, further straining the already limited window for optimal treatment. Furthermore, MRI scans can cause brain damage, resulting in secondary injuries. Therefore, the industry is increasingly favoring a faster and less invasive approach: computed tomography perfusion (CTP). When using CTP perfusion parameters to aid in the diagnosis of acute ischemic stroke, traditional medical methods include manual and quantitative analysis. These methods are highly subjective and computationally complex, with results influenced by physician bias, leading to errors and instability. To address this issue, artificial intelligence (AI) technology is widely applied to medical image classification. AI enables rapid and automated segmentation and evaluation of large-scale medical images, improving efficiency and accuracy while reducing physician workload. Therefore, using AI to assist in the assessment of acute ischemic stroke has become a mainstream approach. Summary of the Invention

[0005] The purpose of this invention is to address the problems existing in the prior art and provide an AIS lesion prediction method based on attention mechanism for feature fusion of CTP, thereby solving the problem that the existing CTP-based AIS disease prediction has poor automated segmentation effect and is difficult to meet the actual medical needs.

[0006] To achieve the above objectives, the present invention employs the following technical solution:

[0007] An AIS lesion prediction method based on attention mechanism for feature fusion of CTP includes the following steps:

[0008] S1. Acquire a sufficient number of multimodal CTP and MRI images of AIS patients;

[0009] S2. Convert the original CTP sequence into a pseudo-RGB image. Before conversion, remove patient privacy information around the image and save each parameter as a separate image. Register the labeled data (MRI). Then, use translation, rotation and displacement methods to augment the original dataset for offline data augmentation.

[0010] S3. Construct an AIS lesion prediction model for CTP based on attention mechanism feature fusion;

[0011] S4. Input the preprocessed training set from step S2 into the AIS lesion prediction model for CTP feature fusion constructed in step S3 under the attention mechanism for training. Use the dice and cross-entropy hybrid loss function for backpropagation to update the network parameters, and optimize the model through adaptive moment estimation (Adam) to obtain the trained AIS lesion prediction model for CTP feature fusion under the attention mechanism.

[0012] S5. The multimodal CTP image of acute ischemic stroke to be segmented and predicted is fed into the AIS lesion prediction model that performs feature fusion of CTP under the training attention mechanism obtained in step S4 for segmentation, so as to obtain the segmentation prediction result of the multimodal CTP image of the data.

[0013] Preferably, in step S3, the AIS lesion prediction model for CTP feature fusion under the attention mechanism includes 4 encoder modules, 2 WMHCA feature fusion modules, 2 decoder modules, 1 segmentation prediction module and 1 segmentation optimization module.

[0014] Four encoders correspond to four different modalities of CTP data. After the encoders, a WMHCA feature fusion module is connected. The feature fusion module needs to fuse the features of the two encoders and send them to the decoder. The decoder gradually recovers the detailed information through upsampling and skip connections. The results of the two decoders are input into the segmentation prediction module to generate preliminary prediction results and then sent to the segmentation optimization module. The segmentation optimization module outputs the final prediction result after optimization.

[0015] Furthermore, each stage of the encoder module consists of two 3×3 convolutional units (including activation and normalization layers), one 1×1 convolutional unit (except for the first stage), and one max-pooling layer unit. The output of the previous stage serves as the input of the next stage, connected to the input of the first 3×3 convolutional unit. The output of the first 3×3 convolutional unit is connected to the input of the second 3×3 convolutional unit. The output of the second 3×3 convolutional unit is connected to the input of the max-pooling layer unit. The output of the max-pooling layer unit is connected to the input of the 1×1 convolutional unit, and the output of the 1×1 convolutional unit serves as the output of the encoder module. For each 3×3 convolutional unit: all layers are concatenated sequentially. The input of the 3×3 convolutional layer forms the input of the 3×3 convolutional unit, the output of the 3×3 convolutional unit forms the input of the normalization layer, the output of the batch normalization layer forms the input of the ReLU activation function, and the output of the ReLU activation function forms the output of the 3×3 convolutional unit.

[0016] Furthermore, the WMHCA feature fusion module contains two inputs and one output, with a total of three branches. The inputs of branches 1 and 2 are the two inputs of the WMHCA feature fusion module, respectively. The input of branch 3 is the pixel-wise intersection result of the two inputs of the WMHCA feature fusion module. The inputs of these three branches are connected to a 1×1 convolution. The output of the 1×1 convolution is connected to the input of an unfold unit (a 1×1 convolution with built-in weights of 1 and bias of 0). The output of the unfold unit is connected to the input of a Flatten layer (flattened feature map). The output of the Flatten layer in branch 1 is connected to the input of a Permute layer (transposed). The output of the Permute layer is multiplied by the output of the unfold unit in branch 2. The output after matrix multiplication is connected to the input of the SEBlock module. The output of the SEBlock module is connected to the input of a Softmax layer. The output of the Softmax layer is multiplied by the output of the unfold unit in branch 3 a second time. The output of the second matrix multiplication is restored to the data form of the input data of the WMHCA feature fusion module after passing through the Fold layer and Concat layer. Finally, it is added to the input of branch 3 pixel by pixel, and the result is connected to the output of the WMHCA feature fusion module.

[0017] Furthermore, each stage of the decoder module consists of two 3×3 convolutional units (including activation and normalization layers), one channel-wise cross attention (CCA) unit (except for the first stage), and one deconvolutional layer unit. The output of the previous stage and the output of the same stage of the encoder serve as the input to the CCA unit of the next stage, and the output of the CCA unit and the output of the previous stage serve as the input to the next stage. The input of a stage is connected to the input of the first 3×3 convolutional unit, the output of the first 3×3 convolutional unit is connected to the input of the second 3×3 convolutional unit, and the output of the second 3×3 convolutional unit is connected to the input of the deconvolutional layer unit. The output of the deconvolutional layer unit serves as the output of this stage.

[0018] Furthermore, the segmentation prediction module consists of one 1×1 convolutional unit and one sigmoid layer unit. The outputs of the two decoder modules serve as the inputs of the segmentation prediction module. The inputs of the segmentation prediction module serve as the inputs of the 1×1 convolutional unit, and the outputs of the 1×1 convolutional unit serve as the inputs of the sigmoid layer unit.

[0019] Furthermore, the segmentation optimization module contains a Network module (an aggregation of all previous modules) and a RefineNet module. The Network module contains all previous modules; the inputs of the four encoders serve as the input to the Network module, and the output of the segmentation prediction module serves as the output of the Network module. The output of the Network module needs to have its uncertainty calculated. Taking binary classification as an example, the uncertainty calculation formula is:

[0020] Y = abs(Seg1 - 0.5)

[0021] Seg1 is the output of the Network module, and Y is the uncertainty.

[0022] Based on the uncertainty, the region with the highest uncertainty in the output of the Network module is identified, and its corresponding region is found in the input of the Network module. This region is then input back into the Network module to obtain the second segmentation result. The second segmentation result and the first segmentation result are used together as the input of the RefineNet module. The RefineNet module contains two 3×3 convolutional units, one 1×1 convolutional unit, and two residual block units. The input of the RefineNet module serves as the input of the first 3×3 convolutional unit. The output of the first 3×3 convolutional unit is connected to the input of the second 3×3 convolutional unit. The output of the second 3×3 convolutional unit is connected to the input of the first residual block unit. The output of the first residual block unit is connected to the input of the second residual block unit. The output of the second residual block unit is connected to the input of the 1×1 convolutional unit. The output of the 1×1 convolutional unit serves as the output of the RefineNet module, which in turn serves as the output of the AIS lesion prediction model that performs feature fusion for CTP under the attention mechanism.

[0023] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0024] To fully explore the disease information in CTP images, this invention models and learns the perfusion parameter image information of CTP, effectively extracts features for each parameter, and achieves efficient information fusion. The deep learning model constructed in this invention can achieve relatively accurate segmentation and prediction of AIS disease. Attached Figure Description

[0025] Figure 1 This is a basic flowchart of the AIS lesion prediction method for CTP feature fusion under the attention mechanism of the present invention;

[0026] Figure 2 This is a schematic diagram of the segmentation optimization module of the present invention;

[0027] Figure 3 This is a schematic diagram of the Network module of the present invention;

[0028] Figure 4 This is a schematic diagram of the encoder module;

[0029] Figure 5 This is a schematic diagram of the WMHCA feature fusion module;

[0030] Figure 6 This is a schematic diagram of a partial convolutional structure;

[0031] Figure 7 This is a schematic diagram of the decoder module;

[0032] Figure 8 This is a schematic diagram of the RefineNet module; Detailed Implementation

[0033] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined in this application.

[0034] Example: As attached Figure 1-8 This invention describes an AIS lesion prediction method based on feature fusion of CTP under an attention mechanism, which is a segmentation method for perfusion in MRI images and computed tomography scans, including the following steps:

[0035] S1: CTP and MRI images of 104 patients with AIS were acquired. This invention is based on data collected by the Department of Radiology, Chongqing Medical University.

[0036] S2: First, redundant text information is removed from the CTP images, and parameters are segmented. Then, medical registration is performed on the MRI images to ensure they match the CTP images. Finally, a set of 512×512 pixel CTP parameter and MRI label images is obtained. The image dataset is divided into training and testing sets in a 7:3 ratio. Finally, data augmentation operations such as random rotation, scaling, and mirroring are performed on all images.

[0037] S3: Construct a model for predicting AIS lesions by feature fusion of CTP under the attention mechanism, and set the model parameters.

[0038] Models for AIS lesion prediction methods that fuse features of CTP under the attention mechanism, such as... Figure 2 and 3As shown, it includes an encoder module, a WMHCA feature fusion module, a decoder module, a segmentation prediction module, and a segmentation optimization module.

[0039] (1) Encoder module

[0040] See Figure 4 The encoder module consists of four stages, with the output of each stage connected to the input of the next stage. Each stage comprises two 3×3 convolutional units (including activation and normalization layers), one 1×1 convolutional unit (except for the first stage), and one max-pooling unit. Within the encoder module, the output of one stage serves as the input to the next stage, connecting to the input of the first 3×3 convolutional unit. The output of the first 3×3 convolutional unit connects to the input of the second 3×3 convolutional unit, the output of the second 3×3 convolutional unit connects to the input of the max-pooling unit, the output of the max-pooling unit connects to the input of the 1×1 convolutional unit, and the output of the 1×1 convolutional unit serves as the output of the encoder module.

[0041] See Figure 4 In (a), for each 3×3 convolutional unit: all layers are connected in series, the input of the 3×3 convolution becomes the input of the 3×3 convolutional unit, the output of the 3×3 convolutional unit becomes the input of the normalization layer, the output of the batch normalization layer becomes the input of the ReLU activation function, and the output of the ReLU activation function becomes the output of the 3×3 convolutional unit.

[0042] (2) WMHCA Feature Fusion Module

[0043] See Figure 5 The WMHCA feature fusion module has two input terminals and one output terminal, with a total of three branches. The inputs of branches 1 and 2 are the two input terminals of the WMHCA feature fusion module, respectively. The input of branch 3 is the pixel-wise intersection result of the two input terminals of the WMHCA feature fusion module.

[0044] See Figure 6 (a) The inputs of the three branches are connected to a 1×1 convolution, and the output of the 1×1 convolution is connected to the input of the unfold unit (a 1×1 convolution with built-in weights of 1 and bias of 0).

[0045] The output of the unfold unit is connected to the input of the Flatten layer (flattened feature map). The output of the Flatten layer in branch 1 is connected to the input of the Permute layer (transposed). The output of the Permute layer is multiplied by the output of the unfold unit in branch 2. The output after matrix multiplication is connected to the input of the SE Block module. The output of the SE Block module is connected to the input of the Softmax layer. The output of the Softmax layer is multiplied by the output of the unfold unit in branch 3 for a second time. The output of the second matrix multiplication is then processed through the Fold and Concat layers to restore the data to the input form of the WMHCA feature fusion module. Finally, it is added pixel-by-pixel to the input of branch 3, and the result is connected to the output of the WMHCA feature fusion module.

[0046] (3) Decoder module

[0047] See Figure 7 Each stage of the decoder module consists of two 3×3 convolutional units (including activation and normalization layers), one channel-wise cross attention (CCA) unit (except for the first stage), and one deconvolutional layer unit. The output D of the previous stage and the output O of the encoder in the same stage serve as the input to the CCA unit of the next stage. The output of the CCA unit and the output of the previous stage also serve as the input to the next stage. The input of a stage is connected to the input of the first 3×3 convolutional unit, the output of the first 3×3 convolutional unit is connected to the input of the second 3×3 convolutional unit, and the output of the second 3×3 convolutional unit is connected to the input of the deconvolutional layer unit. The output of the deconvolutional layer unit serves as the output of the current stage.

[0048] See Figure 6 (b) The input of one stage is connected to the input of the first 3×3 convolutional unit, and the output of the first 3×3 convolutional unit is connected to the input of the second 3×3 convolutional unit. For each 3×3 convolutional unit: all layers are concatenated sequentially, the input of the 3×3 convolution becomes the input of the 3×3 convolutional unit, the output of the 3×3 convolutional unit becomes the input of the normalization layer, the output of the batch normalization layer becomes the input of the ReLU activation function, and the output of the ReLU activation function becomes the output of the 3×3 convolutional unit. For the second 3×3 convolutional unit, its input is also connected to the input of the deconvolutional layer, the output of the deconvolutional layer is connected to the input of the normalization layer, the output of the batch normalization layer becomes the input of the ReLU activation function, and the output of the ReLU activation function becomes the output of the deconvolutional unit.

[0049] (4) Segmentation prediction module

[0050] The segmentation prediction module consists of one 1×1 convolutional unit and one Sigmoid layer unit.

[0051] See Figure 6 (c) The outputs of the two decoder modules serve as the inputs of the segmentation prediction module. The input of the segmentation prediction module serves as the input of a 1×1 convolutional unit, and the output of the 1×1 convolutional unit serves as the input of a Sigmoid layer unit.

[0052] (5) Segmentation optimization module

[0053] The segmentation optimization module contains the Network module (an aggregation of all previous modules) and the RefineNet module.

[0054] See Figure 3 The Network module contains all the previous modules. The inputs of the four encoders are used as the inputs of the Network module, and the output of the segmentation prediction module is used as the output of the Network module.

[0055] See Figure 2 Based on the uncertainty, the region with the highest uncertainty in the output of the Network module is identified, and the corresponding region is found in the input of the Network module. This region is then input into the Network module again to obtain the second segmentation result. The second segmentation result and the first segmentation result are used together as the input of the RefineNet module.

[0056] See Figure 8 The RefineNet module contains two 3×3 convolutional units, one 1×1 convolutional unit, and two residual block units. The input of the RefineNet module serves as the input to the first 3×3 convolutional unit. The output of the first 3×3 convolutional unit is connected to the input of the second 3×3 convolutional unit. The output of the second 3×3 convolutional unit is connected to the input of the first residual block unit. The output of the first residual block unit is connected to the input of the second residual block unit. The output of the second residual block unit is connected to the input of the 1×1 convolutional unit. The output of the 1×1 convolutional unit serves as the output of the RefineNet module, which is used as the output of the AIS lesion prediction model that performs feature fusion for CTP under the attention mechanism.

[0057] Each residual block unit contains two 1×1 convolutional units and one 3×3 convolutional unit. The input of the residual block unit serves as the input of the first 1×1 convolutional unit. The output of the first 1×1 convolutional unit is connected to the input of the normalization layer. The output of the batch normalization layer forms the input of the ReLU activation function. The output of the ReLU activation function serves as the input of the 3×3 convolutional unit, which is also connected to the input of the normalization layer. The output of the batch normalization layer and the input of the residual block unit together serve as the input of the ReLU activation function, and the output of the ReLU activation function serves as the output of the residual block unit.

[0058] After building the deep learning model, the initial parameters of the network need to be set.

[0059] In this embodiment, the network parameters and training process include: 100 training rounds, an initial learning efficiency of 10⁻³, and a learning rate that is gradually reduced using a cosine annealing strategy. The overall training process consists of three steps:

[0060] 1) Initial training: At this stage, the network structure is the simplest, without segmentation optimization module and CCA module. After 100 training rounds, proceed to the next step.

[0061] 2) Add the CCA module, using the network parameter values ​​generated in the previous step as initialization parameters, train for 50 rounds using the same strategy, and then proceed to the next step;

[0062] 3) Based on the above, input the segmentation results into the segmentation optimization module and execute the corresponding network process. At this time, the learning rate is 4e-4, and the training is carried out for 50 rounds. The network training is then completed.

[0063] S4: Input the preprocessed training set from step S2 into the model of the AIS lesion prediction method for CTP feature fusion under the attention mechanism constructed in step S3 for training. Use the dice and cross-entropy hybrid loss function for backpropagation to update the network parameters, and optimize the model through adaptive moment estimation (Adam) to obtain the trained AIS lesion prediction model for CTP feature fusion under the attention mechanism.

[0064] Cross-entropy loss function formula:

[0065]

[0066] Dice loss function formula:

[0067]

[0068] Where N represents the number of samples, r n The label represents the sample n, with 1 for positive and 0 for negative. p n This represents the probability that sample n is predicted to be of the positive class.

[0069] S5: The multimodal CTP image of acute ischemic stroke to be segmented and predicted is fed into the AIS lesion prediction model that performs feature fusion of CTP under the training attention mechanism obtained in step S4 for segmentation, so as to obtain the segmentation prediction result of the multimodal CTP image of the data.

Claims

1. An AIS lesion prediction method based on attention mechanism for CTP feature fusion, characterized in that, Including the following steps: S1. Collect a certain number of multimodal CTP and MRI images of AIS patients; S2. Convert the original CTP sequence into a pseudo-RGB image. Before conversion, remove patient privacy information around the image and save each parameter as a separate image. Register the label data, then expand the original dataset and perform offline data augmentation to obtain the training set. S3. Construct an AIS lesion prediction model for CTP based on attention mechanism feature fusion; The AIS lesion prediction model for CTP using the attention mechanism includes 4 encoder modules, 2 WMHCA feature fusion modules, 2 decoder modules, 1 segmentation prediction module, and 1 segmentation optimization module. Four encoders correspond to four different modal data of CTP. After the encoders, the WMHCA feature fusion module is connected. The feature fusion module needs to fuse the features of the two encoders and send them to the decoder. The decoder gradually recovers detailed information through upsampling and skip connections; the results from the two decoders are input into the segmentation prediction module to generate preliminary prediction results, which are then sent to the segmentation optimization module. The segmentation optimization module outputs the final prediction result after optimization. The WMHCA feature fusion module has two input terminals and one output terminal, with a total of three branches. The inputs of branches 1 and 2 are the two input terminals of the WMHCA feature fusion module, respectively. The input of branch 3 is the pixel-wise intersection result of the two input terminals of the WMHCA feature fusion module. The inputs of the three branches are connected to a 1×1 convolution, the output of the 1×1 convolution is connected to the input of the unfold unit, and the output of the unfold unit is connected to the input of the flatten layer. The output of the Flatten layer in branch 1 is connected to the input of the Permute layer, and the output of the Permute layer is multiplied by the output of the Unfold unit in branch 2. The output after matrix multiplication is connected to the input of the SE Block module, the output of the SE Block module is connected to the input of the Softmax layer, and the output of the Softmax layer is multiplied a second time with the output of the unfold unit of branch 3. The output of the second matrix multiplication is restored to the data form of the input data of the WMHCA feature fusion module after passing through the Fold layer and the Concat layer. Finally, it is added to the input of branch 3 pixel by pixel, and the result is connected to the output of the WMHCA feature fusion module. S4. Input the preprocessed training set from step S2 into the AIS lesion prediction model for CTP under the attention mechanism constructed in step S3 for training. Use the dice and cross-entropy mixed loss function for backpropagation to update the network parameters, and optimize the model through adaptive moment estimation to obtain the trained AIS lesion prediction model for CTP under the attention mechanism. S5. The multimodal CTP image of acute ischemic stroke to be segmented and predicted is fed into the AIS lesion prediction model that performs feature fusion of CTP under the training attention mechanism obtained in step S4 for segmentation, so as to obtain the segmentation prediction result of the multimodal CTP image of the data.

2. The AIS lesion prediction method based on feature fusion of CTP under the attention mechanism according to claim 1, characterized in that, In step S2, the methods for expanding the original dataset include translation, rotation, and displacement.

3. The AIS lesion prediction method based on feature fusion of CTP under the attention mechanism according to claim 1, characterized in that, Each stage of the encoder module consists of two 3×3 convolutional units, one 1×1 convolutional unit, and one max pooling layer unit. The output of the previous stage serves as the input of the next stage, connected to the input of the first 3×3 convolutional unit. The output of the first 3×3 convolutional unit is connected to the input of the second 3×3 convolutional unit. The output of the second 3×3 convolutional unit is connected to the input of the max pooling layer unit. The output of the max pooling layer unit is connected to the input of the 1×1 convolutional unit. The output of the 1×1 convolutional unit serves as the output of the encoder module. For each 3×3 convolutional unit: all layers are concatenated in sequence, the input of the 3×3 convolution becomes the input of the 3×3 convolutional unit, the output of the 3×3 convolutional unit becomes the input of the normalization layer, the output of the batch normalization layer becomes the input of the ReLU activation function, and the output of the ReLU activation function becomes the output of the 3×3 convolutional unit.

4. The AIS lesion prediction method based on feature fusion of CTP under the attention mechanism according to claim 1, characterized in that, Each stage of the decoder module consists of two 3×3 convolutional units, one channel cross-attention unit, and one deconvolutional layer unit; The output of the previous stage and the output of the encoder in the same stage are used together as the input of the CCA unit of the next stage, and the output of the CCA unit and the output of the previous stage are used together as the input of the next stage. The input of a stage is connected to the input of the first 3×3 convolutional unit, the output of the first 3×3 convolutional unit is connected to the input of the second 3×3 convolutional unit, the output of the second 3×3 convolutional unit is connected to the input of the deconvolutional layer unit, and the output of the deconvolutional layer unit is used as the output of this stage.

5. The AIS lesion prediction method based on feature fusion of CTP under the attention mechanism according to claim 1, characterized in that, The segmentation prediction module consists of one 1×1 convolutional unit and one sigmoid layer unit; the outputs of the two decoder modules serve as the inputs of the segmentation prediction module; the inputs of the segmentation prediction module serve as the inputs of the 1×1 convolutional units, and the outputs of the 1×1 convolutional units serve as the inputs of the sigmoid layer units.

6. The AIS lesion prediction method based on feature fusion of CTP under the attention mechanism according to claim 5, characterized in that, The segmentation optimization module contains a Network module and a RefineNet module. The Network module includes all previous modules; the inputs of the four encoders are used as the inputs to the Network module, and the output of the segmentation prediction module is used as the output of the Network module. The output of the Network module needs to have its uncertainty calculated. Taking binary classification as an example, the uncertainty calculation formula is: Y = abs(Seg1 - 0.5) Seg1 is the output of the Network module, and Y is the uncertainty. Based on the uncertainty, the region with the highest uncertainty in the output of the Network module is found, and the corresponding region is found in the input of the Network module. This region is then input into the Network module again to obtain the second segmentation result. The second segmentation result and the first segmentation result are used together as the input of the RefineNet module. The RefineNet module contains two 3×3 convolutional units, one 1×1 convolutional unit, and two residual block units. The input of the RefineNet module serves as the input of the first 3×3 convolutional unit. The output of the first 3×3 convolutional unit is connected to the input of the second 3×3 convolutional unit. The output of the second 3×3 convolutional unit is connected to the input of the first residual block unit. The output of the first residual block unit is connected to the input of the second residual block unit. The output of the second residual block unit is connected to the input of the 1×1 convolutional unit. The output of the 1×1 convolutional unit serves as the output of the RefineNet module. The output of the RefineNet module serves as the output of the AIS lesion prediction model that performs feature fusion for CTP under the attention mechanism.

Images