A method and system for automatic annotation and visualization of multimodal ischemic penumbra in CT / CTP

Abstract

The purpose of the present invention is to provide a CT / CTP multimodal ischemic penumbra automatic annotation and visualization method and system, comprising the following steps: data acquisition and preprocessing; CT and CTP image registration; CTP functional parameter extraction; deep learning model training; model prediction and lesion annotation; lesion volume calculation; result visualization and output; and it is capable of supplementing the lack of functional information of ordinary CT images through advanced technical means.

Description

Technical Field

[0001] The invention belongs to the field of image processing, and in particular relates to a CT / CTP multi-modality ischemic penumbra automatic labeling and visualization method and system. Background Art

[0002] In the field of modern medical imaging, CT (computed tomography) and CTP (CT perfusion imaging) are important tools for diagnosing and evaluating cerebrovascular diseases. CT imaging, with its rapid imaging characteristics, plays a key role in the screening of acute stroke, but its limitation is that it can only provide anatomical information and cannot directly reflect the functional state of brain tissue. Therefore, CT imaging is not capable of judging the hemodynamic changes of brain tissue, especially identifying ischemic penumbra and infarct core areas. In this case, CTP becomes an important supplementary means to help doctors accurately identify low perfusion areas and irreversible damage areas of brain tissue by analyzing key blood flow parameters (such as Tmax and CBF).

[0003] Although CTP has significant clinical value, it faces many limitations in practical application, such as expensive equipment and contrast agent costs, and its complex imaging and post-processing procedures may also prolong the diagnosis time.

[0004] Therefore, there is an urgent need for an innovative method that can supplement the lack of functional information based on ordinary CT images. Summary of the invention

[0005] The purpose of the present invention is to provide a CT / CTP multimodal ischemic penumbra automatic annotation and visualization method and system, which can supplement the lack of functional information on the basis of ordinary CT images.

[0006] To achieve the above object, the present invention provides the following technical solution: a CT / CTP multi-modality ischemic penumbra automatic annotation and visualization method, comprising the following steps:

[0007] S1, data acquisition and preprocessing: collect paired CT images and CTP images to ensure that the two images cover the same anatomical area and the acquisition time interval is less than the time threshold;

[0008] S2. 3D point cloud conversion and feature extraction: After extracting the brain tissue parts from the CT images and CTP images, they are converted into 3D point cloud representations using threshold segmentation and surface reconstruction methods. The 3D distribution of key anatomical structures is obtained through a point cloud feature extraction algorithm based on geometric shape and texture information.

[0009] S3, deep learning registration based on 3D point cloud: the 3D point cloud representation is input into the deep neural network to achieve coarse and fine registration of CT images and CTP images; the registration process includes estimating the initial rigid transformation through the network based on key point matching, and then iteratively updating the nonlinear deformation field through the refinement network, and finally obtaining the optimal transformation parameters for aligning the CT image to the CTP image coordinate space;

[0010] S4, CTP functional parameter generation: extracting the regions corresponding to the Tmax and CBF functional parameters of the CTP image from the three-dimensional point cloud representation, and marking the mask of the low perfusion area and the mask of the infarct core area at the corresponding positions of the CT image in the three-dimensional point cloud representation;

[0011] S5. Deep neural network segmentation model training: In the unified coordinate space after registration, the 3D point cloud representation corresponding to the CT image containing the mask is used as a training sample. Combined with Dice loss or weighted cross entropy loss, a segmentation model that can predict the functional area mask from the CT image is trained.

[0012] S6. Prediction of lesion area and determination of ischemic penumbra: The CT image to be tested is input into the trained segmentation model, and the masks of the low perfusion area and the infarct core area are output; the mask of the ischemic penumbra area is obtained by the difference between the two.

[0013] Furthermore, it also includes S7, volume calculation and visualization output:

[0014] The number of voxels in the lesion mask is counted and the three-dimensional volume is calculated by combining the pixel spacing and slice thickness; the lesion mask is superimposed on the CT image or three-dimensional reconstruction view, and the low perfusion area, infarction core area and ischemic penumbra area are distinguished by different colors, and interactive correction and diagnosis report output are supported; among them, the lesion mask includes the mask of the low perfusion area, the mask of the infarction core area and the mask of the ischemic penumbra area.

[0015] Furthermore, in step S3, the deep learning registration based on the three-dimensional point cloud includes the following specific steps:

[0016] (a) Using the point cloud key point detection network, the feature points in the CT point cloud and CTP point cloud are detected and described;

[0017] (b) Construct an attention-based registration network, input key point coordinates and descriptors, and output initial rigid transformation parameters;

[0018] (c) The initial rigid transformation parameters are nonlinearly modified through the refinement network to obtain the final registration result.

[0019] Furthermore, the ischemic penumbra area Ω Penumbra Determined by the following set difference formula:

[0020] ,

[0021] Among them, Ω LowPerfusion is the low perfusion area, Ω InfarctCore The infarct core area.

[0022] Furthermore, in step S1, the mask normalization of the CT image is performed using the following formula:

[0023]

[0024]

[0025] in, and It is used to extract the CT value corresponding to the window width and window position of brain tissue. It is a mask for masking CT value and CTP value. is the gray value of the voxel in the CT image, is the gray value of the midpoint in the CT image.

[0026] Furthermore, in step S5, the Dice loss is calculated as follows:

[0027]

[0028] in, p i For the model i The predicted probability of training samples is g i For the i The Dice loss can enhance the recognition ability of small lesions and reduce the imbalance between foreground and background, which is used to improve the segmentation accuracy of the ischemic penumbra area.

[0029] Another technical solution of the present invention: a CT / CTP multi-modality ischemic penumbra automatic annotation and visualization system, comprising:

[0030] Data acquisition module: used to collect paired CT images and CTP images, ensuring that the two images cover the same anatomical area and the acquisition time interval is less than the time threshold;

[0031] Point cloud conversion and feature extraction module: used to extract brain tissue from CT images and CTP images, respectively, and then convert them into 3D point cloud representations using threshold segmentation and surface reconstruction methods; obtain the 3D distribution of key anatomical structures through a point cloud feature extraction algorithm based on geometric shape and texture information;

[0032] 3D deep learning registration module: used to input 3D point cloud representation into deep neural network to realize coarse and fine registration of CT images and CTP images; the registration process includes estimating the initial rigid transformation through the network based on key point matching, and then iteratively updating the nonlinear deformation field through the refinement network, and finally obtaining the optimal transformation parameters for aligning the CT image to the CTP image coordinate space;

[0033] CTP functional parameter generation module: used to extract the area corresponding to the Tmax and CBF functional parameters of the CTP image from the three-dimensional point cloud representation, and mark the mask of the low perfusion area and the mask of the infarct core area at the corresponding position of the CT image in the three-dimensional point cloud representation;

[0034] Segmentation model training module: It is used to use the 3D point cloud representation corresponding to the CT image containing the mask as a training sample in the unified coordinate space after registration, and train a segmentation model that can predict the functional area mask from the CT image in combination with Dice loss or weighted cross entropy loss;

[0035] Lesion area prediction and ischemic penumbra determination module: used to input the CT image to be tested into the trained segmentation model, output the masks of the low perfusion area and the infarct core area; and obtain the ischemic penumbra area mask through the difference between the two.

[0036] Furthermore, the 3D deep learning registration module obtains the optimal transformation parameter θ by minimizing the following loss function: ∗ :

[0037] ,

[0038] in, I CTP is the CTP image, I CT is the CT image, T θ (⋅) is the mapping function.

[0039] Furthermore, the visualization and interaction module has a slice-by-slice browsing function and allows doctors to manually edit the prediction results, and finally output a diagnostic report containing volume information of the low perfusion area, infarct core area and ischemic penumbra area.

[0040] Furthermore, it also includes a volume calculation module for outputting the three-dimensional volume data and segmentation visualization results of the low perfusion area, infarct core area and ischemic penumbra area in a standardized format to facilitate subsequent viewing and treatment plan evaluation.

[0041] Beneficial effects of the present invention:

[0042] 1. Multimodal fusion to improve the accuracy of lesion identification;

[0043] By deeply fusing the functional parameters (Tmax, CBF, etc.) in CT images and CTP images, it is possible to better identify low perfusion areas and infarct core areas; due to the introduction of multimodal data, the network can simultaneously utilize density information and functional perfusion information, which can more comprehensively and accurately reflect the brain tissue damage status than using CT or CTP alone.

[0044] 2. Driven by deep neural networks, reducing manual intervention;

[0045] The use of deep learning models to automatically extract features, align and segment CT and CTP can minimize the subjectivity and uncertainty caused by manual operations; in clinical practice, it can greatly reduce the workload of doctors in segmentation and measurement and improve diagnostic efficiency.

[0046] 3. Fine registration of 3D point clouds to reduce anatomical structure mismatch;

[0047] The registration strategy combining three-dimensional point cloud and deep neural network can make full use of the morphological characteristics of brain tissue to ensure the spatial alignment accuracy of multimodal images; reduce the misclassified areas caused by improper registration, thereby improving the positioning reliability of the ischemic penumbra.

[0048] 4. Automatically determine the ischemic penumbra and assist in thrombolysis time window decision-making;

[0049] The difference operation of the segmentation results can accurately identify the scope of potentially salvageable brain tissue, providing a quantitative reference for treatment options such as thrombolysis or mechanical thrombectomy; helping clinicians quickly determine whether the patient is suitable for thrombolysis and within what time window to intervene.

[0050] 5. Multi-scale network structure to improve the ability to identify small lesions;

[0051] Multi-scale convolution or feature fusion design is used to enhance the segmentation ability of small or blurred lesions and avoid averaging or loss of information in large scale. Combined with Dice loss or weighted cross entropy loss, the robustness to unbalanced data (large disparity in foreground / background ratio) is further improved.

[0052] 6. Visual superposition and volume analysis to facilitate clinical evaluation;

[0053] Based on the prediction results, the three-dimensional volume of the lesion and the corresponding thermal map or color annotation are automatically generated to assist doctors in observing the location and size of the lesion from different angles; by outputting the segmentation results in DICOM or PACS-compatible format, subsequent storage, transmission and multidisciplinary consultation are facilitated.

[0054] 7. Significantly improve the efficiency of diagnosis and decision-making;

[0055] Compared with traditional manual segmentation or simple threshold methods, deep learning models run fast and can generate results within minutes or even seconds. They play a key role in rescue decisions within the golden treatment time window of acute stroke, helping to improve prognosis and reduce complications.

[0056] 8. Meet individual clinical needs and be scalable;

[0057] The system or method can introduce more physiological indicators (such as CBV, MTT, etc.) or other modality imaging data according to actual conditions to achieve more detailed analysis of cerebrovascular lesions; in the later stage, the scanning equipment and imaging parameters of different hospitals can be adapted by updating the training set or changing the network structure, so that the model can be continuously iterated and upgraded. Through the above technical characteristics and effects, the present invention has significant advantages in reducing the workload of doctors, improving the accuracy of lesion identification, reducing missed diagnosis or misdiagnosis, and accelerating clinical decision-making, providing strong technical support for the early detection of stroke and optimization of treatment plans. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic diagram of the overall structure of the present invention. DETAILED DESCRIPTION

[0059] First, one of the main limitations of CTP imaging is its high dependence on equipment and resources. CTP equipment is expensive and can only be afforded by large medical institutions or comprehensive hospitals, while primary medical institutions and areas with limited resources often have difficulty obtaining these devices. This limitation in equipment costs has directly led to the low penetration rate of CTP in the medical system, making it impossible for many acute stroke patients to receive functional imaging support in a timely manner, thus missing the best time for treatment. In addition, the imaging process of CTP is also very complicated, requiring dynamic injection of contrast agents and real-time scanning, as well as high-performance computing equipment to complete subsequent image post-processing analysis. This high-tech imaging process places high demands on technical personnel and hardware conditions, further limiting its application in primary hospitals.

[0060] At the same time, the high cost of CTP examinations is not only reflected in the cost of equipment, but also in the use and maintenance costs of contrast agents, which makes the examination costs unaffordable for patients. Especially in cases where multiple follow-up examinations or long-term monitoring of the condition are required, the high cost of CTP may be prohibitive for patients. In addition, the use of contrast agents may also bring potential health risks, especially for patients with renal insufficiency, which may cause complications such as contrast-induced nephropathy (CIN); some patients may even be unable to undergo CTP examinations due to allergic reactions. These problems have exacerbated the difficulty of using CTP examinations, and clinicians may face technical limitations when facing complex cases.

[0061] More importantly, the diagnosis and treatment of acute stroke is a race against time, and the lack of CTP equipment often leads to delays in diagnosis and treatment. For hospitals that lack CTP imaging capabilities, patients need to be transferred to higher-level hospitals for examination. This process not only wastes precious treatment time, but also increases the risk of worsening of the patient's condition. Even for hospitals with CTP equipment, their complex imaging and post-processing processes may prolong the diagnosis time and further increase the degree of brain tissue damage in patients. Studies have shown that for every minute of delay, patients with ischemic stroke may lose approximately 1.9 million brain cells, and time delays have a significant negative impact on patient prognosis.

[0062] Although ordinary CT can quickly provide anatomical information, its lack of functional information makes it face major limitations in the diagnosis of cerebrovascular diseases. Doctors often need to rely on experience to make inferences when making diagnoses based solely on CT images, which may increase the risk of misdiagnosis or missed diagnosis. In the absence of CTP support, it is difficult for doctors to determine the specific range of brain tissue that can be salvaged, which may affect the accuracy of treatment decisions. In summary, although the current CTP technology has important value in functional imaging, its expensive equipment, complex operation, high cost, and health risks limit its scope of application, especially in medical environments with limited resources.

[0063] By introducing artificial intelligence technologies such as deep learning, the functional parameters of CTP can be inferred from ordinary CT images to achieve low-cost and high-efficiency diagnosis of cerebrovascular diseases. This method can not only reduce the complexity of medical equipment and operations, but also effectively reduce the economic burden on patients. At the same time, it adapts to the actual conditions of primary medical institutions and provides high-quality diagnosis and treatment support for more patients, thus solving the shortcomings of existing technologies.

[0064] The present invention proposes a CT / CTP multimodal ischemic penumbra automatic annotation and visualization method and system, aiming to solve the problem that ordinary CT cannot directly present brain perfusion information in clinical practice, and CTP equipment has limitations such as high cost and high threshold for use. To this end, the present invention fundamentally uses deep neural networks in conjunction with three-dimensional point cloud registration technology to infer lesion area segmentation results equivalent to CTP functional parameters based on conventional CT images, and automatically determines ischemic penumbras. The specific implementation method is as follows:

[0065] First, in the data collection and preprocessing stage, in order to minimize the difference caused by the progression of stroke, the time interval between CT and CTP scans should be strictly controlled within 30 minutes. Since the condition of acute stroke evolves very quickly over time, if the interval is too long, the anatomical structure provided by CT and the perfusion status provided by CTP may no longer match.

[0066] The present invention requires that after acquiring the CT image, grayscale normalization processing is performed on it to The CT value is mapped to the interval [0,1] in this way, thereby unifying the CT intensity distribution generated by different manufacturers and different scanning parameters, that is, eliminating the grayscale differences of different scanning devices and settings through grayscale normalization, improving data consistency and model convergence. Among them, and It is used to extract the CT value corresponding to the window width and window position of brain tissue. It is a mask that masks the CT value and CTP value and normalizes the unmasked part to ensure data consistency and improve the model convergence speed.

[0067] Subsequently, noise and artifacts are filtered out through algorithms such as Gaussian filtering or non-local means (NL-means). By removing artifacts and noise interference through noise, anatomical features can be highlighted; then, ROI cropping is performed using threshold or morphological operations to remove irrelevant areas such as the skull and scalp, and finally only the brain substance is retained, focusing on the brain tissue area, reducing the data size, and improving the model's computational efficiency and prediction accuracy. Such preprocessing can not only reduce the data dimension and focus on the key parts of the lesion, but also prevent subsequent 3D reconstruction and network training from being disturbed by noise or background.

[0068] Next, in the three-dimensional point cloud conversion and feature extraction, the present invention performs threshold segmentation on the pre-processed CT and CTP images, and then uses a surface reconstruction algorithm (Marching Cubes) to convert the voxel form into a three-dimensional grid, and then extracts the vertex coordinates of the grid as three-dimensional point cloud data. The point cloud form is chosen because compared with traditional 2D plane processing, three-dimensional point clouds can more flexibly handle the morphological changes of brain tissue during rotation, translation, and even local deformation. In order to improve the accuracy of subsequent registration, the present invention extracts geometric and texture features from the point cloud, such as descriptors such as SIFT-3D, FPFH, or Harris-3D, so that the registration network can rely on curvature and gradient information to automatically identify and match feature-rich key points.

[0069] In the CTP function parameter generation part, the present invention uses the deconvolution method of the time series contrast agent concentration curve C(x,t) to calculate key indicators such as Tmax (blood flow peak time) and CBF (cerebral blood flow). Specifically, if Tmax(x)>6 seconds, it can be determined as a low perfusion area Ω LowPerfusion If CBF(x)≤30%, it is determined as the infarct core areaΩ InfarctCore. Here, x usually represents a specific position (pixel or voxel coordinates) in an image or three-dimensional space. In other words, C(x,t) refers to the contrast agent concentration curve that changes with time t at the "position" x; accordingly, Tmax(x) and CBF(x) are also functional parameter values ​​calculated at the position x. These thresholds have been widely recognized in large-scale international clinical studies and literature, and can effectively distinguish between reversible and irreversible brain tissue damage in functional images. The idea of ​​the present invention is to combine the CTP functional zoning mask with the CT image to train the deep learning model, so that the model can "simulate" the lesion range corresponding to the CTP when only the CT image is input.

[0070] In order to achieve high-precision alignment between CT and CTP, the present invention introduces a two-stage registration strategy in the deep learning registration process based on 3D point cloud. In the first stage, the attention mechanism registration network is used to estimate the rigid transformation parameters (rotation matrix R and translation vector t) based on the previously extracted point cloud key points and descriptors. Initially align the CT point cloud and the CTP point cloud; in the second stage, a refinement network is introduced to iteratively update the local nonlinear deformation field Φ so that it can handle the slight differences caused by local physiological deformation of brain tissue or scanning posture. By minimizing different forms of L functions based on mutual information (MI), mean square error (MSE) or adversarial loss, CT and CTP can eventually achieve a more precise matching effect in the same coordinate system.

[0071] In the training of the deep neural network segmentation model, the present invention regards the registered ordinary CT images as input, uses the low perfusion area and infarct core area masks generated by CTP functional parameters as labels, and uses the structure of a multi-scale convolutional network (such as 3D U-Net or Attention U-Net) to learn the mapping relationship between CT texture and functional partitioning. Since the foreground (lesion area) and background (normal brain tissue) are often seriously unbalanced in actual clinical practice, the present invention uses Dice loss or weighted cross entropy loss to improve the sensitivity to small lesions. Among them, Dice loss It can not only intuitively reflect the degree of overlap between the segmentation results and the true annotations, but also guide the network to correctly focus on the lesions when there is very little data prospect.

[0072] Among them, p i It represents the "prediction value" of the prediction result at pixel or voxel i. If it is binary segmentation, it is usually the foreground probability output by the segmentation network (in the range [0,1]). If it is multi-category segmentation, it can refer to the probability of the corresponding category, or the final binary mask value (0 / 1). When the entire image has many pixels / voxels, i represents the i-th pixel / voxel.

[0073] g iRepresents the "true value" of the true label at pixel / voxel i; usually a binary value (0 or 1): 1 indicates that the current pixel / voxel belongs to the foreground or lesion area, and 0 indicates the background or normal area; it can also correspond to the true value mask of a single category in multi-category segmentation, or it may be a segmentation probability of [0,1] in the soft label scenario.

[0074] ∑ i p i g i The sum (or integration) of all pixels or voxels is taken to measure the overlap between the prediction and the true label in the foreground area.

[0075] If the overlap is greater, then ∑ i p i g i The larger the value, the higher the segmentation accuracy.

[0076] ∑ i p i The predicted values ​​of all pixels / voxels are summed up to represent the total "amount" of foreground area predicted by the network.

[0077] If the predicted foreground area is too large or too small, it will affect how well it matches the true area.

[0078] ∑ i g i The true labels of all pixels / voxels are summed to represent the total “amount” of true foreground area.

[0079] If the true lesion is small in the image / volume, the segmentation network needs to be able to handle the foreground / background imbalance problem.

[0080] i represents the index of all pixels / voxels traversed in the image or volume;

[0081] In 2D segmentation, i traverses every pixel in the image;

[0082] In 3D segmentation, i traverses the voxels;

[0083] This pass can be understood as calculating at all positions in one (or more) images / volumes and then summing or averaging them.

[0084] The traditional definition of the Dice coefficient multiplies the numerator by 2 to ensure that when the predicted and true masks overlap perfectly, the Dice coefficient is 1 (corresponding to a loss of 0). This is consistent with the original definition of the Dice coefficient in the set similarity metric.

[0085] Intuitive explanation of Dice loss:

[0086] The loss is essentially: , when the segmentation is more similar to the true label, ∑ i p i g i The larger the denominator ∑ i p i +∑ i g i The larger the ratio to the numerator, the closer the Dice coefficient is to 1 and the loss is closer to zero.

[0087] Compared with the conventional cross entropy loss, Dice loss tends to better reflect the degree of overlap of the foreground area when the foreground / background distribution is very unbalanced (such as when the lesion is very small).

[0088] In medical image segmentation, p i Usually the probability score output by the network (or the binary segmentation result), g i is the true annotation value (0 / 1), and the weighted statistics of the two on all pixels or voxels can measure the overlap of the foreground area. Dice loss promotes the model to accurately identify the lesion area by maximizing overlap (or minimizing 1-overlap).

[0089] After the training is completed, the prediction of the lesion area and the determination of the ischemic penumbra become the core links in the clinical application of the present invention: doctors only need to collect ordinary CT scans for new cases, input them into the trained model after preprocessing, and then they can get the predicted low perfusion area Ω ∗ LowPerfusion Infarct core area ∗ InfarctCore .pass , the ischemic penumbra that has not yet suffered irreversible damage can be automatically calculated. Since the volume of the reversible part of the disease is crucial to the therapeutic value when making decisions on thrombolysis or thrombectomy, the automated determination provided by the present invention at this moment can greatly save time.

[0090] Finally, volume calculation and visualization output further meet the quantitative evaluation needs of clinicians. voxel , combining pixel spacing Δx, Δy and slice thickness Δz to calculate the three-dimensional volume V=N voxel ×Δx×Δy×Δz, and the segmentation results are superimposed on the CT image or 3D reconstruction view in different colors or pseudo colors. Doctors can browse the lesion area slice by slice, and can also make manual revisions; in addition, the present invention also supports saving and outputting reports in DICOM or PACS compatible formats for subsequent use in multidisciplinary consultations or hospital information system archiving.

[0091] To prove the effectiveness and robustness of the present invention, the examples section gives the results of large-scale data testing, cross-device data verification, and emergency real-time application scenarios in tertiary hospitals. In large-scale tests, by comparing the gold standard of manual segmentation, the Dice coefficients of the low perfusion area and the infarct core area can reach 0.88 and 0.85 respectively, and the error of the ischemic penumbra volume calculation is less than 10%, which is significantly better than the traditional threshold method based only on single-modality CT; in cross-device scenarios, through unified grayscale normalization and deep learning registration network, the present invention still shows good cross-hospital adaptability on different CT / CTP models in multiple hospitals; and under emergency conditions, when some patients cannot undergo CTP scanning or have contraindications to contrast agents such as renal insufficiency, the present invention can directly use ordinary CT to complete functional segmentation prediction, and quickly generate visualization results and three-dimensional volume information in 2 to 3 minutes, which helps to improve the efficiency of stroke treatment.

[0092] In summary, the present invention realizes the automatic annotation and visualization of low perfusion areas, infarct core areas and ischemic penumbras by constructing a three-dimensional point cloud registration and deep learning segmentation model between conventional CT images and CTP functional zoning, greatly reducing the dependence on expensive CTP equipment, and has significant advantages in clinical decision-making, cross-device application and rapid diagnosis. Any equivalent replacement or simple modification of the method flow or network structure within the core concept of the present invention shall be deemed to fall within the protection scope of the present invention.

Claims

1. A method for automatic annotation and visualization of multimodal ischemic penumbra in CT / CTP, characterized in that: The following steps are involved: S1, data acquisition and preprocessing: collect paired CT images and CTP images to ensure that the two images cover the same anatomical area and the acquisition time interval is less than the time threshold; S2. 3D point cloud conversion and feature extraction: After extracting the brain tissue parts from the CT images and CTP images, they are converted into 3D point cloud representations using threshold segmentation and surface reconstruction methods. The 3D distribution of key anatomical structures is obtained through a point cloud feature extraction algorithm based on geometric shape and texture information. S3, deep learning registration based on 3D point cloud: the 3D point cloud representation is input into the deep neural network to achieve coarse and fine registration of CT images and CTP images; the registration process includes estimating the initial rigid transformation through the network based on key point matching, and then iteratively updating the nonlinear deformation field through the refinement network, and finally obtaining the optimal transformation parameters for aligning the CT image to the CTP image coordinate space; S4, CTP functional parameter generation: extracting the regions corresponding to the Tmax and CBF functional parameters of the CTP image from the three-dimensional point cloud representation, and marking the mask of the low perfusion area and the mask of the infarct core area at the corresponding positions of the CT image in the three-dimensional point cloud representation; S5. Deep neural network segmentation model training: In the unified coordinate space after registration, the 3D point cloud representation corresponding to the CT image containing the mask is used as a training sample. Combined with Dice loss or weighted cross entropy loss, a segmentation model that can predict the functional area mask from the CT image is trained. S6. Prediction of lesion area and determination of ischemic penumbra: The CT image to be tested is input into the trained segmentation model, and the masks of the low perfusion area and the infarct core area are output; the mask of the ischemic penumbra area is obtained by the difference between the two.

2. The method for automatic annotation and visualization of multimodal ischemic penumbra in CT / CTP according to claim 1, further comprising S7, volume calculation and visualization output: Count the number of voxels in the lesion mask and calculate the three-dimensional volume by combining the pixel spacing and slice thickness; superimpose the lesion mask on the CT image or three-dimensional reconstruction view, distinguish the low perfusion area, infarct core area and ischemic penumbra area with different colors, and support interactive correction and diagnosis report output; Among them, The lesion mask includes the mask of the low perfusion area, the mask of the infarct core area and the mask of the ischemic penumbra area.

3. The method for automatic annotation and visualization of multimodal ischemic penumbra in CT / CTP according to claim 2, characterized in that: In step S3, the deep learning registration based on three-dimensional point cloud includes the following specific steps: (a) Using the point cloud key point detection network, the feature points in the CT point cloud and CTP point cloud are detected and described; (b) Construct an attention-based registration network, input key point coordinates and descriptors, and output initial rigid transformation parameters; (c) The initial rigid transformation parameters are nonlinearly modified through the refinement network to obtain the final registration result.

4. The method for automatic annotation and visualization of multimodal ischemic penumbra in CT / CTP according to claim 2, characterized in that: The ischemic penumbra area Ω Penumbra Determined by the following set difference formula: ； Among them, Ω LowPerfusion is the low perfusion area, Ω InfarctCore The infarct core area.

5. The method for automatic annotation and visualization of multimodal ischemic penumbra in CT / CTP according to claim 3 or 4, characterized in that: In step S1, the mask normalization of the CT image is performed using the following formula: ； ； in, and It is used to extract the CT value corresponding to the window width and window position of brain tissue. It is a mask for masking CT value and CTP value. is the gray value of the voxel in the CT image, is the gray value of the midpoint in the CT image.

6. The method for automatic annotation and visualization of multimodal ischemic penumbra in CT / CTP according to claim 1, characterized in that: The Dice loss in step S5 is calculated as follows: ； in, p i For the model i The predicted probability of training samples is g i For the i The Dice loss can enhance the recognition ability of small lesions and reduce the imbalance between foreground and background, which is used to improve the segmentation accuracy of the ischemic penumbra area.

7. A CT / CTP multimodal ischemic penumbra automatic annotation and visualization system, characterized in that: include: Data acquisition module: used to collect paired CT images and CTP images, ensuring that the two images cover the same anatomical area and the acquisition time interval is less than the time threshold; Point cloud conversion and feature extraction module: used to extract brain tissue from CT images and CTP images, respectively, and then convert them into 3D point cloud representations using threshold segmentation and surface reconstruction methods; obtain the 3D distribution of key anatomical structures through a point cloud feature extraction algorithm based on geometric shape and texture information; 3D deep learning registration module: used to input 3D point cloud representation into deep neural network to realize coarse and fine registration of CT images and CTP images; the registration process includes estimating the initial rigid transformation through the network based on key point matching, and then iteratively updating the nonlinear deformation field through the refinement network, and finally obtaining the optimal transformation parameters for aligning the CT image to the CTP image coordinate space; CTP functional parameter generation module: used to extract the area corresponding to the Tmax and CBF functional parameters of the CTP image from the three-dimensional point cloud representation, and mark the mask of the low perfusion area and the mask of the infarct core area at the corresponding position of the CT image in the three-dimensional point cloud representation; Segmentation model training module: It is used to use the 3D point cloud representation corresponding to the CT image containing the mask as a training sample in the unified coordinate space after registration, and train a segmentation model that can predict the functional area mask from the CT image in combination with Dice loss or weighted cross entropy loss; Lesion area prediction and ischemic penumbra determination module: used to input the CT image to be tested into the trained segmentation model, output the masks of the low perfusion area and the infarct core area; and obtain the ischemic penumbra area mask through the difference between the two.

8. The CT / CTP multimodal ischemic penumbra automatic annotation and visualization system according to claim 7, characterized in that: The 3D deep learning registration module obtains the optimal transformation parameter θ by minimizing the following loss function: ∗ : , in, I CTP is the CTP image, I CT is the CT image, T θ (⋅) is the mapping function.

9. The CT / CTP multimodal ischemic penumbra automatic annotation and visualization system according to claim 6 or 7, characterized in that: The visualization and interaction module has a slice-by-slice browsing function and allows doctors to manually edit the prediction results, and finally outputs a diagnostic report containing volume information of the low perfusion area, infarct core area and ischemic penumbra area.

10. The CT / CTP multimodal ischemic penumbra automatic annotation and visualization system according to claim 6 or 7, characterized in that: It also includes a volume calculation module, which is used to output the 3D volume data and segmentation visualization results of the low perfusion area, infarct core area and ischemic penumbra area in a standardized format for subsequent review and treatment plan evaluation.

Images