Detection device for intracranial aneurysm rupture risk based on multi-dimensional feature fusion

Abstract

The present invention discloses a detection device for the risk of intracranial aneurysm rupture based on multidimensional feature fusion, comprising: an automatic segmentation module, used to obtain a segmented aneurysm image by introducing a preset segmentation model of a lesion-aware attention mechanism and a multi-level semantic supervision mechanism; a three-dimensional reconstruction module, used to establish a three-dimensional model of the aneurysm using the segmented aneurysm image as input to obtain a three-dimensional model of the aneurysm; a multidimensional feature extraction module, based on angiography images, segmented images and a three-dimensional model of an aneurysm of an object to be detected as input, extracts multidimensional features related to the risk of aneurysm rupture; a risk detection module, used to eliminate irrelevant features in a multidimensional feature set according to a preset optimal feature set, and then input the remaining features of the object to be detected into a preset detection model, and finally obtain the rupture risk of the aneurysm of the object to be detected. The present invention combines the aneurysm segmentation and recognition task with the rupture risk detection task to reduce the interference of irrelevant information.

Description

Technical Field

[0001] The present invention relates to the field of intracranial aneurysm detection, and in particular to a device for detecting the risk of intracranial aneurysm rupture based on multi-dimensional feature fusion. Background Art

[0002] Intracranial aneurysms (IAs) are saccular malformations formed by abnormal expansion of the local lumen due to destruction of the intracranial arterial wall or local congenital defects, with an incidence of about 3%-5% in the population. Once an aneurysm ruptures, it will cause subarachnoid hemorrhage, accompanied by a high mortality rate and permanent disability rate. According to surveys, 80-90% of spontaneous subarachnoid hemorrhages are caused by ruptured cerebral aneurysms. With the advancement of vascular neuroimaging technology, more and more unruptured aneurysms are detected, but because doctors need to carefully weigh the serious consequences of aneurysm rupture, the risks during treatment, and subsequent complications, aneurysm treatment decisions are still an important challenge in clinical practice. Therefore, it is of great clinical significance to detect aneurysms as early as possible and predict the risk of aneurysm rupture.

[0003] The imaging techniques commonly used in clinical aneurysm examinations include digital subtraction angiography (DSA), computed tomography angiography (CTA), and magnetic resonance angiography (MRA). At present, the risk of aneurysm rupture is mainly determined by the doctor's experience, combined with the aneurysm image and patient-specific clinical factors (such as hypertension, smoking, gender, age, etc.). However, this evaluation method is highly subjective, lacks consistency among experts, and due to the many factors that affect rupture, manual comprehensive judgment is by no means easy. Therefore, it is necessary to use artificial intelligence to develop a computer-aided diagnosis system that can detect the risk of aneurysm rupture, so as to assist doctors in diagnosis and guide subsequent treatment plans.

[0004] When evaluating the rupture risk of aneurysms, traditional methods generally use calculations or manual measurements of aneurysm morphological features as the basis for judging aneurysm rupture, which fails to fully utilize the rich information in angiographic images and has the problem of inaccurate evaluation. Radiomics technology converts image data into high-dimensional features according to a pre-set calculation formula to quantify the corresponding phenotypic characteristics, and shows broad application prospects in medical fields such as tumor staging and cancer diagnosis. Some studies have pointed out that the texture features contained in radiomics can potentially reflect the blood flow of aneurysms. In recent years, deep learning technology has shown great advantages in handling classification, segmentation and detection tasks in medical imaging. This technology can extract features from pre-trained convolutional neural networks, allowing modeling of high-level abstract feature representations in the data, and automatically learning complex information patterns and structural features in image data that are invisible to the naked eye. After comprehensively utilizing the features in angiographic images, combined with the clinical information of the object to be detected, an aneurysm rupture risk detection model is constructed through machine learning technology. At present, few studies have combined deep learning technology, radiomics technology and machine learning for the detection of aneurysm rupture risk. Summary of the invention

[0005] The present invention provides a detection device for the risk of intracranial aneurysm rupture based on multi-dimensional feature fusion. The present invention realizes an integrated process of aneurysm auxiliary diagnosis by combining the aneurysm segmentation and recognition task with the rupture risk detection task, which can reduce the involvement of human factors and make full use of angiography images to extract features related to rupture, so that the rupture risk detection result is more objective and accurate. See the following description for details:

[0006] A device for detecting the risk of intracranial aneurysm rupture based on multi-dimensional feature fusion, the device comprising:

[0007] An automatic segmentation module, used to obtain a segmented aneurysm image by introducing a preset segmentation model with a lesion-aware attention mechanism and a multi-level semantic supervision mechanism;

[0008] A three-dimensional reconstruction module is used to establish a three-dimensional model of the aneurysm using the segmented aneurysm image as input to obtain a three-dimensional model of the aneurysm;

[0009] The multidimensional feature extraction module extracts multidimensional features related to the risk of aneurysm rupture based on the angiography image, segmentation image and aneurysm 3D model of the object to be detected, including: geometric morphological features, imaging omics features, deep learning features and patient-specific clinical factors;

[0010] The risk detection module is used to eliminate irrelevant features in the multidimensional feature set according to the preset optimal feature set, and then input the remaining features of the object to be detected into the preset detection model, and finally obtain the rupture risk of the aneurysm of the object to be detected.

[0011] Among them, the preset segmentation model is a 3D lesion-aware attention U-Net improved based on U-Net, which is composed of an encoder, a decoder, multi-level skip connections, a lesion-aware attention mechanism and a multi-level semantic supervision mechanism.

[0012] Furthermore, the lesion-aware attention mechanism is defined as follows:

[0013]

[0014] Attention value=Boundary+Prediction

[0015]

[0016] Among them, Prediction represents the prediction map output by the preset segmentation model at each stage of upsampling, Boundary represents the prediction map calculated by the prediction map Prediction that only focuses on the target boundary contour area; Attentionvalue represents the attention value of the target area and the target boundary contour pixels; represents the element-by-element matrix multiplication operation, represents the element-by-element addition operation of the matrix; E represents the feature map of the encoder [E1, E2, E3, E4], E att Represents the feature map of the encoder after calculation by the attention mechanism.

[0017] The multidimensional features include: geometric morphological features extracted from the three-dimensional model of the aneurysm; imaging features extracted from the angiographic image and the segmentation mask of the aneurysm; deep learning features extracted from the angiographic image and the segmentation mask of the aneurysm; and patient-specific clinical factors.

[0018] Furthermore, the geometric morphological features include: manually measured morphological features and calculated curvature features,

[0019] The curvature characteristics include: Gaussian curvature, minimum principal curvature, maximum principal curvature, and average curvature of the aneurysm;

[0020] The maximum value, minimum value, mean value and standard deviation of the above curvature are calculated as curvature features respectively, and the curvature features are extracted using the pymeshlab toolkit.

[0021] Wherein, the deep learning features are extracted using a three-dimensional convolutional neural network.

[0022] The classification network 3D EfficientNet-B0 was selected as the feature extractor. The angiography image and the corresponding aneurysm mask image were used as the input of the convolutional neural network. Three different deep learning features were calculated through the preset classification model.

[0023] The training data of the preset classification model are angiography images of subjects with aneurysms and 3D segmentation mask images of aneurysms annotated by experienced medical experts, and the classification network is trained using the aneurysm rupture status as a label.

[0024] Furthermore, the three different deep learning features calculated by the preset classification model are:

[0025] 1) No-sigmoid type features: The feature map is passed through each layer of the network structure of EfficientNet-B0 until the final fully connected layer, and the output obtained is used as the final No-sigmoid type deep learning feature, expressed as:

[0026] f no-sigmoid =model(x), x = img or mask

[0027] Where model is the pre-trained 3D EfficientNet-B0 model, img is the angiography image, and mask is the corresponding mask image;

[0028] 2) Sigmoid type features: The No-sigmoid type features are passed through a sigmoid function to obtain the final Sigmoid type deep learning features, expressed as:

[0029] f sigmoid =σ(model(x)),x=img or mask

[0030] 3) Binarized features: The above Sigmoid features are binarized to obtain the final binary deep learning features, which are expressed as:

[0031] f binarization =B(σ(model(x))),x=img or mask

[0032] The function B in the formula is a binarization operation.

[0033] The beneficial effects of the technical solution provided by the present invention are:

[0034] (1) The present invention first performs automatic aneurysm segmentation on intracranial angiography images, and strengthens the aneurysm area and boundary contour information through the proposed lesion-aware attention mechanism, so that the network pays more attention to the target part and reduces the interference of irrelevant information; constructs three-dimensional modeling data based on the segmented aneurysm image, and then extracts multi-dimensional features based on the segmented image, the original angiography image and the three-dimensional modeling data, and constructs a classification model by fusing multi-source image features, especially the curvature features mentioned in the present invention and the deep learning features based on the pre-classification network, as well as the patient's clinical characteristics, so as to detect the state of aneurysm rupture, promote timely management of patients, and provide some targeted guidance for treatment decisions;

[0035] (2) The present invention uses machine learning technology to obtain a key feature set through feature extraction and feature screening steps, and establishes a classification model to detect the risk of aneurysm rupture, thereby solving the problem that clinicians are highly subjective in their assessment due to the large number of factors that affect aneurysm rupture in the clinic.

[0036] (3) The present invention makes full use of the rich information of angiographic images, uses radiomics technology and deep learning technology to extract features at all scales of the image, and combines it with the shape parameters and curvature characteristics of the aneurysm three-dimensional model and the patient's relevant clinical characteristics, greatly enriching the feature representation of aneurysm rupture risk detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic diagram of the working process of a device for detecting the risk of intracranial aneurysm rupture based on multi-dimensional feature fusion;

[0038] Figure 2 This is the architecture diagram of the 3D LAU-Net (3D Lesion-Aware Attention U-Net) segmentation model;

[0039] Figure 3 This is the implementation schematic diagram of the lesion-aware attention module;

[0040] Figure 4 A schematic diagram of a multi-dimensional feature set extracted for aneurysm rupture risk detection involved in the device of the present invention; DETAILED DESCRIPTION

[0041] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention are described in further detail below.

[0042] Example 1

[0043] In view of the above problems, the present invention proposes a device for detecting the risk of intracranial aneurysm rupture based on multi-dimensional feature fusion by combining machine learning technology, imaging omics technology and deep learning technology. Figure 1The device includes: a data acquisition module, an automatic segmentation module, a three-dimensional reconstruction module, a multi-dimensional feature extraction module and a risk detection module.

[0044] The data acquisition module is used to obtain the intracranial vascular imaging data and corresponding clinical information (for example, gender, age, history of hypertension, smoking history, family history, etc.) of the subject to be detected.

[0045] The automatic segmentation module is used to obtain the segmented aneurysm image through a preset segmentation model.

[0046] Among them, the preset segmentation model adopts the improved 3D lesion-aware attention U-Net network based on U-Net (known to those skilled in the art), which fully focuses on the target area and target boundary through the attention mechanism to achieve accurate segmentation of aneurysms. In addition, a multi-level semantic supervision mechanism is introduced in the training process of the preset segmentation model, through which more global semantic representations are learned, the segmentation accuracy is improved, and the segmentation results are refined.

[0047] The three-dimensional reconstruction module is used to use the segmented aneurysm image as input to establish a three-dimensional model of the aneurysm, thereby obtaining a three-dimensional model of the aneurysm.

[0048] The multidimensional feature extraction module is used to extract multidimensional features related to the risk of aneurysm rupture based on the angiography image, segmentation image and aneurysm three-dimensional model of the object to be detected as input, including: geometric morphological features, imaging genomics features, and deep learning features, and combine the above features with the patient's clinical characteristics to form a multidimensional feature set.

[0049] After obtaining the segmented aneurysm image, the embodiment of the present invention combines deep learning technology with radiomics technology to extract feature representations related to aneurysm rupture from the medical image, further improving the performance of aneurysm rupture risk detection. Specifically, this multidimensional feature representation related to aneurysm rupture risk factors includes: radiomics features, geometric morphology features, deep learning features, and patient-specific clinical factors.

[0050] The multidimensional feature extraction module comprehensively considers the above multidimensional feature sets to help make the prediction results of aneurysm rupture risk more accurate.

[0051] The risk detection module is used to eliminate irrelevant features in the multidimensional feature set according to the preset optimal feature set, and then input the remaining features of the object to be detected into the preset detection model, and finally obtain the rupture risk of the aneurysm of the object to be detected.

[0052] In specific implementation, the rupture risk detection of aneurysms is usually based on the positioning and segmentation of aneurysms. Therefore, the embodiment of the present invention combines the automatic segmentation task of aneurysms with the rupture risk detection task. First, the automatic segmentation algorithm (i.e., the automatic segmentation module) is used to obtain the location and structure of the aneurysm, so as to facilitate the subsequent extraction of relevant image features based on the results of the segmentation task to detect the rupture risk.

[0053] In summary, compared with other inventions, the embodiment of the present invention proposes to use a pre-classified convolutional neural network as a feature extractor, and calculates three different deep learning features through a preset classification model, enriching the types of features for detecting the risk of aneurysm rupture. In addition, the embodiment of the present invention additionally incorporates the curvature feature of the aneurysm as a supplementary morphological feature into the aneurysm rupture risk detection task, enriching the morphological features of the traditional detection of aneurysm rupture risk. Comprehensive consideration of multiple features can enrich the feature representation of medical images and further improve the prediction performance of aneurysm rupture risk.

[0054] Example 2

[0055] Combine the following Figure 1-Figure 4 , the scheme in Example 1 is further introduced, as described below:

[0056] 1. Data Acquisition Module

[0057] The data involved in the embodiment of the present invention is divided into two parts: the patient's intracranial vascular imaging data and the corresponding clinical data. The intracranial imaging data is a single type of 3D angiography image, including but not limited to any one of DSA (digital subtraction angiography) data, CTA (computed tomography angiography) data and MRA (magnetic resonance angiography) data. The clinical data includes the patient's clinical information such as gender, age, history of hypertension, smoking history, family history, etc. These clinical characteristics have a certain correlation with the risk of aneurysm rupture.

[0058] 2. Automatic Segmentation Module

[0059] The automatic segmentation module first preprocesses the aneurysm medical image of the object to be detected, and then inputs the preprocessed image into a preset segmentation model to obtain a segmentation mask image of the aneurysm. The preset segmentation model is trained with a 3D intracranial angiography image of the object with aneurysm and a corresponding 3D segmentation mask image (Ground Truth) of the aneurysm annotated by a medical expert.

[0060] Among them, see Figure 2The preset segmentation model is the 3D Lesion-aware Attention U-Net (LAU-Net) based on the improved U-Net. The 3D LAU-Net model consists of an encoder block, a decoder block, multi-level skip connections, a lesion-aware attention mechanism (LAB) and a multi-level semantic supervision mechanism. That is, the lesion-aware attention mechanism and the multi-level semantic supervision mechanism are introduced on the basis of the original 3D U-Net, making the network more suitable for the segmentation of aneurysm images.

[0061] 1. The preprocessing steps include: performing data conversion, denoising, skull removal, resampling and grayscale normalization operations on intracranial image data to improve image quality and facilitate the subsequent model training and verification process. Among them, data conversion refers to converting the original DICOM (Digital Imaging Communication Protocol) file collected from the medical image imaging machine into a NIFTI (Neuroimaging Information Technology Initiative) format file for subsequent data processing. Denoising refers to removing the noise introduced during the formation and display of medical images by the imaging system. Skull removal refers to removing the interference of skull structure in angiography data. Resampling refers to adjusting all voxels of different sizes in 3D intracranial images to the same size, reducing the inconsistency of different images, and facilitating the model to extract common features.

[0062] 2. 3D LAU-Net segmentation algorithm

[0063] 3D medical image segmentation aims to predict specific organs or lesions, and plays an important role in evaluating the progression of the disease and formulating treatment plans. The preset segmentation model 3D LAU-Net is trained in a supervised learning mode. The training data is the intracranial angiography images of subjects with aneurysms and the 3D segmentation mask images (Ground Truth) of aneurysms annotated by experienced medical experts. When the intracranial angiography image of the subject to be detected is input into the preset segmentation model, the segmented aneurysm image of the subject to be detected can be directly obtained.

[0064] Among them, the 3D LAU-Net model introduces the lesion-aware attention block (LAB) and multi-level semantic supervision mechanism based on the original 3D U-Net (composed of an encoder, a decoder, and multi-level skip connections).

[0065] The following is a detailed description of the 3D LAU-Net model structure:

[0066] 1) Lesion perception attention mechanism

[0067] The encoder uses convolution and pooling operations to extract features of aneurysm images at different scales for the feature extractor. The entire downsampling process represented by the encoder consists of five encoder blocks (Encoder Block), and the output after each encoder block is defined as [E1, E2, E3, E4, E5].

[0068] The multi-level skip connection of the original U-Net model supplements the decoder with more shallow details by passing feature maps from each encoder block. However, this simple feature concatenation is not enough to achieve accurate segmentation results because the encoder block contains a lot of information that is irrelevant to the target area. By using the lesion-aware attention mechanism to strengthen the aneurysm area and boundary contour information, the network pays more attention to the target part and reduces the interference of irrelevant information.

[0069] The lesion-aware attention mechanism is defined as follows:

[0070]

[0071] Attention value=Boundary+Prediction (2)

[0072]

[0073] Where Prediction represents the prediction map output by the preset segmentation model at each stage of upsampling, and Boundary represents the prediction map calculated by the prediction map Prediction that only focuses on the target boundary contour area. Attentionvalue represents the attention value of the target area and the target boundary contour pixels. represents the element-by-element matrix multiplication operation, represents the element-by-element addition operation of the matrix, E represents the feature map of the encoder [E1, E2, E3, E4], E att represents the feature map of the encoder after the attention mechanism is calculated. The above attention mechanism is applied to the feature maps [E1, E2, E3, E4] of each stage of the encoder to obtain [E att1 ,E att2 ,E att3 ,E att4 ], which can avoid the interference of irrelevant information and make the network pay more attention to the target area. Therefore, in the multi-level skip connection operation in the 3D LAU-Net model, the feature map of the encoder enhanced by the attention mechanism is transmitted to the decoder.

[0074] 2) Multi-level semantic supervision mechanism

[0075] In order to solve the problem of gradient vanishing during network training and speed up the convergence of the network, a multi-level semantic supervision mechanism is introduced into the preset segmentation model. The loss function calculated by accumulating the prediction graph output by the decoder at each upsampling stage and the Ground Truth is used to refine the segmentation result and achieve accurate segmentation.

[0076] 3. 3D reconstruction module

[0077] The segmentation mask image of the aneurysm obtained above is used as input to establish a three-dimensional model of the aneurysm. The segmentation mask image obtained above is in NIFTI format, and 3D Slicer software (https: / / discourse.slicer.org / , which is an open source software for medical image processing and visualization) is used to convert it into STL (stereolithography) format to obtain a segmented three-dimensional model of the aneurysm, which is convenient for subsequent steps to extract the geometric morphological features of the aneurysm according to the three-dimensional model.

[0078] 4. Multidimensional feature extraction module

[0079] After the angiography image, segmentation image and aneurysm three-dimensional model data of the object to be detected are input into the module, multi-dimensional feature extraction is performed to obtain a multi-dimensional feature set related to the risk of aneurysm rupture.

[0080] The multidimensional feature set consists of four types of features: ① geometric morphological features extracted from the 3D model of the aneurysm. ② radiomics features extracted from the angiographic images and the segmentation mask of the aneurysm. ③ deep learning features extracted from the angiographic images and the segmentation mask of the aneurysm. ④ patient-specific clinical factors.

[0081] The above four types of characteristics are all risk factors related to aneurysm rupture. Therefore, comprehensive consideration of the above characteristics can make the prediction results of aneurysm rupture risk more accurate and reliable. The following describes the four types of characteristics in detail.

[0082] 4.1 Geometric morphological features based on 3D models

[0083] Based on the three-dimensional model of the aneurysm, geometric morphological features significantly associated with aneurysm rupture are extracted, including manually measured morphological features and calculated curvature features.

[0084] 4.1.1 Based on the three-dimensional model of the aneurysm, the width, length, maximum diameter and volume of the intracranial aneurysm are measured.

[0085] 4.1.2 Calculate the curvature of the three-dimensional model of the aneurysm, including the Gaussian curvature, principal curvature (including the minimum principal curvature and the maximum principal curvature), and average curvature of the aneurysm.

[0086] The maximum value, minimum value, mean value and standard deviation of the above curvature are calculated as curvature features. The extraction of this part of curvature features can be implemented by programming using the pymeshlab toolkit (https: / / pymeshlab.readthedocs.io / en / latest / about.html#).

[0087] 4.2 Imaging features based on angiography

[0088] The goal of radiomics is to extract high-throughput quantitative features from medical images, with the ultimate goal of generating imaging biomarkers as decision support tools for clinical practice. The Pyradiomics toolkit (https: / / pyradiomics.readthedocs.io / en / v3.0.1 / index.html) was run using Python software to extract radiomic features of the region of interest. The region of interest was obtained by ANDing the angiography image and the corresponding segmentation mask image. The extracted radiomic features included the grayscale features, shape features, and texture features of the aneurysm.

[0089] Specifically, the extracted features can be divided into the following 7 groups, including: first-order statistical features, three-dimensional shape features, grayscale co-occurrence matrix, grayscale run length matrix, grayscale size area matrix, adjacent grayscale difference matrix and grayscale dependence matrix. Related studies have pointed out that the texture pattern in the aneurysm area is likely to be caused by the uneven distribution of contrast agents, and this uneven distribution pattern is considered to be related to turbulence. Therefore, the texture pattern of the aneurysm indirectly reflects the hemodynamics within the aneurysm to a certain extent.

[0090] 4.3 Deep Learning Features Based on Angiography

[0091] In order to obtain the corresponding high-level image features, a three-dimensional convolutional neural network method is used to extract deep learning features of the image. In an embodiment of the present invention, a classification network 3D EfficientNet-B0 (known to those skilled in the art) is selected as a feature extractor, and the angiography image and the corresponding mask are respectively used as inputs of the convolutional neural network, and three different deep learning features are calculated by a preset classification model. The training data of the preset classification model are angiography images of subjects with aneurysms and 3D segmentation mask images (Ground Truth) of aneurysms annotated by experienced medical experts, and the classification network is trained with the rupture state of the aneurysm as a label, wherein the rupture state of the aneurysm is obtained by following up the training data object.

[0092] The three different deep learning features are described below:

[0093] 1) No-sigmoid type features: The feature map is passed through each layer of the network structure of EfficientNet-B0 until the final fully connected layer. The output is used as the final No-sigmoid type deep learning feature, which can be expressed as:

[0094] f no-sigmoid =model(x),x=img or mask (4)

[0095] In formula (4), model is the pre-trained 3D EfficientNet-B0 model, img is the angiography image, and mask is the corresponding mask image.

[0096] 2) Sigmoid type features: The above No-sigmoid type features are passed through a sigmoid function to obtain the final Sigmoid type deep learning features, which can be expressed as:

[0097] f sigmoid =σ(model(x)),x=img or mask (5)

[0098] The σ in formula (5) is the sigmoid function defined by the following formula:

[0099]

[0100] 3) Binarized features: The above Sigmoid features are binarized to obtain the final binary deep learning features, which can be expressed as:

[0101] f binarization =B(σ(model(x))),x=img or mask (7)

[0102] The function B in formula (7) is a binarization operation, which is defined by the following formula:

[0103]

[0104] 4.4 Based on patient-specific clinical characteristics

[0105] Non-numerical clinical information (such as gender, smoking history, history of hypertension, etc.) is quantified and encoded to obtain numerical clinical characteristics.

[0106] Step 5: Risk Detection Module

[0107] According to step 4, the multidimensional features of the object to be detected are obtained and input into the preset aneurysm rupture risk detection model to obtain the rupture risk of the object to be detected. The aneurysm rupture risk detection model reflects the correlation between the features most relevant to the aneurysm rupture risk and the aneurysm rupture situation.

[0108] The preset aneurysm rupture risk detection model is built on the basis of training data, which includes: clinical data of subjects diagnosed with aneurysms, intracranial angiography images, 3D segmentation mask images (Ground Truth) of aneurysms annotated by medical experts, and the rupture status of aneurysms. The rupture status of aneurysms is obtained by following up the training data subjects, and recording whether the aneurysm ruptures within 5 years of the follow-up subjects. The establishment of the preset model is divided into two steps: ① feature screening; ② classifier construction.

[0109] 5.1 Feature Screening

[0110] In order to improve the stability and generalization performance of the model and avoid the influence of redundant features on the classification results, a feature screening algorithm is used to select key features and obtain the best feature subset. Specifically, the training data undergoes the above steps 1-4 in sequence to obtain a multidimensional feature set containing four multi-source features. First, these feature values ​​are standardized to eliminate the scale differences between different types of features. Then, the support vector machine recursive feature elimination method (SVM-RFE) can be used to select the optimal feature subset for the standardized data for the subsequent construction of the risk detection model.

[0111] 5.2 Classifier Construction

[0112] The features retained in the best feature subset of the above training data are used as input, and the aneurysm rupture status of the training data is used as the true label in the training process. A machine learning classifier (for example, support vector machine model, random forest model, etc.) is used to build an aneurysm rupture risk detection model. The aneurysm rupture status corresponding to the training data samples comes from the follow-up of the training data objects, and whether the aneurysm ruptures within 5 years of the follow-up objects is recorded. The pre-trained model characterizes the correlation between the features most relevant to the risk of aneurysm rupture and the aneurysm rupture status.

[0113] The above operations are performed on the object to be detected in sequence to obtain a multi-dimensional feature set of the object to be detected. First, irrelevant features in the object to be detected are eliminated to obtain the optimal feature set, which is then input into a preset detection model to obtain the aneurysm rupture risk detection result of the object to be detected.

[0114] To sum up, the main purpose of the embodiment of the present invention is to propose a detection device for the risk of intracranial aneurysm rupture based on multi-dimensional feature fusion. First, the aneurysm image is obtained through deep learning segmentation technology, and then the rupture status of the aneurysm is classified and detected based on the geometric morphological features and image features of the aneurysm and the machine learning technology.

[0115] Example 3

[0116] The feasibility of the schemes in Examples 1 and 2 is verified by combining specific experiments, as described below:

[0117] First, collect the intracranial angiography images and related clinical data of the subject to be tested. The imaging data can be DSA, CTA, MRA, etc., and it is required to be three-dimensional imaging data. The clinical data is divided into two parts: demographic data (including gender, age, etc.) and the patient's medical history (history of hypertension, smoking history, family history related to aneurysms, etc.).

[0118] Next, a series of preprocessing operations are performed on the image data of the object to be detected, such as denoising, skull removal, resampling and grayscale normalization, etc. The processed image is input into the preset segmentation model 3D LAU-Net to obtain the segmented aneurysm image. The segmented image is used to establish a three-dimensional model of the aneurysm through a three-dimensional reconstruction module. Then the original image and segmented image, the reconstructed three-dimensional model and the corresponding clinical data obtained above are input into the multidimensional feature extraction module for feature extraction to obtain a multidimensional feature set. The feature set comes from the following four features: geometric morphological features based on the three-dimensional model, imaging features and deep learning features based on angiography images and segmented images, and clinical features. Finally, irrelevant features are eliminated according to the preset optimal feature set, and the rupture risk of the object to be detected is obtained through the preset aneurysm rupture risk detection model.

[0119] Detection of the risk of rupture of cerebral aneurysms is the key to guiding the clinical selection of surgical timing, preventing aneurysm rupture and bleeding, and reducing mortality and disability. The invention can effectively achieve the task of detecting the risk of rupture of aneurysms, provide auxiliary diagnosis suggestions for clinicians, and obtain objective social and economic benefits.

[0120] The best implementation plan is to adopt patent transfer, technical cooperation or product development. The products developed based on this invention can be used to assist clinicians in the routine diagnosis of aneurysm patients and have a positive guiding significance for the subsequent formulation of treatment plans.

[0121] Unless otherwise specified, the models of the components in the embodiments of the present invention are not limited, and any device that can perform the above functions may be used.

[0122] Those skilled in the art will appreciate that the accompanying drawing is only a schematic diagram of a preferred embodiment, and the serial numbers of the embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0123] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A detection device for intracranial aneurysm rupture risk based on multi-dimensional feature fusion, characterized in that: The device comprises: An automatic segmentation module, used to obtain a segmented aneurysm image by introducing a preset segmentation model with a lesion-aware attention mechanism and a multi-level semantic supervision mechanism; A three-dimensional reconstruction module is used to establish a three-dimensional model of the aneurysm using the segmented aneurysm image as input to obtain a three-dimensional model of the aneurysm; The multidimensional feature extraction module extracts multidimensional features related to the risk of aneurysm rupture based on the angiography image, segmentation image and aneurysm 3D model of the object to be detected, including: geometric morphological features, imaging omics features, deep learning features and patient-specific clinical factors; The risk detection module is used to eliminate irrelevant features in the multidimensional feature set according to a preset optimal feature set, and then input the remaining features of the object to be detected into a preset detection model, and finally obtain the rupture risk of the aneurysm of the object to be detected; The lesion-aware attention mechanism is defined as follows: Attentionvalue=Boundary+Prediction Among them, Prediction represents the prediction map output by the preset segmentation model at each stage of upsampling, Boundary represents the prediction map calculated by the prediction map Prediction that only focuses on the target boundary contour area; Attention value represents the attention value of the target area and the target boundary contour pixels; represents the matrix element-by-element multiplication operation, ⊕ represents the matrix element-by-element addition operation; E represents the feature map of the encoder [E1, E2, E3, E4], E att Represents the feature map of the encoder after the attention mechanism calculation; The preset segmentation model is an improved 3D lesion-aware attention U-Net based on U-Net, which consists of an encoder, a decoder, multi-level skip connections, a lesion-aware attention mechanism and a multi-level semantic supervision mechanism.

2. The device for detecting the risk of intracranial aneurysm rupture based on multidimensional feature fusion according to claim 1, characterized in that: The multidimensional features include: geometric morphological features extracted from the three-dimensional model of the aneurysm; imaging features extracted from the angiographic image and the segmentation mask of the aneurysm; deep learning features extracted from the angiographic image and the segmentation mask of the aneurysm; and patient-specific clinical factors.

3. The device for detecting the risk of intracranial aneurysm rupture based on multi-dimensional feature fusion according to claim 2, characterized in that: The geometric morphological features include: manually measured morphological features and calculated curvature features, The curvature characteristics include: Gaussian curvature, minimum principal curvature, maximum principal curvature, and average curvature of the aneurysm; The maximum value, minimum value, mean value and standard deviation of the above curvature are calculated as curvature features respectively, and the curvature features are extracted using the pymeshlab toolkit.

4. The device for detecting the risk of intracranial aneurysm rupture based on multi-dimensional feature fusion according to claim 2, characterized in that: The deep learning features are extracted using a three-dimensional convolutional neural network. The classification network 3D EfficientNet-B0 was selected as the feature extractor. The angiography image and the corresponding aneurysm mask image were used as the input of the convolutional neural network. Three different deep learning features were calculated through the preset classification model. The training data of the preset classification model are angiography images of subjects with aneurysms and 3D segmentation mask images of aneurysms annotated by experienced medical experts, and the classification network is trained using the aneurysm rupture status as a label.

5. The device for detecting the risk of intracranial aneurysm rupture based on multi-dimensional feature fusion according to claim 4, characterized in that: The three different deep learning features calculated by the preset classification model are: 1) No-sigmoid type features: The feature map is passed through each layer of the network structure of EfficientNet-B0 until the final fully connected layer, and the output obtained is used as the final No-sigmoid type deep learning feature, expressed as: f no-sigmoid =model(x),x=img or mask Where model is the pre-trained 3D EfficientNet-B0 model, img is the angiography image, and mask is the corresponding mask image; 2) Sigmoid type features: The No-sigmoid type features are passed through a sigmoid function to obtain the final Sigmoid type deep learning features, expressed as: f sigmoid =σ(model(x)),x=img or mask 3) Binarized features: The above Sigmoid features are binarized to obtain the final binary deep learning features, which are expressed as: f binarization =B(σ(model(x))),x=img or mask The function B in the formula is a binarization operation.

Images