Deep learning-based abdominal aortic aneurysm identification method and device, storage medium, and electronic device

Abstract

The application discloses an abdominal aortic aneurysm identification method and device based on deep learning, a storage medium and an electronic device, wherein the method comprises the following steps: acquiring a CT or CTA image, and cutting and extracting an image patch, wherein the CT or CTA image comprises abdominal aortic aneurysm data; inputting the image patch into a preset image classification model to obtain a classification result of an abdominal aorta region in the image patch, wherein the preset image classification model is obtained by training a convolutional neural network using multiple groups of data, each group of data in the multiple groups of data comprising abdominal aortic aneurysm data and a lesion region label corresponding to the abdominal aortic aneurysm data; and performing segmentation processing on the classification result of the abdominal aorta region to obtain an abdominal aorta diameter measurement value as an abdominal aortic aneurysm identification result. The application realizes automatic and accurate identification of abdominal aortic aneurysm and improves the diagnosis efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and medical technology, in particular to an abdominal aortic aneurysm recognition method and device based on deep learning, a storage medium and an electronic device. BACKGROUND

[0002] An abdominal aortic aneurysm is a cardiovascular disease, and its main manifestation is the local deformation (swelling or enlargement) of the aorta occurring between the renal artery and the iliac artery. The normal diameter of the aorta is about 20 mm, and when the diameter of the infrarenal aorta is increased by 50% compared with the normal diameter, it is generally considered to be an aneurysm. Abdominal aortic aneurysm is an asymptomatic disease before rupture, but if the abdominal aortic aneurysm ruptures, the mortality rate will be as high as more than 50%. Abdominal aortic aneurysm is one of the most common diseases in the elderly population, especially in men. Among men over 65 years old, about 1.3% of deaths are caused by abdominal aortic aneurysm rupture.

[0003] Smaller abdominal aortic aneurysms can generally be identified together with other diseases in CT, but due to the attention to other clinical problems and the time-consuming nature of detailed analysis of abdominal aortic aneurysms, it may lead to missed reporting and delayed diagnosis of abdominal aortic aneurysms. Therefore, patients may be discharged without finding early abdominal aortic aneurysms, leading to delayed treatment. If larger abdominal aortic aneurysms can be operated or intervened, spontaneous rupture can be prevented. Therefore, the segmentation and recognition method of abdominal aortic aneurysm is particularly important.

[0004] In related technologies, radiologists will use CT or CTA images to segment abdominal aortic aneurysms to diagnose the severity of the disease and accurately measure the aortic diameter. However, abdominal aortic aneurysm three-dimensional (3D) visualization is a time-consuming and labor-intensive task because it needs to be done slice by slice, and human errors may occur in diagnosis. Today, the overall diagnosis rate of abdominal aortic aneurysm is low, only 6.3% in ten years. Therefore, it would be very beneficial to propose a fully automatic method for abdominal aortic aneurysm segmentation, recognition and diameter measurement.

[0005] In recent years, researchers have proposed various methods for segmenting and diagnosing abdominal aortic aneurysms, which can be roughly divided into two categories: semi-automatic (human intervention in diagnosis) and fully automatic (completed entirely by a computer). Generally speaking, there are several categories of abdominal aortic aneurysm segmentation techniques, such as machine learning techniques, deformable models, probabilistic graphical models, and knowledge-based methods. Most of the existing research methods for detecting and diagnosing abdominal aortic aneurysms are semi-automatic methods, and human intervention is unavoidable, which is time-consuming and labor-intensive, and prone to human errors. Although a few use fully automatic methods, they lack accuracy, with an error rate of 10% to 20%. Therefore, it is necessary to invent a more accurate fully automatic method.

[0006] In the related art, the accuracy of abdominal aortic aneurysm identification is not high, and no effective solution has been proposed. SUMMARY

[0007] The main purpose of the present application is to provide a deep learning-based abdominal aortic aneurysm identification method and device, storage medium, and electronic device to solve the problem of low accuracy of abdominal aortic aneurysm identification.

[0008] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a deep learning-based abdominal aortic aneurysm identification method is provided.

[0009] According to the deep learning-based abdominal aortic aneurysm identification method, it comprises

[0010] Obtaining a CT or CTA image, and cutting and extracting an image patch, wherein the CT or CTA image comprises abdominal aortic aneurysm data;

[0011] Inputting the image patch into a preset image classification model to obtain a classification result of an abdominal aortic region in the image patch, wherein the preset image classification model is obtained by training a convolutional neural network using multiple sets of data, each set of data in the multiple sets of data comprising abdominal aortic aneurysm data and a lesion region label corresponding to the abdominal aortic aneurysm data; and

[0012] Segmenting the classification result of the abdominal aortic region to obtain an abdominal aortic diameter measurement value as an abdominal aortic aneurysm identification result.

[0013] Further, the preset image classification model comprises a convolutional neural network structure, which at least comprises:

[0014] One data layer as input, three convolutional layers for feature extraction, five RELU layers as activation function layers, one pooling layer for reducing network parameters, two fully connected layers, one discard layer, one precision layer for calculating training and validation data set precision, and one loss layer for calculating data loss in the training phase.

[0015] Further, the preset image classification model further comprises a 3DResNet convolutional neural network for three-dimensional classification of different abdominal regions of the image patch,

[0016] For the three-dimensional classification data training set, the result of each scan is classified into no abdominal aortic aneurysm level or abdominal aortic aneurysm level;

[0017] In each scan, the axial position of the intracranial left renal artery opening of the aorta is used as the positioning point, and then the standardized size of the sub-volume is automatically extracted in the training;

[0018] or,

[0019] For the three-dimensional classification data training set, a stratified five-fold cross-validation method is used.

[0020] Further, the image patch is input into a preset image classification model to obtain a classification result of an abdominal aorta region in the image patch, including:

[0021] Five indexes are used to evaluate the quality of the predicted classification of the abdominal aorta region, wherein the predicted classification includes: true positive TP representing a sample correctly identified as an abdominal aortic aneurysm, true negative TN representing a correctly classified non-abdominal aortic aneurysm sample, false positive FP representing an abdominal aortic aneurysm sample incorrectly classified as a non-abdominal aortic aneurysm sample, and false negative FN representing a non-abdominal aortic aneurysm sample incorrectly classified as an abdominal aortic aneurysm sample.

[0022] The consistency between the abdominal aorta pair network decision related region and the predetermined aorta position is evaluated, wherein the influence score of the abdominal aorta pair network decision includes: 1-no aorta correlation, 2-low aorta correlation, 3-medium aorta correlation, 4-high aorta correlation, and 5-very high aorta correlation.

[0023] Further, the classification result of the abdominal aorta region is segmented to obtain an abdominal aorta diameter measurement value as an abdominal aortic aneurysm identification result, including:

[0024] Based on the Hough circle detection algorithm of the OpenCV library, the classification result of the abdominal aorta region is segmented,

[0025] The segmentation processing result is measured to obtain an abdominal aorta diameter measurement value.

[0026] Further, after obtaining the abdominal aorta diameter measurement value as the abdominal aortic aneurysm identification result, it further includes

[0027] If the measured diameter is less than 24 pixels, output the accurate diameter in millimeters and the first abdominal aortic aneurysm risk assessment result;

[0028] If the measured diameter is greater than 24 pixels and less than 40 pixels, output the accurate diameter in millimeters and the second abdominal aortic aneurysm risk assessment result;

[0029] If the measured diameter exceeds 40 pixels, output the accurate diameter in millimeters and the third abdominal aortic aneurysm risk assessment result.

[0030] Further, a CT or CTA image including abdominal aortic aneurysm data is obtained, and an image patch is extracted by cropping, including:

[0031] obtaining a CT or CTA image and adjusting the CT or CTA image to a preset size;

[0032] scanning the CT or CTA image through a preset size window to obtain an image patch by cropping the CT or CTA image;

[0033] After the image patch is extracted, the image patch is divided into at least one of the following regions: an abdominal region, an aorta, a body boundary, and a bone.

[0034] To achieve the above object, according to another aspect of the present application, there is provided an abdominal aortic aneurysm recognition device based on deep learning.

[0035] According to the abdominal aortic aneurysm recognition device based on deep learning, comprising:

[0036] An obtaining module is configured to obtain a CT or CTA image, and crop and extract an image patch, wherein the CT or CTA image includes abdominal aortic aneurysm data.

[0037] A classification module is configured to input the image patch into a preset image classification model to obtain a classification result of an abdominal aorta region in the image patch, wherein the preset image classification model is obtained by training a convolutional neural network using a plurality of sets of data, each set of data in the plurality of sets of data including abdominal aortic aneurysm data and a lesion region label corresponding to the abdominal aortic aneurysm data; and

[0038] A segmentation measurement module is configured to perform segmentation processing on the classification result of the abdominal aorta region to obtain an abdominal aorta diameter measurement value as an abdominal aortic aneurysm recognition result.

[0039] To achieve the above object, according to still another aspect of the present application, there is provided a computer readable storage medium having a computer program stored therein, wherein the computer program is configured to execute the method when running.

[0040] To achieve the above object, according to still another aspect of the present application, there is provided an electronic device comprising a memory and a processor, wherein the memory has a computer program stored therein, and the processor is configured to execute the computer program to execute the method.

[0041] The method and device for identifying abdominal aortic aneurysm based on deep learning, the storage medium and the electronic device in the embodiments of the present application adopt the method of obtaining CT or CTA images, cropping and extracting image patches, inputting the image patches into a preset image classification model, and obtaining the classification result of the abdominal aortic region in the image patches, so as to achieve the purpose of segmenting the classification result of the abdominal aortic region, obtaining the abdominal aortic diameter measurement value as the identification result of the abdominal aortic aneurysm, thereby realizing the technical effects of accurately identifying the abdominal aortic aneurysm and improving the diagnosis efficiency, and further solving the technical problem of low accuracy of abdominal aortic aneurysm identification.

[0042] The method in the embodiments of the present application is a full-automatic deep learning three-dimensional abdominal aortic aneurysm segmentation and identification algorithm which is easy to train. The method can be run as a background program of a clinical workflow, and has the characteristics of robustness, reliability, high accuracy and full automation. BRIEF DESCRIPTION OF DRAWINGS

[0043] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The illustrative embodiments of the application, and their description, do not limit the application. In the drawings:

[0044] Figure 1 FIG. 1 is a schematic diagram of the hardware structure of the method for identifying abdominal aortic aneurysm based on deep learning according to the embodiments of the present application;

[0045] Figure 2 FIG. 2 is a schematic diagram of the method flow for identifying abdominal aortic aneurysm based on deep learning according to the embodiments of the present application;

[0046] Figure 3 FIG. 3 is a schematic diagram of the structure of the device for identifying abdominal aortic aneurysm based on deep learning according to the embodiments of the present application;

[0047] Figure 4 FIG. 4 is a schematic diagram of the method flow for identifying abdominal aortic aneurysm based on deep learning according to the preferred embodiments of the present application. DETAILED DESCRIPTION

[0048] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0049] It should be noted that the terms "first", "second" and "third" and the like in the description and in the claims of the present application are used for distinguishing between similar elements and not necessarily for describing a specific sequential or chronological order. It is to be understood that the use of these terms is interchangeable under appropriate circumstances such that the embodiments of the present application described herein are, for example, capable of orderly or inverse order. Moreover, the terms "include", "have" and "contain" and their variations, when used in this description and in the claims of the present application, are intended to cover a non-exclusive inclusion such that a process, method, system, product or apparatus that comprises a list of steps or units are not necessarily limited to those steps or units which are clearly recited.

[0050] In the present application, the terms "upper", "lower", "left", "right", "front", "rear", "top", "bottom", "inner", "outer", "middle", "vertical", "horizontal", "lateral", "longitudinal" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. These terms are mainly used for better describing the present application and its embodiments, and are not intended to limit the indicated devices, elements or components to have a specific orientation, or to be constructed and operated in a specific orientation.

[0051] In addition, in addition to being used to indicate the orientation or positional relationship, the above-mentioned partial terms can also be used to indicate other meanings, for example, the term "upper" can also be used to indicate a certain dependent relationship or connection relationship in some cases. For those skilled in the art, the specific meaning of these terms in the present application can be understood according to the specific circumstances.

[0052] In addition, the terms "mount", "set", "provided with", "connected", "linked", "sleeved" should be interpreted broadly. For example, it can be fixedly connected, detachably connected, or integrally constructed; it can be mechanically connected, or electrically connected; it can be directly connected, or indirectly connected through an intermediate medium, or it can be internal communication between two devices, elements or components. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0053] It should be noted that the embodiments and features in the embodiments in the present application can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0054] The inventors found that deep learning can quickly and accurately analyze image data, and thus is very suitable for the management of abdominal aortic aneurysms. As deep learning is introduced into various applications in medicine, a new era is emerging in which doctors are provided with technical support and guidance. In existing research, the growth prediction algorithm for abdominal aortic aneurysms based on CTA scans and its potential in detecting aortic dissection and monitoring complications of endovascular aneurysm repair treatment have been proven.

[0055] In the related art, image intensity features and discriminative random forest (RF) classifiers are used for segmentation of abdominal aortic aneurysm regions. However, in this method, the program parameters of a large data set need to be fine-tuned, and the model needs to be pre-trained.

[0056] In the related art, a region-level set method based on energy minimization introduces Bayesian risk into the level set, which can deal with classification errors in the segmentation process. However, this method is semi-automatic, and the user is still required to select or draw a polygon around the abdominal aortic aneurysm region to perform initial contour drawing, which is prone to human error.

[0057] In the related art, a deep belief network (DBN) classifier is the existing fully automatic method, however, this algorithm is affected by the size of the aorta, and in a training network with a batch size of 160, patches containing large abdominal aortic aneurysms are missed; in a network with a batch size of 100, patches in other abdominal regions are missed. The error rate is between 10% and 20%, and the accuracy is poor.

[0058] To address the above problems, the deep learning-based abdominal aortic aneurysm recognition method in the embodiments of the present application does not need to fine-tune and pre-train the classifier, and early model construction has been completed through a large data set before this. At the same time, the method in the embodiments of the present application is a fully automatic algorithm, compared with the semi-automatic algorithm, the determination of the aortic boundary will be completed through the Hough circle detection algorithm, reducing the manual intervention.

[0059] Further, the deep learning-based abdominal aortic aneurysm recognition method in the embodiments of the present application is based on a convolutional neural network classifier, and compared with the existing fully automatic algorithm, the result has better accuracy (98.62%) in aortic detection, and is not affected by the size of the aorta and the degree of rotation of the extracted patch.

[0060] As shown in FIG. 1, Figure 1 The method includes the following steps S101 to S103:

[0061] In step S101, a CT or CTA image is obtained, and an image patch is extracted by cropping, wherein the CT or CTA image includes abdominal aortic aneurysm data.

[0062] First, a preset image classification model needs to be trained, and a CT or CTA image needs to be obtained when identifying.

[0063] Specifically, during training, a CT or CTA image is input, abdominal aortic aneurysm data is obtained, and the abdominal aortic aneurysm data is adjusted to an image with a size of 384x384. A 64x64 size window is used to scan the CT or CTA image to crop a patch from the image at an adjustable step rate.

[0064] In step S102, the image patch is input to the preset image classification model to obtain a classification result of the abdominal aortic region in the image patch. The preset image classification model is trained by a convolutional neural network using multiple sets of data, and each set of data includes abdominal aortic aneurysm data and a lesion region label corresponding to the abdominal aortic aneurysm data.

[0065] It can be understood that the convolutional neural network (CNN) is a famous deep learning architecture, which is inspired by the natural visual perception mechanism of living beings. A basic convolutional neural network is composed of three types of layers, namely convolutional layers, pooling layers and fully connected layers. Convolutional neural network is one of the most important deep learning algorithms, and its multi-layer structure can be trained in a powerful way. Since the convolutional neural network has strong feature extraction capability, it is often used in image processing projects. The convolutional neural network needs to be trained through two steps: (1) Feed Forward, (2) Backpropagation. First, the network learns features through the convolution process and calculates the network error using the loss function; second, the chain rule in the backpropagation algorithm is used to minimize the resulting error. Repeat these two steps until the network error reaches a minimum.

[0066] The image patch is input to the preset image classification model to obtain a classification result of the abdominal aortic region in the image patch.

[0067] The preset image classification model is trained by a convolutional neural network using multiple sets of data.

[0068] Each set of data in the multiple sets of data is trained as a training set, and each set of data includes abdominal aortic aneurysm data and a lesion region label corresponding to the abdominal aortic aneurysm data, i.e., abdominal aortic aneurysm data (region) and a label corresponding to the region.

[0069] In step S103, the classification result of the abdominal aortic region is segmented to obtain an abdominal aortic diameter measurement as an abdominal aortic aneurysm identification result.

[0070] The Hough transform is a method of detecting curves by exploiting the duality between points on a curve and parameters on that curve. Hough circle detection algorithms and generalized Hough transforms can be used to detect arbitrary shapes in an image.

[0071] By segmenting the classification result of the abdominal aortic region in the above manner, an abdominal aortic diameter measurement value is obtained, and the severity can be further identified according to the abdominal aortic diameter measurement value.

[0072] From the above description, it can be seen that the present application achieves the following technical effects:

[0073] By obtaining CT or CTA images, cropping and extracting image patches, inputting the image patches into a preset image classification model, and obtaining the classification result of the abdominal aortic region in the image patches, the classification result of the abdominal aortic region is segmented and processed, and an abdominal aortic diameter measurement value is obtained as an abdominal aortic aneurysm identification result, thereby achieving the technical effects of accurately identifying abdominal aortic aneurysms and improving diagnostic efficiency, and further solving the technical problem of low accuracy of abdominal aortic aneurysm identification.

[0074] In the embodiments of the present application, a convolutional neural network is used as a classifier to detect the aortic region, and a Hough circle detection algorithm is used to measure the aortic diameter. A fixed learning strategy is used to reduce the number of layers and learning rate, and the indicators have the most advanced results, improving the accuracy of detection. By using the most advanced detection results, abdominal aortic aneurysms are successfully detected and classified, which can be used in clinical practice to improve diagnostic efficiency.

[0075] As a preferred embodiment in the present embodiment, the preset image classification model includes a convolutional neural network structure, which at least includes one data layer as input, three convolutional layers for feature extraction, five RELU layers as activation function layers, one pooling layer for reducing network parameters, two fully connected layers, one discard layer, one precision layer for calculating training and validation data set precision, and one loss layer to calculate data loss in the training phase.

[0076] As Figure 2As shown, in particular implementation, in order to construct a classifier for abdominal aortic aneurysm patch detection and abdominal region classification, the design and training of the convolutional neural network is performed on an Nvidia Geoforce 1060 graphics card with 6GB of memory. The reason for choosing the convolutional neural network to solve the problem of abdominal aortic aneurysm image segmentation is that it is believed that the advantages of this deep architecture can be used to effectively solve the problem. Although convolutional neural networks require large data sets, they are very powerful and they can also provide the most advanced results. Each convolutional neural network is composed of several layers of data layers, convolutional layers, RELU layers, pooling layers, fully connected layers, dropout layers, etc.

[0077] Based on the above, a convolutional neural network classifier is proposed.

[0078] The convolutional neural network layers are arranged in various ways, and the parameters of the layers also change frequently. Each designed network is trained by a training data set and verified by a verification data set. Finally, the best design obtained is as shown in Figure 2 .

[0079] The structure includes a data layer as input, three convolutional layers for feature extraction, five RELU layers as activation function layers, a pooling layer for reducing network parameters, two fully connected layers, a dropout layer with a ratio of 0.1, an accuracy layer to calculate the accuracy of the training and verification data sets, and a loss layer to calculate the data loss in the training phase.

[0080] Convolutional layers (Conv1, Conv2, Conv3), convolutional layers use different kernel sizes to extract features using the output of the previous layer.

[0081] ReLU layers (ReLU1, ReLU2, ReLU3, ReLU4), the rectified function f(x) = Max(0, x) is an activation function, which is used to add nonlinearity to the network and can only calculate linear functions.

[0082] Pooling layer (Pooling 1), the pooling layer reduces the sampling of network parameters, reduces parameters, and facilitates network calculation.

[0083] Fully connected layers (Fully Connected layers, FC1, FC2), most convolutional neural networks output in fully connected layers. These fully connected layers are connected to all the neurons of the previous layer, and usually extract high-level features from the data and output.

[0084] Dropout layer (Dropout 1), this layer randomly reduces network parameters to prevent the network from being overfitted.

[0085] Accuracy layer, this layer is used to calculate the percentage of correct predictions in the data set.

[0086] loss layer, which is usually used as the last layer of a convolutional neural network, is used to define the loss of the network during the training and testing phase.

[0087] In this network, by using SoftMax (formula 1) with loss (i.e. normalized exponential function or SoftMax regression), it can derive the probability of each target class given the model prediction and calculate the multinomial logistic loss for output.

[0088]

[0089] where yi is the network output and k is the number of outputs.

[0090] In some embodiments, solver parameters need to be set.

[0091] Training a well-designed convolutional neural network requires modifying some hyperparameters. In the Caffe framework, these parameters are modified in a solver.prototxt file, which specifies the learning algorithm. Adam is a gradient-based stochastic objective function optimization algorithm. Since the convergence speed of Adam algorithm is very fast and easy, Adam solver is chosen as the solver type.

[0092] In practice, although most of the designed convolutional neural networks have many layers, the number of layers is reduced in this classifier. In order to compensate for the effect of reducing the number of layers, the learning rate is reduced to 0.0001 (without further reduction) using a fixed learning strategy, which works well. The parameters of the solver file are modified as follows:

[0093] Type: Adam, test iterations: 15, test interval: 200, learning rate (Lr-base): 0.0001, maximum number of iterations: 4000, learning strategy: fixed.

[0094] In some embodiments, five-fold cross-validation is used. After the first round of training, five cross-validations are performed to estimate the stability of the classifier. Each time, when the mirror conversion is true in the training phase, one fold is excluded (as a validation dataset), and the other data is used for training.

[0095] As preferred in the present embodiment, the preset image classification model further comprises: a 3DResNet convolutional neural network, used for three-dimensional classification of different abdominal regions of the image patch, and for a three-dimensional classification data training set, the result of each scan is divided into no abdominal aortic aneurysm level or abdominal aortic aneurysm level; in each scan, the axial position of the beginning of the intracranial left renal artery of the aorta is taken as the positioning point, and then the positioning point is used to automatically extract a standardized size sub-volume in the training.

[0096] Experiments have verified that the 3D ResNet in the embodiments of the application achieves the same high TPR and AUC values as the 2D screening algorithm, and the task of classifying the entire three-dimensional volume is more complex than classifying the selected two-dimensional patches, which is the first application of a three-dimensional convolutional neural network in CT abdominal aortic aneurysm screening. Compared with two-dimensional, three-dimensional screening is more comprehensive and accurate for the diagnosis of abdominal aortic aneurysm, and has higher benefits for assisting clinical diagnosis.

[0097] In specific implementation, the training data set is annotated in two ways. First, each CT scan is classified into two categories (0: no abdominal aortic aneurysm level, 1: abdominal aortic aneurysm level). Second, in each scan, the axial position of the beginning of the intracranial left renal artery of the aorta is used as a positioning point, and then a standardized size sub-volume is automatically extracted using the positioning point in training.

[0098] Alternatively, for the three-dimensional classification data training set, a stratified five-fold cross-validation method is used.

[0099] In specific implementation, the stratified five-fold cross-validation method is used for the initial training data set. The data is divided into five non-intersecting test sets. For each fold, one of the test sets is used, and 6 cases are selected from the non-test data for validation, and the remaining non-test cases are used for training. A three-dimensional patch of 320x384x224 is extracted from the CT set. This reduces the demand for memory and removes the part of the image related to air and the scanning bed. In order to make predictions, the patch is centered on the center of the anchor slice. The density window of the CT image is in the range of [-200HU, 400HU], thereby covering the extended soft tissue range, and then mapped to the interval [-1, 1]. All images are resampled to a pitch of 0.9x0.9x1.5mm, which is equal to the median resolution of the data set.

[0100] During training, the anchor slice is moved up to 10 voxels along the cranio-caudal axis, and the patch center is randomly positioned on the slice. In addition, data augmentation is performed by rotating around the cranio-caudal axis by an angle α ∈ [-12.6°, 12.6°] to simulate possible patient positions, and scaling by 10% to simulate possible patient sizes. In addition, a density jitter of up to ±3HU is also applied.

[0101] The method in the embodiments of the application is designed to be trained on minimally annotated data. The positioning point is set to achieve full automatic screening of the validation data by a heuristic method, and can also be achieved by anatomical marker detection. This automation provides an opportunity for future training and validation on larger data sets.

[0102] The high robustness and reliability in the embodiments of the present application enable people to establish necessary trust in artificial intelligence systems, which is a key factor for realizing clinical use. The level-relatedness propagation (LRP) analysis can generate a graphical analysis of the network decision-related areas in the image, and the medical image system (PACS) outputting these results can significantly improve its clinical acceptance.

[0103] In some embodiments, the level-relatedness propagation needs to be considered. The level-relatedness propagation (LRP) can be used to calculate the voxel decomposition of the convolutional neural network decision, and therefore, it can be used to provide explainability for the convolutional neural network. The correlation is propagated backward layer by layer from the network output end until it reaches the input layer. The result is equivalent to a correlation map, which provides a correlation value for each input value on the output class. In our map, the positive correlation value represents the correlation with the abdominal aortic aneurysm class, and the negative correlation value represents no correlation with the abdominal aortic aneurysm class. We normalize the map by their sum.

[0104] In some embodiments, full-automatic screening is realized, and in order to realize full-automatic application of the screening algorithm, an automatic extraction method of the abdominal image region is adopted. The basic algorithm first analyzes the HU distribution along the z-axis in the soft tissue HU value range to determine the upper and lower boundaries of the abdomen. Then, according to the distribution of high HU values, the abdominal center corresponding to the region between the top of the hip bone and the lower rib bone is determined. Then, a subset is extracted at the abdominal center, and input into the network for classification.

[0105] After designing and training the convolutional neural network, the designed network is tested, and a total accuracy of 97.93% is obtained. The total sensitivity, precision and accuracy of the classifier proposed in the present application are 97.93%, 97.94% and 97.93% respectively, the obtained abdominal aorta region detection accuracy is 98.62%, and is not affected by the size of the abdominal aorta.

[0106] Through network comparison, LRP map, repeated training of the best network, verification on additional data sets and other experiments, it is proved that the 3DResNet shows high performance and robustness in the full-automatic abdominal aortic aneurysm detection of abdominal CT scans. The AUC of the network on the verification data set is 0.971, and the accuracy is 0.953. These results show that the 3D ResNet is suitable for clinical screening.

[0107] Metal interference has been proved to have a negative impact on the performance of the convolutional neural network, but in the present method, the accuracy of the algorithm is still high even if there are these artifacts, which is likely to be because the artifacts are included in the training data set, and the improvement of data enhancement in the training process.

[0108] In addition, the algorithm can achieve higher accuracy on a second, unseen dataset of different construction, despite only using a small dataset.

[0109] For the additional validation on a mixed dataset of abdominal external and internal CT scans, on the one hand, there is a risk of validating on a dataset that does not fully reflect the clinical reality. The external data contain fewer metal artifacts, and the abdominal aortic aneurysm cases tend to be simpler cases.

[0110] On the other hand, the inclusion of these additional cases also increases the variability of the CT data from international groups and different manufacturer hardware. This helps to understand the potential performance of the algorithm in different clinical conditions and the generalizability of the proposed method, demonstrating that the algorithm is sufficiently robust to obtain reliable results on a validation dataset composed of heterogeneous data from multiple sources.

[0111] As preferred in the embodiment, the inputting the image patches into the preset image classification model to obtain a classification result of the abdominal aortic region in the image patch comprises: adopting five indexes to evaluate the quality of the predicted classification of the abdominal aortic region, wherein the predicted classification comprises: true positive TP representing a sample correctly identified as an abdominal aortic aneurysm, true negative TN representing a non-abdominal aortic aneurysm sample correctly classified, false positive FP representing an abdominal aortic aneurysm sample incorrectly classified as a non-abdominal aortic aneurysm, and false negative FN representing a non-abdominal aortic aneurysm sample incorrectly classified as an abdominal aortic aneurysm.

[0112] In particular implementation, five indexes are adopted to evaluate the quality of the predicted classification. We use a discrimination threshold (DT) of 0.5 for the four binary indexes. They have four results: true positive (TP) representing a sample correctly identified as an abdominal aortic aneurysm, true negative (TN) representing a non-abdominal aortic aneurysm sample correctly classified, false positive (FP) representing an abdominal aortic aneurysm sample incorrectly classified as a non-abdominal aortic aneurysm, and false negative (FN) representing a non-abdominal aortic aneurysm sample incorrectly classified as an abdominal aortic aneurysm.

[0113] Thus, the accuracy (A), precision (P), true positive rate (TPR), false positive rate (FPR), and F1 score (F1) are calculated. These calculation results are compared with the predictions of the labels on the network. When the discrimination threshold of the binary classifier changes, the receiver operating characteristic (ROC) curve plots the ratio between TP and FP decisions, which assesses performance based on the predicted probabilities of the network and the labels. The area under the curve (AUC) provides a measure of the performance of the classifier.

[0114] Further, for evaluating the consistency between the abdominal aorta and the network decision related region and the predetermined aorta position, the abdominal aorta influence score of the network decision includes: 1-no aorta correlation, 2-low aorta correlation, 3-medium aorta correlation, 4-high aorta correlation, and 5-very high aorta correlation.

[0115] In particular implementation, in order to evaluate the correspondence between the correlation map and the aorta, we use a Likert 5-point scale to measure the consistency between the algorithm decision related region and the aorta position determined by the human radiologist. In the evaluation, the positive and negative correlation values are equal. The abdominal aorta influence score of the network decision is: 1 (no aorta correlation), 2 (low aorta correlation), 3 (medium aorta correlation), 4 (high aorta correlation), and 5 (very high aorta correlation).

[0116] As preferred in this embodiment, the classification result of the abdominal aorta region is segmented to obtain an abdominal aorta diameter measurement value as an abdominal aorta aneurysm identification result, including:

[0117] Based on the Hough circle detection algorithm of the OpenCV library, the classification result of the abdominal aorta region is segmented to measure the segmentation result and obtain an abdominal aorta diameter measurement value.

[0118] As preferred in this embodiment, after obtaining the abdominal aorta diameter measurement value as the abdominal aorta aneurysm identification result, it further includes: if the measured diameter is less than 24 pixels, output the accurate diameter in millimeters and the first abdominal aorta aneurysm risk assessment result; if the measured diameter is greater than 24 pixels and less than 40 pixels, output the accurate diameter in millimeters and the second abdominal aorta aneurysm risk assessment result; if the measured diameter exceeds 40 pixels, output the accurate diameter in millimeters and the third abdominal aorta aneurysm risk assessment result.

[0119] In particular implementation, after measuring the aorta diameter, it is divided into the following categories:

[0120] If the measured diameter is less than 24 pixels, output the accurate diameter in millimeters and display "no abdominal aorta aneurysm risk".

[0121] If the measured diameter is greater than 24 pixels and less than 40 pixels, output the accurate diameter in millimeters and display "moderate abdominal aorta aneurysm risk".

[0122] If the measured diameter exceeds 40 pixels, output the accurate diameter in millimeters and display "high abdominal aorta aneurysm risk".

[0123] The Hough circle detection algorithm has high sensitivity, precision and accuracy (98.41%, 98.33% and 98.41% respectively) in detecting the aortic edge and measuring the aortic diameter.

[0124] As preferred in the embodiment, the CT or CTA image including abdominal aortic aneurysm data is acquired and cropped to obtain the image patch, including: acquiring the CT or CTA image and adjusting the image to a preset size; scanning the CT or CTA image through a preset size window to crop the image patch from the image; after extracting the image patch, the image patch is divided into at least one of the following regions: abdominal inner region, aorta, body boundary, and bone.

[0125] In specific implementation, the patch extracted from the Marco Imaging diVision Lite software is in jpg format and has a size of 512*512, and the field of view (FOV) value captured is 480. Since the size of the network input is 64*64, the extracted image is first adjusted to have a size of 384*384 pixels to make the patch constitute a larger field of view, and then the CT or CTA image is scanned through a 64*64 window, and the patch is cropped from the original image by using the MATLAB code.

[0126] After the patch is extracted, it is divided into four categories: abdominal inner region, aorta, body boundary, and bone. In order to improve the quality of the data set, all patches are denoised by using the median filtering code of MATLAB. Since the training method is supervised training, all patches are labeled according to their corresponding categories. Labels 0, 1, 2, and 3 represent the abdominal inner region, aorta, body boundary, and bone, respectively.

[0127] In addition, in order to avoid overtraining of a certain specific category, the number of patches of all categories is the same. Before creating the basic LMDB file (the data set format required by the Caffe framework), all patches of all categories are shuffled. The more data sets, the better the training network, so in the training stage, the number of training data sets is doubled by using mirror transformation.

[0128] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0129] According to the embodiment of the present application, an abdominal aortic aneurysm recognition device based on deep learning is also provided, as shown in Figure 3 the device 300 comprises:

[0130] The acquisition module 310 is configured to acquire a CT or CTA image, and crop and extract an image patch from the CT or CTA image, wherein the CT or CTA image comprises abdominal aortic aneurysm data.

[0131] The classification module 320 is configured to input the image patch into a preset image classification model to obtain a classification result of an abdominal aortic region in the image patch, wherein the preset image classification model is obtained by training a convolutional neural network using a plurality of groups of data, and each group of data comprises abdominal aortic aneurysm data and a lesion region label corresponding to the abdominal aortic aneurysm data.

[0132] The segmentation measurement module 330 is configured to perform segmentation processing on the classification result of the abdominal aortic region to obtain an abdominal aortic diameter measurement value as an abdominal aortic aneurysm identification result.

[0133] In the acquisition module 310, a preset image classification model needs to be trained first, and a CT or CTA image needs to be acquired during identification.

[0134] Specifically, during training, a CT or CTA image is input, abdominal aortic aneurysm data is acquired, and the abdominal aortic aneurysm data is adjusted to an image with a size of 384x384. A 64x64 window is used to scan the CT or CTA image to crop a patch from the image at an adjustable step rate.

[0135] In the classification module 320, it can be understood that a convolutional neural network (CNN) is a famous deep learning architecture, which is inspired by the natural visual perception mechanism of biology. A basic convolutional neural network is composed of three types of layers, namely convolutional layers, pooling layers and fully connected layers. The convolutional neural network is one of the most important deep learning algorithms, and its multi-layer structure can be trained in a powerful way. Since the convolutional neural network has strong feature extraction capability, it is often used in image processing projects. The convolutional neural network needs to be trained through two steps: (1) feed forward (Feed Forward), (2) back propagation (Backpropagation). First, the network learns features through the convolution process, and calculates the network error using a loss function; second, the chain rule in the back propagation algorithm is used to minimize the error. Repeat these two steps until the network error reaches a minimum.

[0136] The image patch is input into the preset image classification model to obtain a classification result of an abdominal aortic region in the image patch.

[0137] The preset image classification model is obtained by training a convolutional neural network using a plurality of groups of data.

[0138] Further comprising training each group of data in the multiple groups of data as a training set, each group of data including abdominal aortic aneurysm data and the lesion region label corresponding to the abdominal aortic aneurysm data, i.e., including abdominal aortic aneurysm data (region) and the label of the corresponding region.

[0139] The classification result of the abdominal aortic region in the segmentation measurement module 330 of the embodiment of the present application is segmented and processed to obtain the abdominal aortic diameter measurement value as the abdominal aortic aneurysm recognition result.

[0140] The Hough transform is a method of detecting a curve by utilizing the duality between a point on the curve and a parameter on the curve. The Hough circle detection algorithm and the generalized Hough transform can be used to detect any shape in an image.

[0141] The classification result of the abdominal aortic region is segmented and processed in the above manner to obtain the abdominal aortic diameter measurement value, and the severity can be further recognized according to the abdominal aortic diameter measurement value.

[0142] In a specific embodiment, it can be understood that the preconditions to be met refer to the first registered user as the calling user end and the second registered user as the called user end.

[0143] In a preferred embodiment, the first registered user preinstalls the instant messaging type social application program through the first terminal, and the second registered user preinstalls the instant messaging type social application program through the second terminal. This can ensure the communication premise between the called user end and the calling user end.

[0144] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present application can be realized by a general computing device, which can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Alternatively, they can be realized by program codes executable by a computing device, so that they can be stored in a storage device and executed by a computing device, or they can be respectively manufactured into individual integrated circuit modules, or a plurality of modules or steps among them can be manufactured into a single integrated circuit module to realize. Thus, the present application is not limited to any specific combination of hardware and software.

[0145] In order to better understand the above-mentioned abdominal aortic aneurysm recognition method based on deep learning, the above-mentioned technical solutions will be explained in combination with preferred embodiments, but not used to limit the technical solutions of the embodiments of the present application.

[0146] The deep learning-based abdominal aortic aneurysm identification method in this application employs a convolutional neural network as a classifier to detect the aortic region and the Hough circle detection algorithm to measure the aortic diameter. A fixed learning strategy is used to simultaneously reduce the number of layers and the learning rate, achieving state-of-the-art results for each metric and improving detection accuracy. The algorithm successfully detects and classifies abdominal aortic aneurysms using the most advanced detection results, and can be applied clinically to improve diagnostic efficiency.

[0147] Furthermore, the highly accurate fully automated algorithm reduces human error, avoids misdiagnosis, significantly shortens diagnosis time, and improves diagnostic accuracy. Integrating this deep learning-based screening for abdominal aortic aneurysms into daily workflows can optimize patient monitoring, facilitate early diagnosis, improve patient treatment, and potentially reduce the risk of rupture.

[0148] like Figure 4 The diagram shown is a flowchart of the deep learning-based abdominal aortic aneurysm identification method in this application embodiment. The specific process includes the following steps:

[0149] Step S410: Input a CT or CTA image, scan the CT or CTA image, and crop patches from the image at an adjustable step rate.

[0150] Input CT or CTA images to acquire abdominal aortic aneurysm data and resize them to 384×384 pixels. Scan the CT or CTA images using a 64×64 window and crop patches from the images at an adjustable step rate. After extracting the patches, categorize them into four groups: internal abdominal region, aorta, body boundaries, and skeleton.

[0151] Step S420: Use a convolutional neural network classifier to classify the extracted patches and determine the aortic region.

[0152] By designing a convolutional neural network and training it with a 3D ResNet convolutional neural network, the extracted patches can be classified and the aortic region can be identified.

[0153] In step S430, the aorta is segmented using the Hough circle detection algorithm, and its diameter is measured.

[0154] The Hough circle detection algorithm from the OpenCV library, developed under the BSD license, was used to accurately determine the aortic boundary and measure its diameter by modifying some algorithm parameters.

[0155] Step S440: Determine the severity of the disease based on the aortic diameter.

[0156] If the measured diameter is less than 24 pixels, output the accurate diameter in millimeters and the first abdominal aortic aneurysm risk assessment result;

[0157] If the measured diameter is greater than 24 pixels and less than 40 pixels, output the accurate diameter in millimeters and the second abdominal aortic aneurysm risk assessment result;

[0158] If the measured diameter is greater than 40 pixels, output the accurate diameter in millimeters and the third abdominal aortic aneurysm risk assessment result.

[0159] The method in the embodiment of the application is an easy-to-train fully automatic deep learning three-dimensional abdominal aortic aneurysm segmentation and identification algorithm, which can be run as a background program of a clinical workflow and has the characteristics of robustness, reliability, high accuracy and full automation.

[0160] The embodiment of the application also provides a storage medium having a computer program stored therein, wherein the computer program is configured to execute the steps in any of the method embodiments when running.

[0161] Optionally, in the embodiment, the storage medium can be configured to store a computer program for executing the following steps:

[0162] S1, acquiring a CT or CTA image, and cutting and extracting an image patch, wherein the CT or CTA image comprises abdominal aortic aneurysm data;

[0163] S2, inputting the image patch into a preset image classification model to obtain a classification result of an abdominal aorta region in the image patch, wherein the preset image classification model is obtained by training a convolutional neural network using a plurality of groups of data, each group of data in the plurality of groups of data comprising abdominal aortic aneurysm data and a lesion region label corresponding to the abdominal aortic aneurysm data; and

[0164] S3, performing segmentation processing on the classification result of the abdominal aorta region to obtain an abdominal aorta diameter measurement value as an abdominal aortic aneurysm identification result.

[0165] Optionally, in the embodiment, the storage medium can include but is not limited to a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various storage media that can store computer programs.

[0166] The embodiment of the present application further provides an electronic device, comprising a memory and a processor, the memory stores a computer program, and the processor is configured to execute the computer program to perform the steps in any of the above method embodiments.

[0167] Optionally, the electronic device further comprises a transmission device and an input and output device, wherein the transmission device is connected with the processor, and the input and output device is connected with the processor.

[0168] Optionally, in the embodiment, the processor is configured to execute the following steps through the computer program:

[0169] S1, acquiring a CT or CTA image, and cutting and extracting an image patch, wherein the CT or CTA image comprises abdominal aortic aneurysm data;

[0170] S2, inputting the image patch into a preset image classification model to obtain a classification result of an abdominal aorta region in the image patch, wherein the preset image classification model is obtained by training a convolutional neural network using multiple groups of data, each group of data in the multiple groups of data comprising abdominal aortic aneurysm data and a lesion region label corresponding to the abdominal aortic aneurysm data; and

[0171] S3, performing segmentation processing on the classification result of the abdominal aorta region to obtain an abdominal aorta diameter measurement value as an abdominal aortic aneurysm recognition result.

[0172] Optionally, specific examples in the embodiment can refer to the examples described in the above embodiments and optional implementation manners, and the embodiment will not be described here again.

[0173] The above merely describes preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A deep learning-based abdominal aortic aneurysm identification method, characterized in that, The method comprises: acquiring a CT or CTA image, and cutting and extracting an image patch from the CT or CTA image, wherein the CT or CTA image comprises abdominal aortic aneurysm data, and the method comprises: acquiring the CT or CTA image and adjusting the CT or CTA image to a preset size; scanning the CT or CTA image through a preset size window to cut and extract an image patch from the image; after extracting the image patch, dividing the image patch into at least one of the following regions: an abdominal region, an aorta, a body boundary, and a bone; inputting the image patch into a preset image classification model to obtain a classification result of an abdominal aorta region in the image patch, wherein the preset image classification model is obtained by training a convolutional neural network using a plurality of sets of data, and each set of data in the plurality of sets of data comprises abdominal aortic aneurysm data and a lesion region label corresponding to the abdominal aortic aneurysm data; and segmenting the classification result of the abdominal aorta region to obtain an abdominal aorta diameter measurement value as an abdominal aortic aneurysm recognition result, comprising: segmenting the classification result of the abdominal aorta region based on a Hough circle detection algorithm of an OpenCV library, measuring the segmentation result to obtain the abdominal aorta diameter measurement value; after obtaining the abdominal aorta diameter measurement value as the abdominal aortic aneurysm recognition result, further comprising if the measured diameter is less than 24 pixels, outputting an accurate diameter in millimeters and a first abdominal aortic aneurysm risk assessment result; if the measured diameter is greater than 24 pixels and less than 40 pixels, outputting an accurate diameter in millimeters and a second abdominal aortic aneurysm risk assessment result; if the measured diameter is greater than 40 pixels, outputting an accurate diameter in millimeters and a third abdominal aortic aneurysm risk assessment result; the preset image classification model further comprises a 3DResNet convolutional neural network for three-dimensional classification of different abdominal regions of the image patch, for a three-dimensional classification data training set, each scan result is classified into a non-abdominal aortic aneurysm level or an abdominal aortic aneurysm level; in each scan, the axial position of the left renal artery opening of the aorta is used as a positioning point, and then a standardized size sub-volume is automatically extracted in the training; alternatively, for a three-dimensional classification data training set, a hierarchical five-fold cross-validation method is used.

2. The method of claim 1, wherein, The preset image classification model comprises a convolutional neural network structure, and the convolutional neural network structure comprises at least: one data layer as an input, three convolutional layers for feature extraction, five RELU layers as activation function layers, one pooling layer for reducing network parameters, two fully connected layers, one dropout layer, one precision layer for calculating training and validation data set precision, and one loss layer for calculating data loss in the training stage.

3. The method of claim 2, wherein, the inputting of the image patch into the preset image classification model to obtain the classification result of the abdominal aorta region in the image patch comprises: Five indicators are used to evaluate the quality of the abdominal aorta region prediction classification, wherein the prediction classification includes: true positive TP representing samples correctly identified as abdominal aortic aneurysm, true negative TN representing correctly classified non-abdominal aortic aneurysm samples, false positive FP representing abdominal aortic aneurysm samples incorrectly classified as non-abdominal aortic aneurysm, and false negative FN representing non-abdominal aortic aneurysm samples incorrectly classified as abdominal aortic aneurysm; The consistency between the abdominal aorta and the network decision related region and the predetermined aorta position is evaluated, wherein the influence score of the abdominal aorta on the network decision includes: 1-no aorta relevance, 2-low aorta relevance, 3-medium aorta relevance, 4-high aorta relevance, and 5-very high aorta relevance.

4. A deep learning-based abdominal aortic aneurysm recognition device, characterized by, The device comprises: An acquisition module is configured to acquire a CT or CTA image, and to crop and extract an image patch, wherein the CT or CTA image includes abdominal aortic aneurysm data; The acquisition module is further configured to: Acquire the CT or CTA image and adjust it to a preset size; Scan the CT or CTA image through a window of a preset size to crop an image patch from the image; After extracting the image patch, divide the image patch into at least one of the following regions: an abdominal inner region, an aorta, a body boundary, and a skeleton; A classification module is configured to input the image patch into a preset image classification model to obtain a classification result of an abdominal aorta region in the image patch, wherein the preset image classification model is obtained by training a convolutional neural network using multiple sets of data, and each set of data includes abdominal aortic aneurysm data and a lesion region label corresponding to the abdominal aortic aneurysm data; and A segmentation measurement module is configured to perform segmentation processing on the classification result of the abdominal aorta region to obtain an abdominal aorta diameter measurement value as an abdominal aortic aneurysm identification result; The segmentation measurement module is further configured to: Perform segmentation processing on the classification result of the abdominal aorta region based on a Hough circle detection algorithm of an OpenCV library, Measure the segmentation processing result to obtain an abdominal aorta diameter measurement value; After obtaining the abdominal aorta diameter measurement value as an abdominal aortic aneurysm identification result, further comprising If the measured diameter is less than 24 pixels, output the accurate diameter in millimeters and a first abdominal aortic aneurysm risk assessment result; If the measured diameter is greater than 24 pixels and less than 40 pixels, output the accurate diameter in millimeters and a second abdominal aortic aneurysm risk assessment result; If the measured diameter exceeds 40 pixels, output the accurate diameter in millimeters and a third abdominal aortic aneurysm risk assessment result; The preset image classification model further includes a 3DResNet convolutional neural network configured to perform three-dimensional classification of different abdominal regions on the image patch, For a three-dimensional classification data training set, each scan result is classified into a non-abdominal aortic aneurysm level or an abdominal aortic aneurysm level; In each scan, the axial position of the intracranial left renal artery opening of the aorta is used as a positioning point, and then a standardized size sub-volume is automatically extracted using the positioning point in the training; Or For the three-dimensional classification data training set, a stratified five-fold cross-validation method is used.

5. A computer readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is configured to execute the method in any one of claims 1 to 3 when running. 6.An electronic device comprising a memory and a processor, the electronic device comprising: The memory stores a computer program, and the processor is configured to execute the computer program to execute the method in any one of claims 1 to 3.

Images