Detection and classification of large vessel occlusion in medical imaging

Abstract

A system and method for occlusion detection in medical images are provided. The system receives an input medical image of one or more vessels in a patient's anatomical object. One or more anatomical landmarks are identified in the input medical image. Based on the identified anatomical landmarks, a first patch and one or more additional patches are extracted from the input medical image. The first patch and the additional patches depict different parts of the anatomical object. Features are extracted from the first patch and the additional patches using a machine learning-based feature extractor network. Based on the extracted features, with or without modeling features on a probability distribution function, occlusions in one or more vessels are detected in the first patch. The results of the detection are output.

Description

Technical Field

[0001] This invention relates generally to medical image analysis, and more specifically to the detection and classification of large vessel occlusion (LVO) in medical imaging. Background Technology

[0002] A stroke occurs when the blood supply to the brain is interrupted or reduced. Strokes can be classified as ischemic strokes caused by an interruption of blood supply to the brain or hemorrhagic strokes caused by a ruptured blood vessel. In current stroke protocols, if the stroke is ischemic, a computed tomography (CTA) scan is performed to determine if a large vessel occlusion (LVO) is present in one of the major arteries of the brain. Mechanical thrombectomy can then be performed to remove the LVO.

[0003] Various traditional artificial intelligence (AI)-based methods have been proposed for LVO detection. However, these traditional AI-based methods suffer from reduced robustness and performance in the presence of signal loss, noise, vascular tortuosity, calcification, and proximity to bone or bifurcation. Summary of the Invention

[0004] According to one or more embodiments, a system and method for occlusion detection in medical images are provided. An input medical image of one or more blood vessels in an anatomical object of a patient is received. One or more anatomical landmarks are identified in the input medical image. Based on the identified one or more anatomical landmarks, a first patch and one or more additional patches are extracted from the input medical image. The first patch and the one or more additional patches depict different parts of the anatomical object. Features are extracted from the first patch and the one or more additional patches using a machine learning-based feature extractor network. Based on the extracted features, occlusions in one or more blood vessels are detected in the first patch. The detection results are output.

[0005] In one embodiment, bone is removed from an input medical image. A first patch and one or more additional patches are extracted from the bone-removed input medical image.

[0006] In one embodiment, features are extracted from a first patch and one or more additional patches via the following steps: receiving the first patch via a first input channel of a machine learning-based feature extractor network and receiving one or more additional patches via a corresponding one of one or more additional input channels of the machine learning-based feature extractor network; and extracting 1) features from the first patch and 2) features comparing the first patch with one or more additional patches. Occlusions in one or more blood vessels are detected in the first patch.

[0007] In one embodiment, the first patch and one or more additional patches are cut centered on one or more anatomical landmarks.

[0008] In one embodiment, occlusions in one or more blood vessels are detected in a first patch using a probability distribution function (PDF) model fitted to features extracted by a neural network. A Gaussian process model can be used to learn the PDF model.

[0009] In one embodiment, the middle cerebral artery (MCA) bifurcation is identified in the input medical image. A first patch is cut with the MCA bifurcation centered on it. The occlusion is detected as being located in one of the internal carotid artery (ICA), the M1 segment of the MCA, or the M2 segment of the MCA.

[0010] In one embodiment, at least one probability map of the presence of blood vessels is generated for at least one of a first patch or one or more additional patches. Features are extracted from the at least one probability map using a machine learning-based feature extractor network.

[0011] In one embodiment, the anatomical object includes the patient's brain, and different parts include the left and right sides of the brain.

[0012] These and other advantages of the present invention will be apparent to those skilled in the art from the following detailed description and accompanying drawings. Attached Figure Description

[0013] Figure 1 A workflow for automatically detecting occlusions in one or more blood vessels in a patient's brain, according to one or more embodiments, is shown.

[0014] Figure 2 A method for detecting occlusion in one or more blood vessels of a patient anatomical object according to one or more embodiments is illustrated;

[0015] Figure 3 Performance tables of the embodiments described herein are shown during experimental verification;

[0016] Figure 4 An exemplary artificial neural network is shown that can be used to implement one or more embodiments;

[0017] Figure 5 A convolutional neural network that can be used to implement one or more embodiments is shown; and

[0018] Figure 6 A high-level block diagram of a computer that can be used to implement one or more embodiments is shown. Detailed Implementation

[0019] This invention generally relates to methods and systems for the detection and classification of large vessel occlusion (LVO) in medical images. Embodiments of the invention are described herein to provide an intuitive understanding of these methods and systems. Digital images typically consist of digital representations of one or more objects (or shapes). Here, the digital representation of an object is generally described based on the identification and manipulation of the object. This manipulation is a virtual operation performed in the memory or other circuitry / hardware of a computer system. Therefore, it should be understood that embodiments of the invention can be performed within a computer system using data stored within the computer system.

[0020] The embodiments described herein provide LVO detection and classification from input medical images of blood vessels in a patient's brain. In one embodiment, features are extracted from patches depicting the left side of the brain and patches depicting the right side of the brain, and occlusions in blood vessels are detected based on the extracted features. Advantageously, the embodiments described herein provide LVO detection and classification with enhanced robustness and performance in the presence of signal loss, noise, vascular tortuosity, calcification, and proximity to bone or bifurcation, compared to conventional methods. This enhanced robustness and performance is, for example, a result of patch extraction from the input medical image based on anatomical landmarks, allowing the extraction of absolute and differential distribution features for occlusion detection. Furthermore, the application of an output model based on a probability distribution function of a Gaussian process allows for increased confidence in the decision. Moreover, the embodiments described herein achieve increased automation and trust in the deployed system by reducing the number of errors in semantic image analysis. In one example, the embodiments described herein enable full automation of the entire stroke management workflow on a scanner or edge computer, leading to faster patient treatment and application to automated triage or intervention planning.

[0021] Figure 1 A workflow 100 for automatically detecting occlusions in one or more blood vessels in a patient's brain, according to one or more embodiments, is shown. Figure 2 A method 200 for detecting occlusion in one or more blood vessels of a patient anatomical object, according to one or more embodiments, is illustrated. These will be described together. Figure 1 and Figure 2 The steps of method 200 can be performed by one or more suitable computing devices (such as, for example...). Figure 6 The computer (602) is used to execute this.

[0022] exist Figure 2 Step 202 involves receiving input medical images of one or more blood vessels in a patient's anatomical object. In one embodiment, the anatomical object is the patient's brain. However, the anatomical object can be any other suitable anatomical object of interest to the patient, such as, for example, the lungs.

[0023] In one embodiment, the input medical image is a computed tomography (CT) image acquired during CT angiography (CTA). For example, such as Figure 1 As shown in workflow 100, the input medical image is a CTA volume 102. However, the input medical image can be any other suitable modality, such as, for example, magnetic resonance imaging (MRI), ultrasound (US), X-ray, or any other medical imaging modality or combination of medical imaging modalities. The input medical image can be a 2D (two-dimensional) image or a 3D (three-dimensional) volume, and can include a single input medical image or multiple input medical images. When acquiring the input medical image, it can be received directly from an image acquisition device (such as, for example, a CT scanner), or it can be received by loading previously acquired medical images from the storage or memory of a computer system or by receiving medical images transmitted from a remote computer system.

[0024] exist Figure 2 Step 204 involves removing bone from the input medical image. In one example, such as... Figure 1 As shown in workflow 100, bones are removed from CTA volume 102 to generate a bone-removed input medical image 104. In one embodiment, a machine learning-based bone removal network, such as, for example, a deep learning-based image-to-image (I2I) model, is used to remove bones from the input medical image. However, any other suitable method can be used to remove bones from the input medical image.

[0025] exist Figure 2 Step 206 involves identifying one or more anatomical landmarks in the input medical image. In one embodiment, the anatomical landmarks may be vascular landmarks of blood vessels. For example, such as... Figure 1 As shown in workflow 100, vascular landmarks 106 are identified in CTA volume 102. In one example, vascular landmarks include the bifurcation of the middle cerebral artery (MCA) or the basilar artery. Other exemplary vascular landmarks include the brachiocephalic artery, left common carotid artery, left subclavian artery, left subclavian artery and vertebral branch, right subclavian artery and vertebral branch, left vertebral artery (C3, C5), right vertebral artery (C3, C5), left carotid bifurcation, right carotid bifurcation, right carotid artery entering the skull, left carotid artery entering the skull, vertebral artery merging into the basilar artery, left intracranial, right intracranial, left carotid frontal, right carotid anterior, basilar artery branch, and carotid artery merging. However, anatomical landmarks can be any other suitable anatomical landmark in the input medical image.

[0026] In one embodiment, a machine learning-based detection network, such as a multi-scale reinforcement learning-based model, is used to identify anatomical landmarks. For example, anatomical landmarks can be identified by first generating a vascular tree of one or more vessels and then using a multi-scale reinforcement learning-based model to identify or index anatomical landmarks on the vascular tree. However, any other suitable method can be used to identify anatomical landmarks.

[0027] exist Figure 2 Step 208 involves extracting a first patch and one or more additional patches from the debonded input medical image, based on one or more identified anatomical landmarks. For example, as... Figure 1 As shown in workflow 100, a first patch 108 and a second patch 110 are extracted from an input medical image 104 after bone removal, based on vascular landmarks 106. The first patch and one or more additional patches depict different parts of the anatomical object. In one embodiment, where the anatomical object is a patient's brain, the first patch 108 depicts the left side of the brain, and the second patch 110 depicts the right side of the brain (and vice versa).

[0028] In one embodiment, a first patch and one or more additional patches are extracted from an input medical image that has been debonded by cropping, such that the patches are aligned with anatomical landmarks (e.g., centered on the anatomical landmarks). The first patch and one or more additional patches can have any suitable size. In one embodiment, the first patch and one or more additional patches have predetermined sizes.

[0029] exist Figure 2 Step 210 involves extracting features from a first patch and one or more additional patches using a machine learning-based feature extractor network. The machine learning-based feature extractor network receives the first patch and one or more additional patches as input and generates features as output. The first patch is received via a first input channel of the machine learning-based feature extractor network, and one or more additional patches are received via a corresponding one of the one or more additional input channels of the machine learning-based feature extractor network. The machine learning-based feature extractor network can be any suitable machine learning-based network, such as, for example, a network based on a convolutional deep neural network or a visual converter architecture.

[0030] In one example, such as Figure 1 As shown in workflow 100, the LVO feature extractor 114 uses the LVO detector and classifier 112 to extract features from the first patch 108 and the second patch 110. Figure 1As shown in the workflow 100, the LVO feature extractor 114 receives a first patch 108 as input and a second patch 110 as an independent input channel, and generates features as output. In one embodiment, the first input channel receives the first patch 108, and the second input channel receives the second patch 110. These features are latent features extracted from the first patch 108 and the second patch 110. These features include vascular density features specific to the patch received via the first input channel (e.g., the first patch 108 in this embodiment), and differential features comparing the first patch with each of one or more additional patches (e.g., comparing a first portion of an anatomical object with a second portion, respectively). The extraction of differential features is achieved by inputting the first patch 108 and the second patch 110 into the LVO feature extractor 114.

[0031] exist Figure 2 Step 212 involves detecting occlusions (e.g., LVOs) in one or more blood vessels within the first patch based on the extracted features. In one embodiment, an output model is trained to detect occlusions depicted in the patch received via a first input channel of the feature extractor network. In one embodiment, the output model is a probability distribution function (PDF) model fitted to features extracted by, for example, a neural network. The PDF model is learned using a Gaussian process model. However, the output model can be any other suitable model for detecting occlusions based on the extracted features. For example, the output model can be a machine learning-based classifier network, such as, for example, a deep learning-based multi-label classifier or a model based on other probability distributions (Gaussian mixture or kernel density estimation).

[0032] In one example, such as Figure 1 As shown in the workflow 100, the Gaussian process PDF model 116 of the LVO detector and classifier 112 receives features extracted by the LVO feature extractor 114 as input and generates a result 118 as output. Result 118 is the result of occlusion detection in the patch (e.g., the first patch 108 in this embodiment) received via the first input channel of the LVO feature extractor 114. In one example, as... Figure 1As shown in workflow 100, result 118 includes a numerical value (e.g., between 0 and 1) indicating LVO confidence (representing the confidence of the LVO in the blood vessel depicted in the patch received via the first input channel), and confidence that the LVO is part of a specific part of the blood vessel, such as, for example, the internal carotid artery (ICA), the middle cerebral artery (MCA) at the M1 level, and the MCA at the M2 level. Confidences for other blood vessels (e.g., corresponding to the anterior or posterior circulation) may also be included. These values ​​can be compared to one or more thresholds (e.g., 0.5) to generate a classification (e.g., a binary classification of yes or no). In one embodiment, Gaussian process PDF model 116 is trained on a feature distribution extracted by LVO feature extractor 114, which allows for consideration of uncertainties in the input data or model, or for additional adjustments to the model in therapeutic classification applications.

[0033] In one embodiment, where one or more identified anatomical landmarks include the MCA bifurcation and the first patch is cut centered on the MCA bifurcation, the occlusion is detected on one of the internal carotid artery (ICA), the MCA M1 segment (i.e., the sphenoid segment), or the MCA M2 segment (i.e., the insular segment). A similar procedure can be applied to the basilar artery.

[0034] exist Figure 2 Step 214: Output the detection results. For example, the results can be output by displaying them on a display device of the computer system, storing them in the computer system's memory or storage, or transmitting them to a remote computer system. In one embodiment, when deployed on a scanner (e.g., a CT scanner) or a standalone or cloud processing system, the detection results can be used for work list prioritization or automatic notification, or for automated intervention planning, such as, for example, for mechanical thrombectomy.

[0035] In one embodiment, in Figure 2 In step 210, the feature extractor network additionally or alternatively receives as input at least one probability map of the presence of blood vessels in at least one of the first patch or one or more additional patches, and extracts features from the probability map. A vascularity classifier or filter can be used to generate the probability map, which receives the first patch and / or one or more additional patches as input and generates a corresponding probability map as output.

[0036] In one embodiment, in Figure 2 Step 208 may involve extracting a first patch and one or more additional patches from the input medical image in different regions of the vascular tree, such as the posterior circulation (focusing on the basilar artery) or at the carotid bifurcation to the external carotid artery.

[0037] The embodiments described herein were experimentally validated on a database of 2,647 CTA volumes. Figure 3 Table 300 shows the performance of the embodiments described herein during experimental validation. As shown in Table 300, for the hold-out test group including 60 patients, the experiment resulted in a total sensitivity (Sens), a specificity (Spec), and an area under the curve (AUC) of 0.96. Table 300 also shows the sensitivity, specificity, and AUC for detecting LVO in ICA, MCA M1, and MCA M2.

[0038] The embodiments described herein relate to a claimed system and a claimed method. The features, advantages, or alternative embodiments described herein can be assigned to other claims, and vice versa. In other words, the system claims can be modified using features described or claimed in the context of the method. In this case, the functional characteristics of the method are embodied by the target unit providing the system.

[0039] Furthermore, certain embodiments described herein relate to methods and systems for utilizing trained machine learning-based networks (or models), and to methods and systems for training machine learning-based networks. The features, advantages, or alternative embodiments described herein can be assigned to other claimed objects, and vice versa. In other words, the claims to methods and systems for training machine learning-based networks can be modified with features described or claimed in the context of methods and systems for utilizing trained machine learning-based networks, and vice versa.

[0040] Specifically, the trained machine learning-based networks used in the embodiments described herein can be adapted by methods and systems for training machine learning-based networks. Furthermore, the input data of the trained machine learning-based network can include advantageous features and embodiments of the training input data, and vice versa. Similarly, the output data of the trained machine learning network can include advantageous features and embodiments of the output training data, and vice versa.

[0041] Generally, trained machine learning-based networks mimic human cognitive functions associated with other human thought processes. Specifically, through training on training data, trained machine learning-based networks can adapt to new environments and detect and infer patterns.

[0042] Generally, the parameters of machine learning-based networks can be adapted through training. Specifically, supervised training, semi-supervised training, unsupervised training, reinforcement learning, and / or active learning can be used. Additionally, representation learning (another alternative term is "feature learning") can be used. In essence, the parameters of a trained machine learning-based network can be iteratively adapted through several training steps.

[0043] Specifically, the trained machine learning-based network can include neural networks, support vector machines, decision trees, and / or Bayesian networks, and / or the trained machine learning-based network can be based on k-means clustering, Q-learning, genetic algorithms, and / or association rules. Specifically, the neural network can be a deep neural network, a convolutional neural network, or a convolutional deep neural network. Furthermore, the neural network can be an adversarial network, a deep adversarial network, and / or a generative adversarial network.

[0044] Figure 4 An embodiment of an artificial neural network 400 according to one or more embodiments is shown. Alternative terms for "artificial neural network" are "neural network," "artificial neural network," or "neural network." The artificial neural network 400 can be used to implement the machine learning networks described herein, such as, for example... Figure 1 LVO feature extractor 114, in Figure 2 Step 204 uses a machine learning-based bone removal network, in Figure 2 Step 206 uses a machine learning-based detection network, in Figure 2 Step 210 uses a machine learning-based feature extractor network, in Figure 2 Step 212 uses a machine learning-based classifier network.

[0045] The artificial neural network 400 includes nodes 402-422 and edges 432, 434, ..., 436, where each edge 432, 434, ..., 436 is a directed connection from a first node 402-422 to a second node 402-422. Typically, the first node 402-422 and the second node 402-422 are different nodes, but it is also possible that the first node 402-422 and the second node 402-422 are the same. For example, in... Figure 4 In the diagram, edge 432 is a directed connection from node 402 to node 406, while edge 434 is a directed connection from node 404 to node 406. Edges 432, 434, ..., 436 from the first node 402-422 to the second node 402-422 are also represented as "input edges" for the second node 402-422 and as "output edges" for the first node 402-422.

[0046] In this embodiment, nodes 402-422 of the artificial neural network 400 can be arranged in layers 424-430, wherein these layers can include an inherent order introduced by edges 432, 434, ..., 436 between nodes 402-422. Specifically, edges 432, 434, ..., 436 can only exist between adjacent node layers. Figure 4 In the illustrated embodiment, there is an input layer 424, which includes only nodes 402 and 404 and has no input edges; an output layer 430, which includes only node 422 and has no output edges; and hidden layers 426 and 428 between the input layer 424 and the output layer 430. Typically, the number of hidden layers 426 and 428 can be arbitrarily chosen. The number of nodes 402 and 404 in the input layer 424 is typically related to the number of input values ​​of the neural network 400, and the number of nodes 422 in the output layer 430 is typically related to the number of output values ​​of the neural network 400.

[0047] Specifically, (real) numbers can be assigned as values ​​to each node 402-422 of the neural network 400. Here, x (n) i This represents the value of the i-th node 402-422 in the n-th layer 424-430. The values ​​of nodes 402-422 in the input layer 424 are equivalent to the input values ​​of neural network 400, and the value of node 422 in the output layer 430 is equivalent to the output value of neural network 400. Furthermore, each edge 432, 434, ..., 436 can include a weight as a real number, specifically a real number within the interval [-1, 1] or the interval [0, 1]. Here, w... (m，n) i，j This represents the weight of the edge between the i-th node (402-422) of layer m (424-430) and the j-th node (402-422) of layer n (424-430). Additionally, the abbreviation w... (n) i，j Defined as weight w (n，n+1) i，j .

[0048] Specifically, to calculate the output value of neural network 400, the input value is propagated through the neural network. Specifically, the values ​​of nodes 402-422 in layer (n+1) 424-430 can be calculated based on the values ​​of nodes 402-422 in layer n 424-430 using the following formula.

[0049]

[0050] Here, the function f is the transfer function (another term is the "activation function"). Known transfer functions are step functions, sigmoid functions (e.g., logic functions, generalized logic functions, hyperbolic tangent functions, arctangent functions, error functions, smooth step functions), or rectifier functions. Transfer functions are primarily used for normalization purposes.

[0051] Specifically, these values ​​are propagated layer by layer through the neural network, where the value of the input layer 424 is given by the input of the neural network 400, the value of the first hidden layer 426 can be calculated based on the value of the input layer 424 of the neural network, the value of the second hidden layer 428 can be calculated based on the value of the first hidden layer 426, and so on.

[0052] To set the value w for the edge (m，n) i，j Training data is required to train the neural network 400. Specifically, the training data includes training input data and training output data (denoted as t). i For the training step, neural network 400 is applied to the training input data to generate computational output data. Specifically, the training data and the computational output data include multiple values, the number of which is equal to the number of nodes in the output layer.

[0053] Specifically, the comparison between the calculated output data and the training data is used to recursively adapt the weights within the neural network (backpropagation algorithm). Specifically, the weights vary according to the following:

[0054]

[0055] Where γ is the learning rate, and δ (n) j It can be recursively calculated as

[0056]

[0057] Based on δ (n+1) j If the (n+1)th layer is not an output layer, and

[0058]

[0059] If the (n+1)th layer is the output layer 430, where f' is the first derivative of the activation function, y (n+1) j It is the comparison training value used for the j-th node of the output layer 430.

[0060] Figure 5 A convolutional neural network 500 according to one or more embodiments is illustrated. The convolutional neural network 500 can be used to implement the machine learning networks described herein, such as, for example... Figure 1 LVO feature extractor 114, in Figure 2 Step 204 uses a machine learning-based bone removal network, in Figure 2 Step 206 uses a machine learning-based detection network, in Figure 2 Step 210 uses a machine learning-based feature extractor network, in Figure 2 Step 212 uses a machine learning-based classifier network.

[0061] exist Figure 5 In the illustrated embodiment, the convolutional neural network 500 includes an input layer 502, convolutional layers 504, pooling layers 506, fully connected layers 508, and an output layer 510. Alternatively, the convolutional neural network 500 may include several convolutional layers 504, several pooling layers 506, and several fully connected layers 508, as well as other types of layers. The order of the layers can be arbitrarily chosen; typically, the fully connected layer 508 is used as the last layer before the output layer 510.

[0062] Specifically, within a convolutional neural network 500, nodes 512-520 of layer 502-510 can be considered as arranged as a d-dimensional matrix or as a d-dimensional image. Specifically, in the two-dimensional case, the values ​​of nodes 512-520 indexed by i and j in the nth layer 502-510 can be represented as x. (n) [i，j] However, the arrangement of nodes 512-520 in layer 502-510 has no effect on the computation performed within the convolutional neural network 500 in a strict sense, because these are given only by the structure and weights of the edges.

[0063] Specifically, convolutional layer 504 is characterized by the structure and weights of the input edges, and it performs convolution operations based on a specific number of kernels. Specifically, the structure and weights of the input edges are chosen such that the value x of node 514 of convolutional layer 504... (n) k Based on the value x of node 512 of the previous layer 502. (n-1) Calculated as convolution x (n) k =K k *x (n-1) Where convolution* is defined in the two-dimensional case as

[0064]

[0065] Here, the k-th kernel Kk is a d-dimensional matrix (a two-dimensional matrix in this embodiment), which is typically small compared to the number of nodes 512-518 (e.g., a 3×3 or 5×5 matrix). Specifically, this means that the weights of the input edges are not independent, but are chosen such that they produce the convolution equation. Specifically, for a kernel that is a 3×3 matrix, there are only 9 independent weights (each entry of the kernel matrix corresponds to an independent weight), regardless of the number of nodes 512-520 in the corresponding layers 502-510. Specifically, for convolutional layer 504, the number of nodes 514 in the convolutional layer is equal to the number of nodes 512 in the previous layer 502 multiplied by the number of kernels.

[0066] If the nodes 512 of the previous layer 502 are arranged as a d-dimensional matrix, then using multiple kernels can be interpreted as adding another dimension (denoted as the "depth" dimension), so that the nodes 514 of the convolutional layer 504 are arranged as a (d+1)-dimensional matrix. If the nodes 512 of the previous layer 502 have already been arranged as a (d+1)-dimensional matrix including the depth dimension, then using multiple kernels can be interpreted as extending along the depth dimension, so that the nodes 514 of the convolutional layer 504 are also arranged as a (d+1)-dimensional matrix, where the size of the (d+1)-dimensional matrix relative to the depth dimension is a multiple of the number of kernels in the previous layer 502.

[0067] The advantage of using convolutional layer 504 is that it can take advantage of the spatial local correlation of the input data by implementing local connection patterns between nodes of adjacent layers, in particular by connecting each node to only a small region of the node of the previous layer.

[0068] exist Figure 5 In the illustrated embodiment, the input layer 502 comprises 36 nodes 512 arranged in a two-dimensional 6×6 matrix. The convolutional layer 504 comprises 72 nodes 514 arranged in two two-dimensional 6×6 matrices, each of which is the result of the convolution of the input layer values ​​with the kernel. Equivalently, the nodes 514 of the convolutional layer 504 can be interpreted as arranged in a three-dimensional 6×6×2 matrix, where the last dimension is the depth dimension.

[0069] The pooling layer 506 can be characterized by the structure and weights of the input edges and the activation functions of its nodes 516, which form a pooling operation based on a nonlinear pooling function f. For example, in the two-dimensional case, the value x of node 516 of the pooling layer 506 is... (n) It can be based on the value x of node 514 of the previous layer 504. (n-1) The calculation is as follows:

[0070] x (n) [i, j] = f(x) (n-1) [id1, jd2], ..., x (n-1)[id1+d1-1,jd2+d2-1]).

[0071] In other words, by using the pooling layer 506, the number of nodes 514 and 516 can be reduced. This is achieved by replacing the number of neighboring nodes d1·d2 in the previous layer 504 with a single node 516, calculated as a function of the number of neighboring nodes in the pooling layer. Specifically, the pooling function f can be a maximum function, an average function, or an L2 norm function. Specifically, for the pooling layer 506, the weights of the input edges are fixed and are not modified through training.

[0072] The advantage of using pooling layer 506 is that it reduces the number of nodes 514 and 516 and the number of parameters. This results in a reduction in the computational cost of the network and controls overfitting.

[0073] exist Figure 5 In the illustrated embodiment, pooling layer 506 is a maximum pool, which replaces four adjacent nodes with only one node, the value of which is the maximum of the four adjacent nodes. Maximum pooling is applied to each d-dimensional matrix of the previous layer; in this embodiment, maximum pooling is applied to each of the two two-dimensional matrices, reducing the number of nodes from 72 to 18.

[0074] The fully connected layer 508 is characterized by the fact that most, in particular all, of the edges exist between node 516 of the previous layer 506 and node 518 of the fully connected layer 508, and the weight of each edge can be adjusted individually.

[0075] In this embodiment, the nodes 516 of the preceding layer 506 of the fully connected layer 508 are displayed as a two-dimensional matrix and are also displayed as unrelated nodes (indicated as a row of nodes, where the number of nodes is reduced for better presentation). In this embodiment, the number of nodes 518 in the fully connected layer 508 is equal to the number of nodes 516 in the preceding layer 506. Alternatively, the number of nodes 516 and 518 can be different.

[0076] Furthermore, in this embodiment, the value of node 520 in output layer 510 is determined by applying the Softmax function to the value of node 518 in the previous layer 508. By applying the Softmax function, the sum of the values ​​of all nodes 520 in output layer 510 is 1, and all values ​​of all nodes 520 in the output layer are real numbers between 0 and 1.

[0077] The convolutional neural network 500 may also include ReLU (Rectified Linear Unit) layers or activation layers with non-linear transfer functions. Specifically, the number and structure of nodes in the ReLU layer are the same as those in the previous layer. Specifically, the value of each node in the ReLU layer is calculated by applying a rectified function to the value of the corresponding node in the previous layer.

[0078] The inputs and outputs of different convolutional neural network blocks can be connected using summation (residual / dense neural networks), element-wise multiplication (note), or other differentiable operators. Therefore, if the entire pipeline is differentiable, the convolutional neural network architecture can be nested rather than sequential.

[0079] Specifically, a convolutional neural network 500 can be trained based on the backpropagation algorithm. To prevent overfitting, regularization methods can be used, such as dropping nodes 512-520, random pooling, the use of artificial data, and weight decay based on L1 or L2 norm or maximum norm constraints. Different loss functions can be combined to train the same neural network to reflect the joint training objective. A subset of neural network parameters can be excluded from the optimization to retain weights pre-trained on another dataset.

[0080] The systems, apparatus, and methods described herein can be implemented using digital circuitry or using one or more computers employing known computer processors, memory units, storage devices, computer software, and other components. Typically, a computer includes a processor for executing instructions and one or more memories for storing instructions and data. A computer may also include or be coupled to one or more mass storage devices, such as one or more disks, internal hard disks and removable disks, magneto-optical disks, optical disks, etc.

[0081] The systems, apparatus, and methods described herein can be implemented using computers operating in a client-server relationship. Typically, in such a system, the client computer is located remotely from the server computer and interacts via a network. The client-server relationship can be defined and controlled by computer programs running on the respective client and server computers.

[0082] The systems, apparatus, and methods described herein can be implemented in a network-based cloud computing system. In such a system, a server or other processor connected to the network communicates with one or more client computers via the network. For example, a client computer may communicate with the server via a web browser application residing on and running on the client computer. The client computer may store data on the server and access the data via the network. The client computer may transmit data requests or online service requests to the server via the network. The server may perform the requested service and provide data to one or more client computers(s). The server may also transmit data suitable for enabling the client computer to perform specific functions, such as performing calculations, displaying specific data on a screen, etc. For example, the server may transmit one or more steps or functions suitable for enabling the client computer to perform the methods and workflows described herein (including...). Figure 1 A request for one or more steps or functions of the methods and workflows described herein. Certain steps or functions of the methods and workflows described herein (including...) Figure 1 One or more steps or functions of method 2 may be performed by a server or another processor in a web-based cloud computing system. Certain steps or functions of the methods and workflows described herein (including...) Figure 1 (or one or more steps of step 2) can be performed by a client computer in a web-based cloud computing system. The steps or functions of the methods and workflows described herein (including...) Figure 1 (or one or more steps of step 2) can be performed by servers and / or client computers in a web-based cloud computing system in any combination.

[0083] The systems, apparatus, and methods described herein can be implemented using a computer program product tangibly embodied in an information carrier, such as in a non-transitory machine-readable storage device, for execution by a programmable processor; and the methods and workflow steps described herein (including...) Figure 1 (or one or more steps or functions of 2) can be implemented using one or more computer programs executable by such a processor. A computer program is a set of computer program instructions that can be used directly or indirectly in a computer to perform a specific activity or produce a specific result. Computer programs can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0084] Figure 6A high-level block diagram of an example computer 602, which can be used to implement the systems, apparatus, and methods described herein, is depicted. Computer 602 includes a processor 604 operatively coupled to a data storage device 612 and a memory 610. Processor 604 controls the overall operation of computer 602 by executing computer program instructions that define these operations. The computer program instructions may be stored in the data storage device 612 or other computer-readable medium and loaded into memory 610 when execution is required. Therefore, Figure 1 The methods and workflow steps or functions of option 2 can be defined by computer program instructions stored in memory 610 and / or data storage device 612, and controlled by processor 604 that executes the computer program instructions. For example, the computer program instructions can be implemented as computer-executable code programmed by those skilled in the art to perform... Figure 1 Alternatively, the method and workflow steps or functions of step 2. Therefore, by executing computer program instructions, processor 604 performs... Figure 1 Alternatively, methods and workflow steps or functions may be included. Computer 602 may also include one or more network interfaces 606 for communicating with other devices via a network. Computer 602 may also include one or more input / output devices 608 that enable users to interact with computer 602 (e.g., monitor, keyboard, mouse, speakers, buttons, etc.).

[0085] Processor 604 may include general-purpose and special-purpose microprocessors and may be the sole processor or one of multiple processors in computer 602. For example, processor 604 may include one or more central processing units (CPUs). Processor 604, data storage device 612, and / or memory 610 may include one or more application-specific integrated circuits (ASICs) and / or one or more field-programmable gate arrays (FPGAs), supplemented or incorporated therein.

[0086] Each of the data storage device 612 and the memory 610 includes a tangible, non-transitory computer-readable storage medium. The data storage device 612 and the memory 610 may each include high-speed random access memory, such as dynamic random access memory (DRAM), static random access memory (SRAM), double data rate synchronous dynamic random access memory (DDR RAM), or other random access solid-state storage devices, and may include non-volatile memory, such as one or more disk storage devices, such as internal hard disks and removable disks, magneto-optical disk storage devices, optical disk storage devices, flash memory devices, semiconductor storage devices, such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), optical disc read-only memory (CD-ROM), digital universal optical disc read-only memory (DVD-ROM), or other non-volatile solid-state storage devices.

[0087] Input / output device 608 may include peripheral devices such as printers, scanners, displays, etc. For example, input / output device 608 may include display devices such as cathode ray tube (CRT) or liquid crystal display (LCD) monitors for displaying information to a user, keyboards, and pointing devices such as mice or trackballs, through which the user can provide input to computer 602.

[0088] Image acquisition device 614 can be connected to computer 602 to input image data (e.g., medical images) into computer 602. It is possible to implement image acquisition device 614 and computer 602 as a single device. Image acquisition device 614 and computer 602 can also communicate wirelessly via a network. In a possible embodiment, computer 602 may be located remotely relative to image acquisition device 614.

[0089] Any or all systems and devices discussed herein can be implemented using one or more computers, such as computer 602.

[0090] Those skilled in the art will recognize that actual computer or computer system implementations may have other structures and may include other components, and Figure 6 It is a high-level representation of some components of this computer for illustrative purposes.

[0091] The foregoing detailed description should be understood as illustrative and exemplary in every respect, not restrictive, and the scope of the invention disclosed herein is not determined by the specific embodiments, but by the claims interpreted in the full breadth permitted by patent law. It should be understood that the embodiments shown and described herein are merely illustrative of the principles of the invention, and various modifications can be made by those skilled in the art without departing from the scope and spirit of the invention. Various other combinations of features can be implemented by those skilled in the art without departing from the scope and spirit of the invention.

Claims

1. A computer-implemented method, comprising: Receive input medical images of one or more blood vessels from a patient's anatomical object; Identify one or more anatomical landmarks in an input medical image; Based on one or more identified anatomical landmarks, a first patch and one or more additional patches are extracted from the input medical image. The first patch and one or more additional patches depict different parts of the anatomical object. A machine learning-based feature extractor network is used to extract features from the first patch and the one or more additional patches. The machine learning-based feature extractor network receives the first patch and the one or more additional patches as input and generates extracted features as output. The extracted features include 1) differential features that compare the first patch with each of the one or more additional patches, and 2) vascular density features specific to the first patch. Based on the extracted features, occlusions in one or more vessels within the first patch are detected; and Output the detection results.

2. The computer-implemented method of claim 1, further comprising removing bone from the input medical image, wherein extracting a first patch and one or more additional patches from the input medical image based on one or more identified anatomical landmarks comprises: Extract the first patch and one or more additional patches from the input medical image after removing the bone.

3. The computer-implemented method of claim 1, wherein extracting features from the first patch and the one or more additional patches using a machine learning-based feature extractor network comprises: A first patch is received via a first input channel of a machine learning-based feature extractor network, and one or more additional patches are received via a corresponding one of one or more additional input channels of the machine learning-based feature extractor network.

4. The computer-implemented method of claim 1, wherein extracting a first patch and one or more additional patches from the input medical image based on one or more identified anatomical landmarks comprises: Cut the first patch centered on one or more anatomical landmarks and one or more additional patches.

5. The computer-implemented method according to claim 1, wherein detecting occlusion in one or more blood vessels in the first patch based on extracted features comprises: Occlusions in one or more vessels in the first patch are detected using a probability distribution function (PDF) model fitted to features extracted by a neural network.

6. The computer-implemented method of claim 5, wherein, Use a Gaussian process model to learn the PDF model.

7. The computer-implemented method according to claim 1, wherein: Identifying one or more anatomical landmarks in an input medical image includes identifying the bifurcation of the middle cerebral artery (MCA) in the input medical image; Based on one or more identified anatomical landmarks, a first patch and one or more additional patches are extracted from the input medical image, including cropping the first patch centered at the MCA bifurcation; and Detecting occlusions in one or more vessels in the first patch based on the extracted features includes detecting occlusions as located in one of the internal carotid artery (ICA), MCA M1 segment, or MCA M2 segment.

8. The computer-implemented method of claim 1, further comprising generating at least one probability map of the presence of blood vessels for at least one of the first patch or the one or more additional patches, wherein extracting features from the first patch and the one or more additional patches using a machine learning-based feature extractor network comprises: Use a machine learning-based feature extractor network to extract features from at least one probabilistic map.

9. The computer-implemented method of claim 1, wherein, The anatomical object includes the patient's brain, and the different parts include the left and right sides of the brain.

10. A device for detecting occlusion in one or more blood vessels of a patient anatomical object, comprising: A component for receiving input medical images of one or more blood vessels in a patient's anatomical object; Components used to identify one or more anatomical landmarks in an input medical image; Components used to extract a first patch and one or more additional patches from an input medical image based on one or more identified anatomical landmarks, the first patch and one or more additional patches depicting different parts of an anatomical object; A component for extracting features from a first patch and one or more additional patches using a machine learning-based feature extractor network. The machine learning-based feature extractor network receives the first patch and one or more additional patches as input and generates extracted features as output. The extracted features include 1) differential features comparing the first patch with each of the one or more additional patches, and 2) vascular density features specific to the first patch. Components for detecting occlusions in one or more vessels within a first patch based on extracted features; and A component used to output detection results.

11. The apparatus of claim 10, further comprising components for removing bone from the input medical image, and wherein extracting a first patch and one or more additional patches from the input medical image based on one or more identified anatomical landmarks comprises: A component used to extract a first patch and one or more additional patches from an input medical image that has had its bone removed.

12. The apparatus of claim 10, wherein the component for extracting features from the first patch and the one or more additional patches using a machine learning-based feature extractor network comprises: A component for receiving a first patch via a first input channel of a machine learning-based feature extractor network and receiving one or more additional patches via one of one or more additional input channels of the machine learning-based feature extractor network.

13. The apparatus of claim 10, wherein the component for extracting a first patch and one or more additional patches from an input medical image based on one or more identified anatomical landmarks comprises: Components used for cutting a first patch centered on one or more anatomical landmarks and one or more additional patches.

14. The apparatus of claim 10, wherein, The anatomical object includes the patient's brain, and the different parts include the left and right sides of the brain.

15. A non-transitory computer-readable medium storing computer program instructions, which, when executed by a processor, cause the processor to perform operations including: Receive input medical images of one or more blood vessels in a patient's anatomical object; Identify one or more anatomical landmarks in an input medical image; Based on one or more identified anatomical landmarks, a first patch and one or more additional patches are extracted from the input medical image. The first patch and one or more additional patches depict different parts of the anatomical object. A machine learning-based feature extractor network is used to extract features from the first patch and the one or more additional patches. The machine learning-based feature extractor network receives the first patch and the one or more additional patches as input and generates extracted features as output. The extracted features include 1) differential features that compare the first patch with each of the one or more additional patches, and 2) vascular density features specific to the first patch. Based on the extracted features, occlusions in one or more vessels within the first patch are detected; and Output the detection results.

16. The non-transitory computer-readable medium of claim 15, wherein detecting occlusion in one or more vessels in the first patch based on the extracted features comprises: Occlusions in one or more vessels in the first patch are detected using a probability distribution function (PDF) model fitted to features extracted by a neural network.

17. The non-transitory computer-readable medium of claim 16, wherein a Gaussian process model is used to learn the PDF model.

18. The non-transitory computer-readable medium according to claim 15, wherein: Identifying one or more anatomical landmarks in an input medical image includes identifying the bifurcation of the middle cerebral artery (MCA) in the input medical image; Based on one or more identified anatomical landmarks, a first patch and one or more additional patches are extracted from the input medical image, including cropping the first patch centered at the MCA bifurcation; and Detecting occlusions in one or more vessels in the first patch based on the extracted features includes detecting occlusions as located in one of the internal carotid artery (ICA), MCA M1 segment, or MCA M2 segment.

19. The non-transitory computer-readable medium of claim 15, further comprising generating at least one probability map of the presence of blood vessels for at least one of the first patch or the one or more additional patches, and wherein extracting features from the first patch and the one or more additional patches using a machine learning-based feature extractor network comprises: Use a machine learning-based feature extractor network to extract features from at least one probabilistic map.

20. The non-transitory computer-readable medium of claim 15, wherein, The anatomical object includes the patient's brain, and the different parts include the left and right sides of the brain.

Images