Determining values for physical properties of thrombus
By analyzing the stent expansion rate in the X-ray image data, the physical properties of the thrombus are determined in real time, and the failure problem of stent retrieval in handling organized thrombus is solved, which improves the success rate and safety of thrombectomy.
Patent Information
- Application Number
- CN202380031753.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-07-14
- Filing Date
- 2023-03-14
- Publication Date
- 2024-11-19
AI Technical Summary
In mechanical thrombectomy, especially stent retrieval, the risk of failure in handling tissue thrombosis is high, resulting in loss of thrombosis or reocclusion, and the prior art is difficult to determine the physical properties of thrombosis in real time, affecting the treatment effect.
By receiving X-ray image data, the rate of expansion of the stent on the thrombus is analyzed, and the physical properties of the thrombus, such as size and composition are determined, thereby guiding the physician's actions during the retrieval process.
It improves the success rate of stent retrieval, reduces the risk of thrombosis loss and reocclusion, and provides real-time information on the physical properties of thrombosis to support physician operation.
Smart Images

Figure CN118984679A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to determining a value of a physical property of a thrombus. A computer-implemented method, computer program product, and system are disclosed. Background Art
[0002] A thrombus or clot is a blockage in a blood vessel. A thrombus may occur in a vein or an artery. In the former case, venous thrombosis, the blood becomes congested, causing swelling and fluid congestion. In the latter case, arterial thrombosis, the blood supply is restricted, leading to a condition called ischemia, which risks causing damage to the tissues supplied by the artery. In both cases, part of the thrombus may also break away as an embolus. The embolus may lodge elsewhere in the body and form an embolism that also blocks a blood vessel. Thrombosis may occur in various parts of the body, including in the heart and brain, where their effects can be serious unless treated quickly. For example, in the brain, a thrombus or embolism can lead to conditions such as (ischemic) stroke.
[0003] Various intraluminal disposal devices can be used to dispose of thrombi. These include mechanical thrombectomy devices, and they are used to remove thrombi from blood vessels. Currently, there are two main groups of mechanical thrombectomy devices: suction catheters and stent retrievers. The suction catheter typically includes a delivery catheter for delivering a flushing fluid to the thrombus, and an extraction catheter for extracting the flushing fluid and the broken pieces of the thrombus. In use, the suction catheter is positioned close to the clot, and suction is performed at this position, causing the fragments of the clot to be removed from the body. The stent retriever typically includes an expandable wire mesh tube, i.e., a stent, designed to integrally remove the clot. Before its deployment, the wire mesh tube is housed in the delivery catheter. The delivery catheter is positioned so that it overlaps the clot, and then the delivery catheter is withdrawn so that the wire mesh tube expands out from the distal end of the delivery catheter. As the wire mesh tube expands, it captures the clot. The wire mesh tube is then withdrawn into the delivery catheter and removed from the body.
[0004] Stent retriever "SR" thrombectomy is currently the first-line treatment for acute stroke due to intracranial large vessel occlusion "LVO" in the forebrain circulation. However, the stent retriever thrombectomy procedure may fail for various reasons. These include anatomical challenges (e.g., a tortuous arterial tree from the aortic arch to the target occlusion site), large clots, tandem occlusions, clot characteristics (e.g., fresh vs. organized clots), different pathological mechanisms (embolic vs. non-embolic occlusions), etc. The pathological mechanism of the acute LVO or the physical properties of the clot (soft vs. hard / organized) can play a key role in the response to the stent retriever. An organized (hard, fibrin-rich) clot is more elastic and less sticky than a fresh (soft, erythrocyte-rich) clot, which results in less engagement with the stent retriever and may result in loss of part or all of the clot during retrieval by the stent retriever, especially in the case of a tortuous arterial tree.
[0005] In addition, as described, for example, in Choi Dae-Han's article "Gradual expansion of stent retriever mechanical thrombectomy for curved middle cerebral artery..." (Acta Neurochirurgica (2019) 161: 2003-2012), organized clots can cause tension in the stent retriever deployment section of the parent artery, which can cause arterial spasm. This effect increases the likelihood of stent retriever failure.
[0006] For successful recanalization of LVO due to organized clots, the first and simplest option is the intra-arterial administration of vasodilators, which release the tension of the arterial segment where the stent retriever is deployed, thereby increasing the vessel diameter to facilitate clot retrieval. The second option is the intra-arterial administration of a clot dissolving agent (tissue plasminogen activator (tPA)) to soften the clot before removal. Another option is to utilize a stent retriever and aspiration or an intermediate catheter simultaneously. The stent retriever and thrombus aspiration can be performed sequentially or simultaneously. This helps prevent the engaged clot from being lost when the stent retriever is retracted through the tortuosity or narrow segments of the arterial tree. If an organized clot still occludes the vessel after the simultaneous use of a stent retriever and an intermediate catheter, permanent stent placement can be considered a final measure.
[0007] Another factor that increases the risk of stent retriever failure is the damage of the stent to the atheromatous surface if the acute LVO is due to thrombotic occlusion of an intracranial atherosclerotic stenosis. In this case, the use of a stent retriever can lead to further platelet activation, leading to repeated reocclusion in the future. In order to prevent repeated reocclusion in the case of LVO due to ICA, inhibition of platelet function can play a key role. Therefore, the first option is to administer a glycoprotein IIb / IIIa inhibitor that inactivates platelets and thereby prevents repeated reocclusion, as discussed in Kim, BM, "Causes and Solutions of Endovascular Treatment Failure." (J. stroke, vol. 19, no. 2, pp. 131–142, 2017).
[0008] The current workflow in the stent retriever thrombectomy process is to retrieve the stent between three and eight minutes after stent deployment. However, for hard / organized clots, the chance of successfully retrieving the clot in the first pass is lower than that of soft clots. Some physicians may perform multiple rounds of stent retriever thrombectomy to remove the entire clot. Several alternative workflows (such as those discussed above) have also been proposed for handling organized clots. However, a correlation between a stent retriever process completed in a single pass and positive outcomes has been reported in Zaidat, OO et al., "First pass effect: A new measure for stroke thrombectomy devices" (Stroke, vol. 49, no. 3, pp. 660–666, 2018). It has also been reported in the literature by Ng, PP et al., “Intraprocedural predictors of post-stent retriever thrombectomy subarachnoid hemorrhage in middlecerebral artery stroke” (J. Neurointerv. Surg., vol. 11, no. 2, pp. 123–126, 2019) that more than two rounds of SR may increase the risk of post-procedural subarachnoid hemorrhage “SAH”. The same study showed that the risk of post-procedural SAH was higher in patients whose stents were located more than 2 cm distal to the thrombus in their middle cerebral artery.
[0009] Thus, it can be appreciated from the above discussion that removing a thrombus by mechanical thrombectomy is a challenging procedure and that improved support for physicians is needed in this procedure. Summary of the invention
[0010] According to one aspect of the present disclosure, there is provided a computer-implemented method for determining a value of a physical property of a thrombus. The method comprises:
[0011] receiving X-ray image data comprising a time series of X-ray images representing expansion of a stent of an intraluminal stent retriever device over a thrombus;
[0012] determining the value of the physical property of the thrombus based on a rate of expansion of the stent in the time series of X-ray images; and
[0013] The value of the physical property is output.
[0014] The inventors have observed that the rate of expansion of a stent on a thrombus is indicative of various physical properties of the thrombus, including the size of the thrombus (e.g., its length, diameter, and volume) and the composition of the thrombus (e.g., soft versus hard or organized). These physical properties influence the physician's choice of how to extract the thrombus. Thus, by using the above method, the physical properties of the thrombus can be determined during a stent retriever procedure, and thus used by the physician to determine the process of action when extracting the thrombus. Thus, the physician can be better supported in performing such a procedure or intervention.
[0015] Other aspects, features and advantages of the present disclosure will become apparent from the following description of examples with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 is a flow chart illustrating an example of a method of determining a value of a physical property of a thrombus according to some aspects of the present disclosure.
[0017] Figure 2 is a schematic diagram illustrating an example of a system 200 for determining a value of a physical property of a thrombus according to some aspects of the present disclosure.
[0018] Figure 3 is a schematic diagram illustrating an example of various stages of expansion of a stent 130 of an intraluminal stent retriever device on a thrombus 110 according to some aspects of the present disclosure.
[0019] Figure 4 is a diagram illustrating an example of A) training a neural network 140 to generate a latent space encoding of a time series of x-ray images and B) performing inference using the neural network 140 trained to generate a latent space encoding of a time series of x-ray images 120 in accordance with some aspects of the present disclosure.
[0020] Figure 5is a diagram illustrating an example of A) training a neural network 140 to generate a latent space encoding of a time series of x-ray images and B) performing inference using the neural network 140 trained to generate a latent space encoding of a time series of x-ray images 120 and wherein the training data also includes workflow data in accordance with some aspects of the present disclosure.
[0021] Figure 6 is a diagram illustrating an example of A) training a neural network 140 to predict the shape of a stent from a time series of X-ray images and B) performing inference using the neural network 140 trained to predict the shape of a stent from a time series of X-ray images according to some aspects of the present disclosure. DETAILED DESCRIPTION
[0022] Examples of the present disclosure are provided with reference to the following description and accompanying drawings. In the description, many specific details of certain examples are set forth for the purpose of explanation. References in the specification to "example," "implementation," or similar language mean that the features, structures, or characteristics described in conjunction with the example are included in at least one of the examples. It should also be understood that features described with respect to one example may also be used in another example, and for the sake of brevity, all features do not have to be repeated in each example. For example, features described with respect to a computer-implemented method may be implemented in a corresponding manner in a computer program product and system.
[0023] In the following description, reference is made to a computer-implemented method for determining a value of a physical property of a thrombus. Reference is made to an example in which the thrombus is located in a blood vessel. Typically, the thrombus can be located in a vein or an artery. In other words, the thrombus can be a venous thrombus, or alternatively, it can be an arterial thrombus. It should be understood that the thrombus can generally be located in any part of the body. For example, the thrombus can be located in the brain, the heart, the lungs, or a limb (such as a leg). In some examples, the thrombus can be located in the brain, and the purpose of the treatment procedure can be for the treatment of an (ischemic) stroke.
[0024] Note that the computer-implemented method disclosed herein may be provided as a non-transient computer-readable storage medium including computer-readable instructions stored thereon, which cause the at least one processor to perform the method when executed by at least one processor. In other words, the computer-implemented method may be implemented as a computer program product. The computer program product may be provided by dedicated hardware or hardware capable of running software in association with appropriate software. When provided by a processor, the function of the method feature may be provided by a single dedicated processor, by a single shared processor, or by a plurality of separate processors that may be shared by some of them. The function of one or more method features may be provided, for example, by a processor shared within a networked processing architecture (such as a client / server architecture, a peer-to-peer architecture, the Internet, or a cloud).
[0025] The explicit use of the term "processor" or "controller" should not be interpreted as exclusively referring to hardware capable of running software, and may implicitly include, but is not limited to, digital signal processor "DSP" hardware, read-only memory "ROM" for storing software, random access memory "RAM", non-volatile storage devices, etc. In addition, examples of the present disclosure may take the form of a computer program product accessible from a computer-usable storage medium or a computer-readable storage medium, which provides a process code for use by or in conjunction with a computer or any instruction execution system. For the purpose of this description, a computer-usable storage medium or a computer-readable storage medium may be any device that can include, store, communicate, propagate or transmit a process for use by or in conjunction with an instruction execution system, device or device. The medium may be an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system or device or a propagation medium. Examples of computer-readable media include semiconductor or solid-state memory, magnetic tape, removable computer disk, random access memory "RAM", read-only memory "ROM", hard disk and optical disk. Current examples of optical disks include compact disk - read only memory "CD-ROM", compact disk - read / write "CD-R / W", Blu-ray Disc TM and DVD.
[0026] As mentioned above, removing a thrombus by mechanical thrombectomy is a challenging procedure, and understanding the physical properties of a thrombus will help the physician determine the course of action upon its extraction.
[0027] Figure 1 is a flow chart illustrating an example of a method of determining a value of a physical property of a thrombus according to some aspects of the present disclosure. Figure 1 , the method comprising:
[0028] receiving S110 X-ray image data, the X-ray image data comprising a time series of X-ray images 120 representing expansion of a stent 130 of an intraluminal stent retriever device on a thrombus 110;
[0029] determining S120 a value of a physical property of the thrombus 110 based on an expansion rate of the stent 130 in the time series of the X-ray images 120 ; and
[0030] The value of the physical property is outputted at S130.
[0031] The inventors have observed that the rate of expansion of a stent on a thrombus is indicative of various physical properties of the thrombus, including the size of the thrombus (e.g., its length, diameter, and volume) and the composition of the thrombus (e.g., soft versus hard or organized). These physical properties influence the physician's choice of how to extract the thrombus. Thus, by using the above method, the physical properties of the thrombus can be determined during the stent retriever procedure, and thus used by the physician to determine the process of action when extracting the thrombus.
[0032] Reference above Figure 1 The method described can also be Figure 2 The system implementation shown, Figure 2 is a schematic diagram illustrating an example of a system 200 for determining a value of a physical property of a thrombus according to some aspects of the present disclosure. Figure 1 The operations described by the flowchart shown can also be performed by Figure 2 The system 200 shown performs, and vice versa.
[0033] refer to Figure 1 In operation S110 , X-ray image data is received. The X-ray image data includes a time series of X-ray images 120 representing expansion of a stent 130 of an intraluminal stent retriever device on a thrombus 110 .
[0034] Typically, the X-ray image data received in operation S110 may be generated by a projection X-ray imaging system. A projection X-ray imaging system typically includes a support arm that supports an X-ray source-detector arrangement structure, such as a so-called "C-arm" or "O-arm". The projection X-ray imaging system may alternatively include a support arm having a shape different from these examples. The projection X-ray imaging system typically generates a projection X-ray image with the support arm maintained in a static position relative to the imaging area during acquisition of the image data. The time series of X-ray images 120 received in operation S110 may be acquired periodically (i.e., at regular intervals) or intermittently (i.e., at irregular intervals). The time series of X-ray images 120 may be fluoroscopic (i.e., live) images. This allows the operation of determining S120 the value of a physical property of the thrombus 110 to be performed in real time. For example, the X-ray image data may be generated by Figure 2 The projection X-ray imager 220 is shown generating the X-ray image data. As an example, the X-ray image data may be generated by a Philips Azurion 7 X-ray imaging system sold by Philips Healthcare of Best, The Netherlands.
[0035] The X-ray image data received in operation S110 includes a time series of X-ray images 120 representing the expansion of the stent 130 of the intraluminal stent retriever device on the thrombus 110. In this regard, a time series can be generated during the stent retriever process to extract the thrombus from the vein. As described above, the stent retriever generally includes an expandable wire mesh tube, i.e., a stent, designed to integrally remove the clot. Prior to its deployment, the wire mesh tube is housed in a delivery catheter. The delivery catheter is positioned so that it overlaps the clot, and then the delivery catheter is withdrawn so that the wire mesh tube expands out from the distal end of the delivery catheter. As the wire mesh tube expands, it captures the clot. The wire mesh tube is then withdrawn into the delivery catheter and removed from the body. The time series of X-ray images 120 represents the stent and thrombus during at least some periods of expansion of the wire mesh tube or stent.
[0036] For example, it is possible to obtain an image from a projection X-ray imaging system such as Figure 2 The X-ray image data received in operation S110 may be received by a projection X-ray imager 220 as shown in FIG. 1 or from a computer readable storage medium or from the Internet or a cloud. Figure 2 The one or more processors 210 shown receive the X-ray image data. The X-ray image data may be received via any form of data communication, including wired, optical, and wireless communications. As some examples, when wired or optical communications are used, the communication may be via signals transmitted over cables or optical cables, and when wireless communications are used, the communication may be, for example, via RF or optical signals.
[0037] In operation S120, a value of a physical property of the thrombus 110 is determined based on the expansion rate of the stent 130 in the time series of the X-ray images 120. In this regard, the physical property may be, for example, the size of the thrombus (such as, for example, its length, or its diameter, or its volume) or the composition of the thrombus (such as, for example, whether it is soft or hard, or organized).
[0038] Figure 3 is a schematic diagram illustrating an example of various stages of expansion of a stent 130 of an intraluminal stent retriever device on a thrombus 110 according to some aspects of the present disclosure. Figure 3 The illustration A in FIG. 1 shows a thrombus before the stent retriever "SR" is introduced into the blood vessel. In the illustration A, the thrombus 110 completely blocks the blood vessel. Figure 3The illustration B in FIG. 1 shows a thrombus and a stent of a stent retriever device shortly after the stent is deployed from the stent retriever. In the illustration B, the stent has been positioned so that it overlaps the thrombus, and the stent has begun to expand on the thrombus. The stent is laterally squeezed against the thrombus relative to the axis of the vessel, and the outer surface of the stent is conformed to the outer surface of the thrombus. At about 3 to 5 minutes after the stent is deployed, the illustration C shows a stent and thrombus for a soft clot. In this example, the stent has cut through the soft clot, completely encapsulating the soft clot. At about 3 to 5 minutes after the stent is deployed, the illustration D shows a stent and thrombus for a hard or organized clot. In this example, compared to the illustration B, the stent further laterally squeezes the thrombus relative to the axis of the vessel, and the outer surface of the stent is still conformed to the outer surface of the thrombus. After another period of time, and depending on the physical properties of the thrombus (such as its size and hardness), the stent for the hard clot can remain in the same or similar expanded state as in the illustration D, or it can cut through some or all of the clot, thereby partially or completely encapsulating the clot. As can be seen in particular in panels C and D, a distinction between soft and hard clots can therefore be made based on the rate of expansion of the stent; the rate of expansion of the stent is higher for the soft clot in panel C than for the hard clot in panel D.
[0039] The expansion rate of the stent also provides information about other physical properties of the clot in addition to hardness. For example, the length of the stent can be determined based on the expansion rate of the stent at multiple locations along its length over time. In the initial period of stent expansion, that is, in the time interval between the deployment of the stent and the stage represented by diagram B, the stent squeezes the thrombus in a direction perpendicular to the axis of the vessel, and the outer surface of the stent is adapted to the outer surface of the thrombus. During this initial period, the length of the thrombus can be determined based on the locations where the expansion rate of the stent along the length of the stent undergoes a significant decrease. These locations can correspond to areas where the outer surface of the stent contacts the outer surface of the thrombus. Therefore, the length of the clot can be determined based on the expansion rate of the stent along its length. Since the stent is easier to distinguish than the thrombus in an X-ray image, this technology can provide an accurate method for determining the length of the clot.
[0040] The diameter and volume of a thrombus can also be determined based on the rate of expansion of the stent along its length. For example, a model representing the rate of expansion of the stent along its length can be generated for thrombi with different diameters or volumes. The model can then be used to correlate the diameter and volume of the thrombus with the rate of expansion of the stent along its length. The model can also be used to generate a lookup table of values of the rate of expansion of the stent along its length for different values of diameter or volume. In use, the diameter and volume of the thrombus can be determined from the lookup table using the detected values of the rate of expansion of the stent along its length.
[0041] Various techniques may be used to determine the expansion rate of stent 130 in operation S120. For example, image analysis techniques may be used to determine the expansion rate of the stent at one or more locations along its length in a time series of X-ray images, and the physical properties of the thrombus may be determined from the expansion rate as described above. By registering the images to one another using, for example, image intensity or the ends of the stent, changes in the shape of the stent in the images may be determined, and these changes are used to calculate the expansion rate using the time interval between the images. The expansion rate may then be used to determine the value of the physical property of the thrombus.
[0042] In other examples, a neural network is used to determine a value of a physical property of the thrombus 110 in operation S120, as described in more detail below. In particular, in some examples, the determining step involves a neural network trained using a training sequence of X-ray images, each X-ray image representing expansion of a stent on a thrombus having known values of one or more physical properties of the thrombus. It has been discovered that the neural network can be trained accordingly to learn to derive (one or more) values of the one or more physical properties from such a sequence of X-ray images. During inference, when the trained network is used with a sequence of live X-ray images of a thrombus under examination as input, one or more current values of the physical property of the thrombus under examination can be provided as an output of the network accordingly.
[0043] In operation S130, the value of the physical property is output. The value of the physical property can be output in various ways, including output to a computer-readable storage medium or a display device, such as Figure 2 Monitor 240 is shown in FIG. By outputting the value of the physical property, a physician can use this information to determine the course of action for treating the thrombus.
[0044] As described above, in some examples, a neural network is used to determine a value of a physical property of thrombus 110 in operation S120 and provide the value as an output of the network. Figure 4 and Figure 5 Describe an example of a neural network used for this purpose.
[0045] Figure 4 Schematic diagram illustrating an example of A) training a neural network 140 to generate a latent space encoding of a time series of x-ray images and B) performing inference using the neural network 140 trained to generate a latent space encoding of a time series of x-ray images 120 in accordance with some aspects of the present disclosure. Figure 4 The example neural network 140 shown has a variational autoencoder architecture. The neural network 140 may alternatively have another architecture, such as a convolutional neural network "CNN" architecture, or a recurrent neural network "RNN" architecture with a unidirectional or bidirectional long short-term memory "LSTM" architecture, etc. Figure 4A, training X-ray image 120 showing expansion of a stent of an intraluminal stent retriever device over a thrombus during training T The time series is input into a variational autoencoder neural network 140, and the neural network is trained to generate a latent space encoding z representing the input time series. The variational autoencoder is trained by forcing its decoder to regenerate the training X-ray images 120 input into its encoder according to the latent space encoding z generated by its encoder. T The multiple time series of training X-ray images are used to train the neural network 140. During training, the variational autoencoder also learns the training X-ray images 120 T Distribution of Latent Space Codes for Time Series 160 T The distribution in Figure 4 is represented as a normal distribution with mean m and variance s160 T The training data used to train the neural network 140 is selected so that it represents a thrombus having a physical property with a known value. The training data may, for example, represent a soft clot. In doing so, the neural network 140 learns to generate a latent spatial encoding z of the time series of input images and a distribution 160 of the latent spatial encoding for a thrombus having a known value of the physical property (e.g., for a soft thrombus). T .
[0046] refer to Figure 4 B. At inference time, a new temporal sequence of X-ray images 120 is input into the trained neural network 140. The trained neural network 140 generates a latent spatial encoding 150, z, of the input image in each new sequence. Then, the latent spatial encoding 150, z, is calculated relative to the training X-ray images 120. T Distribution of Latent Space Codes for Time Series Generation 160 T The position of the latent space encoding 150 for the new time series is determined. If the clot represented in the new time series of x-ray images 120 has a similar position to the training x-ray images 120, T For example, if the clot is a soft clot, the latent space encoding 150 will be close to the training X-ray image 120. T Distribution of Latent Space Codes for Time Series Generation 160 T In other words, the latent space encoding 150 will be an inlier of the distribution 160. In contrast, if the clot represented in the new time series of x-ray images 120 is the same as the one in the training x-ray images 120 T For example, if the clot is a hard clot, the latent space encoding 150 will be far away from the training X-ray image 120. TDistribution of Latent Space Codes for Time Series Generation 160 T In other words, the latent space encoding 150 will be the distribution 160 T Outliers. Figure 4 In the example shown, the latent space encoding 150 is an outlier in the distribution space generated using the time series for soft clots, and therefore concludes that the clots in the new time series of x-ray images 120 are hard clots.
[0047] More generally, one can use Figure 4 The neural network shown is based on a latent space encoding 150 of an input image, z, relative to a training X-ray image 120 representing expansion of a stent on a thrombus with known values of physical properties. T Distribution of Latent Space Codes for Time Series Generation 160 T The value of the physical property of the thrombus 110 is determined based on the location of the thrombus 110 .
[0048] In the above example, it should be noted that the decoder part of the neural network that reconstructs the input image during training (i.e. Figure 4 A and Figure 4 B) is not necessary for performing inference. This is because in this example, the value of the physical property of the thrombus 110 is determined using the latent space encoding 150, z, of the input image, and this does not rely on the presence of the reconstructed image. In this example, the decoder portion is only used to train the neural network. In another example described below, the decoder portion of the neural network is used at inference time, and the neural network determines the value of the physical property of the thrombus 110 based on the difference DE between the predicted shape of the stent and the actual shape of the stent in the input image.
[0049] In the above example, "clot hardness" is used as an example of a physical property of a thrombus for which training data is selected, and the known value is "soft". However, by alternatively or additionally using training data representing other known values of the physical property of the thrombus to train the neural network 140, and determining at inference the latent spatial encoding 150, z, of a new time series of x-ray images 120, relative to the position of each of these distributions, the neural network 140 can also be used to determine how close the latent spatial encoding of the new time series of x-ray images 120 is to the mean of the distributions of these other known values of the physical property. In doing so, the neural network 140 can be trained and used to determine values of other physical properties of the thrombus 110.
[0050] Therefore, reference Figure 1 In the method shown, in one example, the operation of determining S120 a value of a physical property of the thrombus 110 may include:
[0051] Inputting multiple images from the time series into the neural network 140;
[0052] Using a neural network 140 to generate a latent space encoding 150, z, of an input image; and
[0053] Based on the latent space encoding 150 of the input image, z, relative to the training X-ray image 120 T Distribution of Latent Space Codes for Time Series Generation 160 T The training X-ray image 120 is used to determine the value of the physical property of the thrombus 110. T representing expansion of the stent over a thrombus having known values of physical properties; and
[0054] The neural network 140 is trained using the training X-ray image 120 T A latent space encoding 150 of an input image is generated using training data of time series of images, z, and wherein each sequence represents expansion of a stent of an intraluminal stent retriever device on a thrombus having a known value of a physical property.
[0055] In the above method, the latent space encoding 150 of the input image is compared with the training X-ray image 120 T Distribution of Latent Space Codes for Time Series Generation 160 T The location of can be determined by calculating the distance between the location of the latent space encoding 150 of the input image, z, and the mean or centroid of the distribution. For example, the distance can be measured using a function such as Euclidean distance or geodesic distance. The distance provides an analog measure of how close the physical property of the thrombus represented in the input image is to a known value of the physical property. The measure can be expressed on a continuous scale as a fraction or percentage representing how close the physical property of the thrombus represented in the input image is to the known value of the physical property. Alternatively, by digitizing the distance using a threshold, a binary classification of the physical property can be provided, such as hard component versus soft component. The threshold can be based on, for example, the distribution 160 of the latent space encoding. T For example, if the distance is less than 1 s for a new time series, the new time series can be classified using the known values of the physical property from the training data.
[0056] refer to Figure 4 The described neural network can be trained to generate a latent space encoding 150, z, of an input image by the following steps:
[0057] Receives a training X-ray image 120 T training data of a plurality of time series, each time series representing expansion of a stent of an intraluminal stent retriever device on a thrombus having a known value of a physical property;
[0058] Inputting training data into neural network 140; and
[0059] For each of the multiple input training X-ray images in the time series:
[0060] Generate a latent space encoding z of the input training X-ray image using a neural network 140;
[0061] reconstructing an input training X-ray image from the latent space encoding z using the neural network 140; and
[0062] adjusting parameters of the neural network 140 based on the difference between the input training X-ray image and the reconstructed input training X-ray image; and
[0063] Generate, rebuild, and adjust repeatedly until a stopping criterion is met.
[0064] Typically, training of a neural network involves inputting a training data set into the neural network and iteratively adjusting the parameters of the neural network until the trained neural network provides accurate outputs. Training is typically performed using a graphics processing unit "GPU" or a dedicated neural processor (such as a neural processing unit "NPU" or a tensor processing unit "TPU"). Training typically employs a centralized approach, where a cloud-based or mainframe-based neural processor is used to train the neural network. After its training using the training data set, the trained neural network can be deployed to a device for analyzing new input data during inference. The processing requirements during inference are significantly less than those required during training, allowing the neural network to be deployed to a variety of systems, such as laptops, tablets, mobile phones, and the like. Inference can be performed, for example, by a central processing unit "CPU", GPU, NPU, TPU on a server or in the cloud.
[0065] Thus, the process of training the neural network 140 described above includes adjusting its parameters. Parameters, or more specifically weights and biases, control the operation of activation functions in the neural network. In supervised learning, the training process automatically adjusts the weights and biases so that when presented with input data, the neural network accurately provides corresponding expected output data. To this end, the value of a loss function or error is calculated based on the difference between the predicted output data and the expected output data. The value of the loss function can be calculated using functions such as negative log-likelihood loss, mean absolute error (or L1 norm), mean squared error, root mean squared error (or L2 norm), Huber loss or (binary) cross entropy loss. When training a variational autoencoder, other loss functions such as Kullback–Leibler divergence can be used in addition to ensure that the training X-ray images 120 are accurately trained. T Distribution of Latent Space Codes for Time Series Generation 160T Similar to a standard Gaussian distribution with a mean of 0 and a standard deviation of 1. During training, the value of the loss function is usually minimized, and training is terminated when the value of the loss function meets a stopping criterion. Sometimes, training is terminated when the value of the loss function meets one or more of multiple criteria.
[0066] Various methods for solving the loss minimization problem (such as gradient descent, quasi-Newton method, etc.) are known. Various algorithms have been developed to implement these methods and their variants, including but not limited to stochastic gradient descent "SGD", batch gradient descent, mini-batch gradient descent, Gauss-Newton, Levenberg Marquardt, Momentum, Adam, Nadam, Adagrad, Adadelta, RMSProp and Adamax "optimizers". These algorithms use the chain rule to calculate the derivatives of the loss function with respect to the model parameters. This process is called back propagation because the derivatives are calculated starting from the last layer or output layer and moving toward the first layer or input layer. These derivatives inform the algorithm how the model parameters must be adjusted in order to minimize the error function. That is, the adjustment of the model parameters starts from the output layer and works backward in the network until the input layer is reached. In the first training iteration, the initial weights and biases are usually randomized. Then, the neural network predicts output data, which is also random. Back propagation is then used to adjust the weights and biases. The training process is iteratively performed by adjusting the weights and biases in each iteration. When the error or difference between the predicted output data and the expected output data is within an acceptable range for the training data or some validation data, the training is terminated. Subsequently, the neural network can be deployed and the trained neural network makes predictions on new input data using the trained values of its parameters. If the training process is successful, the trained neural network accurately predicts the expected output data from the new input data.
[0067] Typically, the above references can be extracted from the procedure records of historical mechanical thrombectomy procedures using intraluminal stent retriever devices. Figure 4 The training data used in the described training method. Training X-ray images 120 T Time series of training X-ray images 120 usually form part of these process records. TTens, hundreds, or thousands or more of such processes may be used to train the neural network 140 over a time series of images. The training data may represent patients of different ages, genders, body mass indexes, and the like. The known physical properties of the thrombus represented in the training data may be known values of thrombus size or known composition of the thrombus, as described above. The size of the thrombus may be known by calculating its value from the training X-ray images using a scaling factor determined based on the geometry of the X-ray source, the X-ray detector, and the relative position of the X-ray source and the X-ray detector relative to the thrombus, or alternatively, it may be determined by measuring the extracted clot during post-procedure histological analysis. Such information is typically stored in the procedure record. The composition of the thrombus may also be determined from histological analysis of the extracted clot, and this information is also typically stored in the procedure record. The physical properties of the thrombus may alternatively be determined from image data (such as magnetic resonance image data representing the thrombus, or CT image data representing the thrombus, or intravascular ultrasound "IVUS" image data representing the thrombus).
[0068] Alternatively, training data in which thrombi with known values of another physical property are represented can be used to train the neural network 140. In doing so, the neural network can be used to determine how close the physical properties of thrombi in the new time series are to other types of physical properties other than their size or their composition, as described in the examples above.
[0069] In one example, reference Figure 4 The neural network described is trained so that it places more emphasis on data within a defined region of interest than on data outside the region of interest. In this example, the region of interest includes stents and thrombi. T For example, a user may define a region of interest by inputting a user input device (such as a display device that displays training X-ray images 120) to the user. T A graphical user interface (GUI) is provided for user input while training the X-ray image 120. T The region of interest may be manually defined in the X-ray image 120. For example, the region of interest may be defined by the user using a bounding box. Alternatively, the region of interest may be automatically determined. For example, a trained neural network or feature detection algorithm may be used to train the X-ray image 120. T Thus, in one example, the neural network 140 is further trained to generate a latent space encoding 150, z, of the input image by the following steps:
[0070] receiving input indicating a region of interest in a time series of training x-ray images, the region of interest comprising a stent and a thrombus; and
[0071] wherein adjusting the parameters of the neural network based on the difference between the input training X-ray image and the reconstructed input training X-ray image within the region of interest is performed by applying weights to the difference, and wherein relatively higher weights are applied to portions of the time series of training X-ray images within the region of interest and relatively lower weights are applied to portions of the time series of training X-ray images outside the region of interest.
[0072] By using the region of interest to train the neural network in this manner, the neural network can be trained to generate a latent space encoding that is more specific to the region of interest. Thus, the neural network can provide a more accurate determination of the value of the physical property of the thrombus 110.
[0073] Various additional data may also be used to train neural network 140, as described in the examples below.
[0074] As described above, the process of extracting thrombus is complicated, and the workflow steps taken by the physician during its extraction depend in part on the physical properties of the thrombus. For example, an organized (hard, fibrin-rich) clot is more elastic and less viscous than a fresh (soft, erythrocyte-rich) clot, which results in less engagement with the stent retriever and may result in the loss of part or all of the clot during retrieval by the stent retriever, particularly in the case of a tortuous arterial tree. Therefore, the delay between the deployment and retrieval of the stent retriever device can vary according to its composition. As another example, an organized clot can cause tension in the stent retriever deployment section of the parent artery, which can cause arterial spasm. In this case, an intra-arterial vasodilator can be administered, or a stent retriever and suction or intermediate catheter can be used. Permanent stent placement using, for example, a balloon angioplasty device can also be considered as the final action. Glycoprotein IIb / IIIa inhibitors can also be administered to inactivate platelets and thereby prevent repeated reocclusion of blood vessels.
[0075] Thus, a physician performing a mechanical thrombectomy procedure is faced with many choices of workflow steps, such as “wait X minutes between stent deployment and retrieval,” “inject vasodilators,” “use a stent retriever in combination with an aspiration catheter / intermediate catheter,” “insert a permanent stent using a balloon angioplasty procedure,” “administer a glycoprotein IIb / IIIa inhibitor.” Each of these steps may impact the success of the procedure. Other workflow steps that may also impact the success of the procedure and are also performed during a mechanical thrombectomy procedure include steps such as “move the distal end of the stent retriever / aspiration catheter to X centimeters from the clot,” “increase aspiration on the aspiration catheter,” “generate 3D CT images to detect risk of SAH.” These latter workflow steps may or may not depend on the physical properties of the clot. However, they may also impact the success of the procedure.
[0076] In some examples, used to train the above reference Figure 4 The training data for the described neural network also includes workflow steps for successfully treating thrombi in a time series of training X-ray images. This data can be extracted from process records of historical mechanical thrombectomy processes. At inference time, these workflow steps are used to provide recommendations for (one or more) subsequent workflow steps during the mechanical thrombectomy process.
[0077] In these examples, the training reference Figure 4 The training data of the described neural network also includes the training X-ray images 120 T The workflow data defines one or more subsequent workflow steps for successfully treating a thrombus in the time series of training X-ray images. Figure 4 The described methods also include:
[0078] Determine 150 a distribution of a latent space encoding of an input image, z, relative to a latent space encoding of common workflow data 160 T location; and
[0079] The common workflow data is output to provide one or more recommended subsequent workflow steps for treating the thrombus 110 .
[0080] By providing such recommendations, physicians performing mechanical thrombectomy procedures are provided with additional guidance that can be used to improve the success of the procedure.
[0081] The above method of recommending workflow steps stems from the insight that if the latent spatial encoding of the image sequence from the current process is similar to the latent spatial encoding of the image sequence from the training data, then successful workflow steps taken in the process represented in the training data may also produce a similar level of success in the current process. The above method can be visualized as follows. Training X-ray Image 120 T The time series is labeled with one or more subsequent steps for successfully treating the thrombus in the sequence. TThe latent space encodings of the time series of images form clusters in the latent space according to the (one or more) subsequent steps for successfully treating the thrombus in each sequence. For example, time series that share a common workflow step of "waiting three minutes between stent deployment and retrieval" can form one cluster in the latent space. Time series that share a common workflow step of "waiting five minutes between stent deployment and retrieval" can form another cluster in the latent space. At inference time, the position of the latent space encoding of the new time series of X-ray images is determined relative to each of these clusters. If the latent space encoding of the new time series is closer to the cluster "waiting five minutes between stent deployment and retrieval", then the workflow step is output as a recommended subsequent workflow step for treating the thrombus. In contrast, if the latent space encoding of the new time series is closer to the cluster "waiting three minutes between stent deployment and retrieval", then the workflow step is output. The position of the latent space encoding for the new time series of images can be determined relative to the cluster by determining the distance between the centroids of the distribution of the latent space encodings that form the clusters. For example, the distance can be calculated in the same manner as described above (i.e., by using a function such as Euclidean distance or geodesic distance).
[0082] refer to Figure 5 Describes relevant examples, Figure 5 Schematic diagram illustrating an example of A) training a neural network 140 to generate a latent space encoding of a time series of x-ray images and B) performing inference using a neural network 140 trained to generate a latent space encoding of a time series of x-ray images 120 and wherein the training data also includes workflow data in accordance with some aspects of the present disclosure. Figure 5 Corresponds to Figure 4 , and additionally includes multiple distributions 160 with latent space encodings of common workflow data T 0..n Description. Share with Figure 4 The same mark as the Figure 5 Feature execution and reference in Figure 4 Same functionality as described in Figure 5 In the example shown, there are common workflow steps 170 T 0..n The latent space encoding of the time series forms clusters, that is, the distribution 160 T 0..n. As an example, a workflow step "typical workflow" distribution may represent a clustering of latent space encodings for successful procedures in which the workflow step "injecting a vasodilator" was performed. A workflow step "workflow variant 1" distribution may, for example, represent a clustering of latent space encodings for successful procedures in which a different workflow step "using a stent retriever in conjunction with an aspiration catheter" was performed. A workflow step "workflow variant 2" distribution may, for example, represent a clustering of latent space encodings for successful procedures in which a different workflow step "inserting a permanent stent using a balloon angioplasty procedure" was performed. At inference time, relative to a workflow step 170 having a corresponding T 0..n Distribution 160 T 0..n The centroid of each of determines the location in the latent space for encoding the new time series. The output is then the distribution with the closest centroid 160 T 0..n The corresponding workflow steps are presented as recommended next steps.
[0083] Therefore, in this example, the training reference Figure 4 The training data of the neural network 140 described includes data 170 having different workflows T 0..n a plurality of time series of training X-ray images; and determining a latent space encoding 150 of the input image, z, with respect to a distribution 160 of a latent space encoder having common workflow data T 0..n The operation of the position includes determining a latent space encoding 150 of the input image, z, with respect to each distribution 160 of the latent space encoder having common workflow data T 0..n In addition, the operation of outputting common workflow data to provide one or more recommended subsequent workflow steps for treating the thrombus 110 includes outputting a distribution 160 for the centroids having the closest position to the latent space encoding 150, z, of the input image. T 0..n Public workflow data.
[0084] In these examples, the workflow step(s) may also include a success metric that represents a probability that one or more subsequent workflow steps result in successful treatment of the thrombus 110. At inference time, the success metric of the output workflow step(s) may also be output. The success metric may also be calculated based on a flow record of historical mechanical thrombectomy flows. For example, the success metric may be estimated by calculating a percentage of successful flows in a certain neighborhood of the location of the latent space encoding 150, z, of the input image, where the distribution 160 of all successful flows executed in the neighborhood of the location of the latent space encoding 150, z, of the input image is closest to the centroid. T 0..n The calculated success metric can be higher if the position of the latent space encoding 150,z, of the input image is close to the centroid. T 0..n , the success metric may be lower because the neighborhood of the location of the latent space encoding 150, z, of the input image may overlap with a distribution different from the distribution having the nearest centroid. Alternatively, the success metric may be output directly by the neural network 140, or may be based on a reconstruction error DE between the predicted image and the input image. By outputting the success metric, the method helps the physician to perform the recommended workflow steps with confidence in their expected results. Therefore, in some examples, the workflow data may also include a success metric corresponding to one or more subsequent workflow steps; the success metric represents the probability that the one or more subsequent workflow steps result in successful treatment of the thrombus 110; and the operation of outputting the common workflow data also includes outputting the corresponding success metric.
[0085] In some examples, the operation of determining the position of the latent space encoding 150, z, of the input image relative to the distribution 160 of the latent space encoding having the common workflow data can be performed using a neural network. In these examples, determining the latent space encoding 150, z, of the input image relative to the distribution 160 of the latent space encoding having the common workflow data TThe operation of determining the position of the input image comprises inputting the latent space encoding 150, z, of the input image into a second neural network. The second neural network is trained to determine one or more subsequent workflow steps for successfully treating the thrombus based on the latent space encoding 150, z, of the input image using training data comprising a plurality of time series of training X-ray images, and wherein each sequence represents expansion of a stent of an intraluminal stent retriever device on a thrombus having a known value of a physical property, and wherein the training data comprises workflow data for each time series of training X-ray images, the workflow data defining one or more subsequent workflow steps for successfully treating the thrombus in the time series of training X-ray images.
[0086] refer to Figure 5 The distribution shown in 160 T 0..n , by using a neural network to perform this operation, determining the latent space encoding of the new time series with multiple distributions 160 T 0..n The task of finding the distance between the centroids of can be relieved. The second neural network can be trained in a similar manner to that described above.
[0087] In some examples, additional data in the form of patient data can be used to train the reference Figure 4 and Figure 5 neural network described. By providing the patient data as a separate input to the neural network, the neural network can learn the dependency between the data and the generated latent space encoding 150. Patient data such as age, gender, etc. can be input into the neural network during training and used to train the neural network 140. During inference, the patient data can also be input into the trained neural network and used to generate a latent space encoding 150, z of the input image. The patient data can be extracted from the health records of patients who have undergone historical mechanical thrombectomy procedures. Similarly, volumetric images representing the vasculature surrounding the thrombus can be used for the patient, and these volumetric images can also be used during training and at inference. The volumetric images can, for example, include CT images or MRI or MRA images of the vasculature. Therefore, in some examples, the neural network 140 is trained to generate a latent space encoding 150, z, of the input image further based on the patient data; and reference is made to Figure 4 The described methods also include:
[0088] receiving patient data related to thrombus 110;
[0089] Inputting patient data into the neural network 140; and
[0090] generating a latent space encoding 150, z, of the input image further based on the input patient data using the neural network 140; and
[0091] The patient data includes one or more of the following: a volume image representing a vascular system surrounding the thrombus 110 ; and electronic health record data related to the thrombus 110 .
[0092] In some examples, additional data in the form of extracted stent data and / or thrombus data may be used to train the reference Figure 4 and Figure 5 The neural network described herein can be extracted from the time series of x-ray images. By providing this data as a separate input to the neural network, as with the additional patient data, the neural network can learn the dependency between this data and the generated latent space encoding 150. Thus, in some examples, the neural network 140 is trained to generate a latent space encoding 150, z, of the input image further based on the stent data and / or thrombus data; and with reference to Figure 4 The described methods also include:
[0093] extracting stent data and / or thrombus data from a time series of X-ray images;
[0094] inputting the stent data and / or thrombus data into the neural network 140; and
[0095] generating a latent space encoding 150, z, of the input image further based on the input stent data and / or thrombus data using the neural network 140; and
[0096] Wherein, the stent data includes one or more of the following: the size, shape and expansion rate of the stent over time in a time series of X-ray images; and
[0097] The thrombus data includes the size and / or composition of the thrombus 110 .
[0098] In these examples, data can be extracted from the X-ray images manually or automatically. For example, thrombus data can be determined using image processing or computer vision algorithms to segment the clot. Neural networks can also be trained to extract these features from the X-ray images. Stent data can be determined using computer vision techniques or neural networks, such as, for example: thresholding, region growing, template matching, level sets, active contour modeling, and neural networks (such as U-Net).
[0099] Reference above Figure 4 and Figure 5 In the depicted example, a neural network is constructed based on a latent space encoding 150 of an input image, z, relative to a training X-ray image 120 representing expansion of a stent on a thrombus having known values of physical properties. TThe time series generates a latent space encoding the position of the distribution 160 to determine the value of the physical property of the thrombus 110.
[0100] An alternative to the technique is to predict the shape of a stent in one or more images in the time series based on an input image, and determine a value of a physical property of the thrombus 110 based on a difference DE between the predicted shape of the stent in one or more images in the time series and the actual shape of the stent in one or more images in the time series.
[0101] Therefore, reference Figure 1 In the illustrated method, in one example, the operation of determining S120 a value of a physical property of the thrombus 110 includes:
[0102] Inputting multiple images from the time series into the neural network 140;
[0103] predicting the shape of a stent in one or more images in the time series from an input image; and
[0104] determining a value of a physical property of thrombus 110 based on a difference DE between a predicted shape of the stent in one or more images in the time series and an actual shape of the stent in one or more images in the time series; and
[0105] The neural network 140 is trained using the training X-ray image 120 T The invention provides a method for predicting the shape of a stent in one or more images in the time series using training data of a plurality of time series, and wherein each sequence represents expansion of a stent of an intraluminal stent retriever device on a thrombus having a known value of a physical property.
[0106] This method is Figure 6 As schematically shown in FIG. Figure 6 is a diagram illustrating an example of A) training a neural network 140 to predict the shape of a stent from a time series of X-ray images and B) performing inference using the neural network 140 trained to predict the shape of a stent from a time series of X-ray images according to some aspects of the present disclosure. Figure 6 B. During inference, multiple images from a time series 120 are input into the neural network. In the illustrated example, the neural network predicts the shape of the stent in a future image after a period Dt from one or more images in the time series. Alternatively, the shape of the stent may be predicted in one or more images of the same input (i.e., when Dt=0). The difference DE between the shape of the stent in the (one or more) input image and the predicted shape of the stent is then calculated. The difference DE represents how close the value of the physical property of the thrombus 110 in the (one or more) input image is to the value of the physical property of the thrombus 110 in the training image used to train the neural network.
[0107] Should Figure 6 The method can be Figure 4 and Figure 5 The difference is that at inference time, the decoder part of these neural networks is now needed in order to calculate the difference DE. Figure 4 and Figure 5 The difference DE is illustrated in the lower part of . The difference DE can be calculated using a loss function such as L1 loss, L2 loss, etc.
[0108] Should Figure 6 The method uses the same Figure 4 and Figure 5 The neural network shown has the same insights described, namely, using training X-ray images 120 representing expansion of a stent of an intraluminal stent retriever device on a thrombus having known values of physical properties. T The neural network 140 is trained using a time series of images of the intraluminal stent retriever device on a thrombus having the same value of the physical property. Thus, at inference time, if the trained neural network is input with a new time series representing an intraluminal stent retriever device on a thrombus having the same value of the physical property, the neural network will be able to accurately reconstruct the input image(s), and in this case, there will be a small difference DE between the predicted shape of the stent in the one or more images in the time series and the actual shape of the stent in the one or more images in the time series. In contrast, at inference time, if the trained neural network is input with a new time series representing an intraluminal stent retriever device on a thrombus having a different value of the physical property, the neural network will not be able to accurately reconstruct the input image(s), and in this case, there will be a large difference DE.
[0109] Figure 4 and Figure 5 The neural network shown can also be trained to be similar to the one in the reference above. Figure 4 The method described above is performed in a similar manner (i.e., by forcing the neural network 140 to reconstruct the input image at inference time). Figure 6 Reference Figure 6 A, during training, training X-ray images 120 T The multiple time series are input into the neural network 140. The neural network predicts the shape of the stent in the image or subsequent images. The loss L between the predicted shape of the stent and the actual shape of the stent in the training image is calculated. The loss L is used to train the neural network. Therefore, the neural network can be trained to predict the shape of the stent in one or more images in the time series by the following steps:
[0110] Receives a training X-ray image 120 Ttraining data of a plurality of time series, each time series representing expansion of a stent of an intraluminal stent retriever device on a thrombus having a known value of a physical property;
[0111] Inputting training data into the neural network; and
[0112] For each of the multiple input training X-ray images in the time series:
[0113] predicting the shape of a stent in one or more images of a time series using a neural network;
[0114] adjusting parameters of the neural network based on a difference L between a predicted shape of the stent in one or more images in the time series and an actual shape of the stent in one or more images in the time series; and
[0115] Prediction and adjustment are repeated until a stopping criterion is met.
[0116] Can be referenced above Figure 4 and Figure 5 Training data is provided in the same manner as described above. Furthermore, as described with respect to the training of these neural networks, the neural network can be trained so that it focuses more on data within a defined region of interest than outside of the defined region of interest. Thus, in one example, the neural network is trained to also predict the shape of a stent in one or more images in the time series by the following steps:
[0117] identifying a region of interest in a time series of training X-ray images, the region of interest comprising a stent and a thrombus; and
[0118] wherein adjusting the parameters of the neural network based on a difference L between a predicted shape of the stent in one or more images in the time series and an actual shape of the stent in one or more images in the time series is performed by applying a weighting to the difference, and wherein a relatively higher weighting is applied to a portion of the time series of training X-ray images within the region of interest and a relatively lower weighting is applied to a portion of the time series of training X-ray images outside the region of interest.
[0119] At inference time, in addition to being used to determine the value of the physical property of the thrombus 110, the difference DE can also be used to recommend one or more subsequent workflow steps for treating the thrombus 110. This can be performed by using training data including workflow data defining one or more subsequent workflow steps for successfully treating the thrombus in the time series of training X-ray images. If the difference DE is small, then the (one or more) successful workflow steps used in the training data can be recommended. The neural network can also be trained to learn the association between the difference DE and the (one or more) subsequent workflow steps for successfully treating the thrombus in the time series of training X-ray images. Alternatively, Figure 6 The neural network shown can also be trained to generate a distribution of latent space encodings 150, z, of the input image, and can be obtained by using the same method as above. Figure 4 and Figure 5 In the same manner as described, determining the position of the latent space encoding of the time series of input X-ray images relative to the distribution 160 of latent space encodings having common workflow data is used to recommend one or more subsequent workflow steps for treating the thrombus 110 .
[0120] The neural network can also be trained to perform the method using additional data such as patient data, stent data, and thrombus data, as described above with respect to Figure 4 and Figure 5 As described.
[0121] Thus, in one example, the neural network 140 is trained to predict the shape of the stent in one or more images in the time series further based on the patient data; and the method further includes:
[0122] receiving patient data related to thrombus 110;
[0123] Inputting patient data into the neural network; and
[0124] further predicting a shape of a stent in one or more images in the time series from the input image based on the patient data; and
[0125] The patient data includes one or more of the following: a volume image representing a vascular system surrounding the thrombus 110 ; and electronic health record data related to the thrombus 110 .
[0126] In another example, the neural network is trained to predict the shape of the stent in one or more images in the time series further based on the stent data and / or the thrombus data; and the method further includes:
[0127] extracting stent data and / or thrombus data from a time series of X-ray images;
[0128] inputting the stent data and / or the thrombus data into the neural network; and
[0129] further predicting the shape of the stent in one or more images in the time series from the input stent data and / or thrombus data based on the patient data; and
[0130] Wherein, the stent data includes one or more of the following: the size, shape and expansion rate of the stent over time in a time series of X-ray images; and
[0131] The thrombus data includes the size and / or composition of the thrombus 110 .
[0132] In another example, a confidence value for the latent space encoding 150, z, or for the predicted shape of the stent may be calculated. The confidence value represents the confidence in the prediction made by the neural network 140, and it allows decisions to be made based on its output. For example, if the confidence is low, a decision may be made not to rely on the determined physical properties of the thrombus. In this regard, a warning may be output if the confidence value is below a predetermined threshold.
[0133] Therefore, reference Figure 4 or Figure 5 The described method may additionally include the following operations:
[0134] Calculating a confidence value for the latent space encoding 150, z, or a confidence value for the predicted shape of the bracket, respectively; and
[0135] The confidence value is output.
[0136] As an example, the neural network 140 may use a discarding technique to generate a confidence value. The discarding technique involves iteratively inputting the same data into the neural network and determining the output of the neural network, while randomly excluding a portion of the neurons from the neural network in each iteration. The output of the neural network is then analyzed to provide a mean and variance value. The mean represents the final output, and the size of the variance indicates whether the neural network is consistent in its predictions (in which case the variance is small and the confidence value is relatively higher), or whether the neural network is inconsistent in its predictions (in which case the variance is larger and the confidence value is relatively lower). Alternatively, the confidence can be calculated using, for example, the Kullback-Leibler "KL" divergence between the distribution from which the sampled input image representation is represented and the distribution on the currently trained encoding. This divergence indicates how well the input sequence is represented by the learned encoding.
[0137] In another example, a computer program product is provided. The computer program product includes instructions that, when executed by one or more processors, cause the one or more processors to perform a computer-implemented method for determining a value of a physical property of a thrombus 110 according to any embodiment or example described above. The method includes:
[0138] receiving S110 X-ray image data, the X-ray image data comprising a time series of X-ray images 120 representing expansion of a stent 130 of an intraluminal stent retriever device on a thrombus 110;
[0139] determining S120 a value of a physical property of the thrombus 110 based on an expansion rate of the stent 130 in the time series of the X-ray images 120 ; and
[0140] The value of the physical property is outputted at S130.
[0141] In another example, a system 200 for determining a value of a physical property of a thrombus 110 is provided. The system includes one or more processors 210 configured to perform the steps of a computer-implemented method for determining a value of a physical property of a thrombus according to any embodiment or example described above. That is, the processor 210 may be configured to:
[0142] receiving S110 X-ray image data, the X-ray image data comprising a time series of X-ray images 120 representing expansion of a stent 130 of an intraluminal stent retriever device on a thrombus 110;
[0143] determining S120 a value of a physical property of the thrombus 110 based on an expansion rate of the stent 130 in the time series of the X-ray images 120 ; and
[0144] The value of the physical property is outputted at S130.
[0145] For example, the one or more processors 210 may be operable to execute instructions of a computer program product as previously described.
[0146] exist Figure 2 An example of system 200 is illustrated in FIG. In some examples, system 200 may also include one or more of the following: Figure 2 The projection X-ray imager 220 for providing a time series of X-ray images 120 is shown; Figure 2The intraluminal stent retriever device 230 shown is used to perform a mechanical thrombectomy procedure on a blood vessel; a display device for displaying values of physical properties of a thrombus 110, a time series of X-ray images 120 representing expansion of a stent 130 of the intraluminal stent retriever device on the thrombus 110, one or more recommended subsequent workflow steps for treating the thrombus 110, (one or more) confidence values, etc., such as Figure 2 Shown are a monitor 240; a patient bed 250; and a user input device, such as a keyboard, mouse, touch screen, etc., for receiving user input related to operations performed by the system.
[0147] The above examples should be understood as explanations of the present disclosure rather than limitations. Other examples are also contemplated. For example, the examples described about the computer-implemented method can also be provided by a computer program product, or by a computer-readable storage medium, or by a system 200 in a corresponding manner. It should be understood that the features described about any one example can be used alone, or in combination with other described features, and can be used in combination with one or more features of another example or a combination of other examples. In addition, without departing from the scope of the present invention defined in the attached claims, equivalents and modifications not described above may also be adopted. In the claims, the word "including" does not exclude other elements or operations, and the word "one" or "an" does not exclude multiple. Although certain features are recorded in mutually different dependent claims, this does not indicate that the combination of these features cannot be used advantageously. Any figure mark in the claims should not be interpreted as limiting its scope.
Claims
1. A computer-implemented method of determining a value of a physical property of a thrombus (110), the method comprising: receiving (S110) X-ray image data, the X-ray image data comprising a time series of X-ray images (120), the time series of X-ray images representing expansion of a stent (130) of an intraluminal stent retriever device on the thrombus (110); determining (S120) the value of the physical property of the thrombus (110) based on the expansion rate of the stent (130) in the time series of the X-ray images (120); and The value of the physical property is output (S130).
2. The computer-implemented method of claim 1 , wherein: Determining (S120) the value of the physical property of the thrombus (110) comprises inputting a plurality of images from the time series into a computer program product using a training X-ray image (120 T ), each such training sequence representing expansion of a stent of an intraluminal stent retriever device on a thrombus having a known value of the physical property.
3. The computer-implemented method of claim 2, wherein: The determining (S120) further includes: generating a latent space encoding (150, z) of the input image using the neural network (140); and The latent space encoding (150, z) based on the input image is relative to the training X-ray image (120 T ) time series generated by the distribution of latent space codes (160 T ) to determine the value of the physical property of the thrombus (110), Wherein the neural network (140) is trained to generate the latent space encoding (150, z) of the input image using the training data.
4. The computer-implemented method of claim 2, wherein: The training data also includes training X-ray images (120 T ), the workflow data defining one or more subsequent workflow steps for successfully treating a thrombus in the time series of training X-ray images; and Wherein, the method further comprises: Determining the distribution of the latent space encoding (150, z) of the input image relative to the latent space encoding having common workflow data (160 T ); and The common workflow data is output to provide one or more recommended subsequent workflow steps for treating the thrombus (110).
5. The computer-implemented method of claim 4, wherein: The workflow data also includes a success metric corresponding to the one or more subsequent workflow steps; the success metric represents a probability that the one or more subsequent workflow steps will result in successful treatment of the thrombus (110); and Wherein, outputting the common workflow data further includes outputting the corresponding success metric.
6. The computer-implemented method of claim 4, wherein: Determining the distribution of the latent space encoding (150, z) of the input image relative to the latent space encoding having common workflow data (160 T ) position includes inputting the latent space encoding (150, z) of the input image into a second neural network; wherein the second neural network is trained using training data comprising a plurality of time series of training X-ray images to: determine the one or more subsequent workflow steps to successfully treat the thrombus based on the latent space encoding (150, z) of the input images, and output the one or more subsequent workflow steps, and wherein each sequence represents expansion of a stent of an intraluminal stent retriever device on a thrombus having a known value of the physical property, and wherein the training data comprises workflow data for each time series of training X-ray images, the workflow data defining one or more subsequent workflow steps for successfully treating the thrombus in the time series of training X-ray images.
7. The computer-implemented method of any one of claims 3 to 6, wherein: The neural network (140) is trained to generate the latent space encoding (150, z) of the input image also based on patient data; And wherein the method further comprises: receiving patient data related to the thrombus (110); inputting the patient data into the neural network (140); and Using the neural network (140), generating the latent space encoding (150, z) of the input image also based on the input patient data; and The patient data includes one or more of the following: a volume image representing a vascular system surrounding the thrombus (110); and electronic health record data related to the thrombus (110).
8. The computer-implemented method of any one of claims 3 to 7, wherein: The neural network (140) is trained to generate the latent space encoding (150, z) of the input image also based on stent data and / or thrombus data; And wherein the method further comprises: extracting the stent data and / or the thrombus data from the time series of the X-ray images; inputting the stent data and / or the thrombus data into the neural network (140); and Using the neural network (140), generating the latent space encoding (150, z) of the input image also based on the input stent data and / or thrombus data; and Wherein, the stent data includes one or more of the following: the size, shape and expansion rate of the stent over time in the time series of the X-ray images; and The thrombus data includes the size and / or composition of the thrombus (110).
9. The computer-implemented method of claim 3, wherein: The neural network (140) is trained to generate the latent space encoding (150, z) of the input image by the following steps: Receiving a training X-ray image (120 T ) and wherein each time series represents expansion of a stent of an intraluminal stent retriever device on a thrombus having a known value of the physical property; inputting the training data into the neural network (140); and For each of the multiple input training X-ray images in the time series: Using the neural network (140) to generate a latent space encoding (z) of the input training X-ray image; using the neural network (140) to reconstruct an input training X-ray image based on the latent space encoding (z); and adjusting parameters of the neural network (140) based on differences between the input training X-ray image and the reconstructed input training X-ray image; and The generating, the reconstructing and the adjusting are repeated until a stopping criterion is met.
10. The computer-implemented method of claim 2, wherein: Determining (S120) the value of the physical property of the thrombus (110) comprises: predicting the shape of the stent in one or more input images in the time series using the neural network (140); and The value of the physical property of the thrombus (110) is determined based on a difference (DE) between a predicted shape of the stent in the one or more images in the time series and an actual shape of the stent in the one or more images in the time series.
11. The computer-implemented method of claim 10, wherein: The neural network (140) is trained to predict the shape of the stent in the one or more images in the time series also based on patient data; and wherein the method further comprises: receiving patient data related to the thrombus (110); inputting the patient data into the neural network; and also predicting the shape of the stent in the one or more images in the time series from the input images based on the patient data; and The patient data includes one or more of the following: a volume image representing a vascular system surrounding the thrombus (110); and electronic health record data related to the thrombus (110).
12. The computer-implemented method of any one of claims 10-11, wherein: The neural network is trained to predict the shape of the stent in the one or more images in the time series also based on stent data and / or thrombus data; and wherein the method further comprises: extracting the stent data and / or the thrombus data from the time series of the X-ray images; inputting the stent data and / or the thrombus data into the neural network; and also predicting the shape of the stent in the one or more images in the time series from the input images based on the input stent data and / or thrombus data; and Wherein, the stent data includes one or more of the following: the size, shape and expansion rate of the stent over time in the time series of the X-ray images; and The thrombus data includes the size and / or composition of the thrombus (110).
13. The computer-implemented method of claim 10, wherein: The neural network is trained to predict the shape of the stent in one or more images in the time series by: Receiving a training X-ray image (120 T ) and wherein each time series represents expansion of a stent of an intraluminal stent retriever device on a thrombus having a known value of the physical property; inputting the training data into the neural network; and For each of the multiple input training X-ray images in the time series: using the neural network to predict a shape of the stent in one or more images of the time series; adjusting parameters of the neural network based on a difference (L) between a predicted shape of the stent in the one or more images in the time series and an actual shape of the stent in the one or more images in the time series; and The predicting and the adjusting are repeated until a stopping criterion is met.
14. A system (200) for determining a value of a physical property of a thrombus (110), comprising one or more processors (210) configured to perform the steps of the computer-implemented method according to any of the preceding claims.
15. The system of claim 14, further comprising a projection X-ray imager (220) for providing a time series of the X-ray images (120).
