Method and apparatus for processing medical images

Abstract

Systems, methods, and apparatuses associated with automatically detecting and enhancing multiple objects in medical scan images are described herein. The detection and / or enhancement can be accomplished with artificial neural networks, such as one or more classification neural networks and / or one or more graph neural networks. The neural networks can be used to detect regions in a medical scan image that can correspond to objects of interest, and to cluster regions belonging to the same object into respective clusters. These tasks can be accomplished, for example, by representing regions corresponding to objects of interest and their interrelationships with a graph and processing the graph with one or more graph neural networks such that regions belonging to individual objects can be appropriately labeled and clustered. The clusters can then be used to enhance objects of interest in one or more output scan images.

Description

Technical Field

[0001] This application relates to the field of medical imaging, specifically to the processing of medical images, and to methods and equipment for processing medical images. Background Technology

[0002] The ability to automatically detect the location of objects within a patient's body (e.g., implanted medical devices, surgical instruments, etc.) using medical imaging techniques is crucial for the success of many medical procedures. Using fluoroscopy as an example, this imaging technique can be used to facilitate the execution of coronary angioplasty, a procedure to open blocked coronary arteries during which one or more stents can be placed. As part of this procedure, fluoroscopic images of the heart can be taken to visualize the heart and interventional devices, such as stents and / or guidewires placed inside the target artery. However, due to limitations in X-ray radiation dose, the captured fluoroscopic images may be noisy, making stent localization difficult. This difficulty becomes even greater when multiple objects (e.g., multiple stents) need to be detected and tracked. Therefore, systems, methods, and apparatuses capable of automatically detecting multiple objects in medical images and / or enhancing the presentation of objects in medical images are highly desirable. Summary of the Invention

[0003] This document describes systems, methods, and apparatuses associated with detecting and / or enhancing objects in medical scan images. An apparatus configured to perform these tasks may include one or more processors configured to: receive a plurality of medical scan images and use one or more artificial neural networks (ANNs) to identify a corresponding first region and a corresponding second region in each of the plurality of medical scan images, the corresponding first region and the corresponding second region corresponding to a first object of interest and a second object of interest, respectively. The plurality of medical scan images may include, for example, fluoroscopic images, and the objects of interest may include, for example, a first stent and a second stent. In response to the detection of the first and second regions in the respective medical scan images, the one or more processors of the apparatus may also be configured to: determine a first cluster that may include the corresponding first regions in each of the plurality of medical scan images, and determine a second cluster that may include the corresponding second regions in each of the plurality of medical scan images. The one or more processors may then generate an output medical scan image depicting at least one of a first object of interest based on the first cluster or a second object of interest based on the second cluster, wherein the depicted first or second object of interest may be enhanced in the output medical scan image.

[0004] In the example, the one or more processors described above can be configured to determine one or more first candidate regions in each of a plurality of medical scan images, and select a first region from the one or more first candidate regions. Similarly, the one or more processors can be configured to determine one or more second candidate regions in each of a plurality of medical scan images, and select a second region from the one or more second candidate regions. In the example, the one or more processors can be configured to determine the first cluster and the second cluster described above by determining a first feature associated with a first object of interest and a second feature associated with a second object of interest based on a plurality of medical scan images. The one or more processors can then determine the first cluster based on the first feature and the second cluster based on the second feature (e.g., using a mean-shift-based clustering technique).

[0005] In the example, the aforementioned one or more processors can be configured to determine graph representations of corresponding first and second regions in each of a plurality of medical scan images, and to determine first and second clusters by processing the graph representations using a graph neural network (GNN). For example, the corresponding first and second regions in each of the plurality of medical scan images can be represented as nodes in the graph representation. The first cluster can then be determined by using the GNN to identify a first subset of nodes corresponding to a first object of interest, and the second cluster can be determined by using the GNN to identify a second subset of nodes corresponding to a second object of interest. The GNN can be trained to perform these tasks by extracting corresponding features of the nodes in the graph representation and identifying the first and second subsets of nodes based on the extracted features.

[0006] In the example, the graph representation above may also include multiple edges, each edge connecting corresponding pairs of nodes and indicating whether the pair of nodes belongs to the same object of interest. Using this graph representation, one or more processors can be configured to identify a first subset of nodes based on a first set of one or more edges indicating that a first subset of nodes belongs to a first object of interest. Similarly, one or more processors can be configured to identify a second subset of nodes based on a second set of one or more edges indicating that a second subset of nodes belongs to a second object of interest. Edges can be labeled by the GNN with corresponding values ​​or confidence scores to indicate whether the pairs of nodes connected by the edge belong to the same object. For example, an edge can be labeled as 1 or true to indicate that the nodes associated with the edge are connected (e.g., belong to the same object), and an edge can be labeled as 0 or false to indicate that the nodes associated with the edge are disconnected (e.g., do not belong to the same object).

[0007] In the example, the graph representation above may also include multiple edges, each edge connecting corresponding pairs of nodes and representing the similarity between the pairs of nodes. Using this graph representation, one or more processors can be configured to identify a first subset of nodes by identifying a first path that includes one or more edges connecting a first subset of nodes. Similarly, one or more processors can be configured to identify a second subset of nodes by identifying a second path that includes one or more edges connecting a second subset of nodes. The first path can be identified as the longest path among a plurality of first candidate paths associated with the first subset of nodes, and the second path can be identified as the longest path among a plurality of second candidate paths associated with the second subset of nodes.

[0008] In the example, one or more of the processors described above can be configured to identify a first subset of nodes and a second subset of nodes through an iterative process. During a first phase of the iterative process, a first subset of nodes associated with a first object of interest can be identified and then removed from the graph representation. Then, during a second phase of the iterative process, a second subset of nodes associated with a second object can be determined from the remaining graph nodes.

[0009] In the example, an output medical scan image depicting at least one of a first object of interest or a second object of interest can be generated by aligning at least a subset of multiple medical scan images based on a first cluster or a second cluster (e.g., based on the position of the first or second object detected in the corresponding cluster) and averaging the subset of multiple medical scan images, such that background and noise signals in the output medical scan image can be suppressed and the contrast of the first or second object can be enhanced. Attached Figure Description

[0010] The examples disclosed herein can be understood in more detail from the following description, which is given by way of example in conjunction with the accompanying drawings.

[0011] Figure 1A This is an example fluorescent perforated image of a human heart, showing two stents placed inside the heart.

[0012] Figure 1B This is a diagram illustrating the first pair of balloon markers associated with the first stent and the second pair of balloon markers associated with the second stent.

[0013] Figure 2 This is a diagram illustrating an example object detection device according to one or more embodiments described herein.

[0014] Figure 3 This is a diagram illustrating an example of clustering and enhancing objects based on multiple medical scan images according to one or more embodiments described herein.

[0015] Figure 4A and Figure 4B This is a diagram illustrating example machine learning techniques that can be used for automatic detection and / or clustering of objects from input medical scan images according to one or more embodiments described herein.

[0016] Figure 5 This is a flowchart illustrating example operations that can be associated with training a neural network to perform one or more tasks as described in one or more embodiments herein.

[0017] Figure 6 This is a block diagram illustrating example components of a device that can be configured to perform one or more tasks as described in one or more embodiments herein. Detailed Implementation

[0018] The present disclosure is illustrated by way of example rather than limitation in the figures.

[0019] Figure 1A An example fluorescence fluoroscopic image of a human heart is shown, illustrating two stents placed within the heart (e.g., in...). Figure 1A (Enclosed by a dashed square 102). Each stent can be associated with a corresponding pair of balloon markers, which can be located at opposite ends of the stent. Figure 1B An example is shown of a first pair of balloon markers associated with the first stent (e.g., pointed to by arrow 104a) and a second pair of balloon markers associated with the second stent (e.g., pointed to by arrow 104b). Due to noise associated with the fluoroscopic images, there may be areas in the fluoroscopic images that may resemble stents and may be incorrectly identified as stents by the image processing system or device (e.g., the area enclosed by the dashed square 106).

[0020] This document may use fluorescent imaging and stent / balloon markings as examples to describe embodiments. However, those skilled in the art will understand that the related techniques described in the embodiments can also be used to process other types of images and / or objects.

[0021] Figure 2 This is a simplified diagram illustrating an example object detection device (ODA) 200 according to one or more embodiments disclosed herein. As shown, the ODA 200 can be configured to process a medical scan image 202 and detect multiple objects (e.g., a first object, a second object, etc.) shown in the medical scan image. The medical scan image 202 may include different types of images, including, for example, fluorescence fluoroscopy images (e.g., Figure 1A and Figure 1B(as shown), MRI images, etc. The objects being examined can also be of different types, including, for example, anatomical structures of the human body (e.g., the apex of the myocardium) and / or artificial structures placed inside the human body (e.g., stents, guide wires, etc.).

[0022] ODA 200 can be configured to perform one or more of the following operations. At 204, ODA 200 can identify a set of corresponding regions (e.g., blocks in the medical scan image) in the various medical scan images 202 that can correspond to an object of interest (e.g., a first stent or a first pair of balloon markers, a second stent or a second pair of balloon markers, etc.). For example, ODA 200 can be configured to detect multiple candidate regions in the various medical scan images 202 that correspond to balloon markers, distinguish between true and false marker regions (e.g., by removing false marker regions from the candidate region list), and further identify which pair of balloon markers belongs to the same stent. In the example, ODA 200 can be configured to automatically detect one or more images in the medical scan images 202 where the object of interest may be obscured by other objects (e.g., by contrast agent), and ODA 200 can be configured to remove these images from the processing operation at 204. In the example, ODA 200 can be configured to perform the task at 204 using one or more classification neural networks. In the example, the classification neural network may include a deep neural network, such as a convolutional neural network (CNN), which has been trained to extract features from the various input medical scan images 202 and classify regions of the medical scan images as containing balloon markers or not containing balloon markers based on the extracted features. In the example, one or more classification neural networks may include a graph neural network (GNN), which has been trained to determine (e.g., label) which regions (e.g., detected by the CNN above) in a list of candidate balloon marker regions are true labeled regions and which regions in the list of candidate balloon marker regions are false labeled regions. From the true labeled regions, the GNN can further determine which pair of balloon markers belong to the same stent. The corresponding structures and training of CNNs and GNNs will be described in more detail below.

[0023] In response to identifying regions in each input medical scan image 202 that may correspond to an object of interest, ODA 200 may group regions on multiple medical scan images 202 that may correspond to the same object into corresponding clusters at 206. For example, ODA 200 may group regions in multiple medical scan images 202 corresponding to a first stent into a first cluster, regions in multiple medical scan images 202 corresponding to a second stent into a second cluster, and so on. ODA 200 may then (e.g., at 208) utilize the clusters determined at 206 to enhance the respective objects of interest in the output image. For example, ODA 200 may align the multiple medical scan images 202 based on the position of a first detected object (e.g., a first stent) indicated by the first cluster, and generate a first output medical scan image (e.g., the first in output image 210) based on the aligned image, such that background and noise signals in the first output medical scan image can be suppressed, and the contrast of the first detected object in the first output medical scan image can be enhanced. Similarly, ODA 200 can align multiple medical scan images 202 based on the location of a second detected object (e.g., a second stent) indicated by a second cluster, and generate a second output medical scan image (e.g., the second in output image 210) based on the aligned images, such that background and noise signals in the second output medical scan image can be suppressed, and the contrast of the second detected object in the second output medical scan image can be enhanced. Enhancement can be achieved using various suitable techniques, including, for example, by time averaging of multiple images after they have been aligned (e.g., averaging the corresponding pixel values ​​of the images). In the example, outliers in each determined cluster can be removed before using the clusters to generate the enhanced image. In the example, ODA 200 can output more images than the number of objects (e.g., images may include copies of stents or false detections), and ODA 200 can provide a user interface to select / switch among these images (e.g., output images can be sorted in descending order based on the probability that the output image contains a unique stent (such as a stent not copied in a previous image).

[0024] Clustering at 206 can be performed using various suitable techniques. In the example, clustering can be performed at 209 using mean-shift clustering, during which features of objects of interest (e.g., multiple stents), such as the corresponding angle, distance, and / or other features of the balloon markers (e.g., machine learning features), can be determined and iteratively assigned (e.g., as data points) towards the nearest cluster centroid until the individual objects in the medical scan image 202 are assigned to clusters. In the example, clustering can be performed at 206 using machine learning (ML) techniques (such as using a graph neural network classifier), where objects of interest (e.g., stent or balloon marker pairs) can be represented as nodes in a graph, and the graph neural network can be used to classify (e.g., label) whether certain nodes belong to the same stent (and thus to the same cluster). Examples of ML-based techniques will be described in more detail below.

[0025] Figure 3 This is a diagram illustrating an example of clustering and enhancing objects based on multiple medical scan images. As shown in the figure, the object detection device described herein (e.g., Figure 2 The ODA 200 can be configured to receive multiple medical scan images (e.g., 302, 304, and 306) and automatically detect balloon marker pairs (or corresponding stents) 302a and 302b in scan image 302, balloon marker pairs (or corresponding stents) 304a and 304b in scan image 304, and balloon marker pairs (or corresponding stents) 306a and 306b in scan image 306. Using one or more clustering techniques described herein (e.g., clustering techniques based on mean shift and / or ML), the object detection device can determine that balloon marker pairs 302a, 304a, and 306a belong to a first cluster (e.g., associated with a first stent) and determine that balloon marker pairs 302b, 304b, and 306b belong to a second cluster (e.g., associated with a second stent). The object detection device can then align medical scan images 302, 304, and 306 (e.g., by aligning the corresponding positions of balloon marker pairs 302a, 304a, and 306a belonging to the first cluster), and the object detection device can generate a first output scan image with an enhanced first stent based on the aligned image. Similarly, the object detection device can also align medical scan images 302, 304, and 306 by aligning the corresponding positions of balloon marker pairs 302b, 304b, and 306b (belonging to the second cluster), and the object detection device can generate a second output scan image with an enhanced second stent based on the aligned image. As described herein, the object detection device can enhance the first stent and / or the second stent by averaging the medical scan images 302, 304, and 306 after the images have been aligned (e.g., averaging the corresponding pixel values ​​of the scan images).

[0026] Figure 4A and Figure 4B Examples of machine learning techniques that can be used for automatic detection and / or clustering of objects from input medical scan images are illustrated. Figure 4A As shown, the example ML technique can utilize one or more artificial neural networks (ANNs) 406, which can be trained to extract features from input medical scan images (e.g., 402a, 402b, etc.) and predict regions in each input medical scan image that correspond to objects of interest (e.g., 404a, 404b, 404c, 404d, etc.). The ANN 406 can include, for example, a fully convolutional neural network (CNN) with an encoder-decoder structure (e.g., a U-Net structure). Such a CNN can include multiple convolutional layers and / or one or more pooling layers. Following the convolutional layers may be batch normalization and / or linear or non-linear activations (e.g., rectified linear units or ReLU activations). Each convolutional layer can include multiple convolutional kernels or filters with corresponding weights, the values ​​of which can be learned through a training process to extract features from medical scan images 402a, 402b, etc. The features extracted by the convolutional layers can be downsampled by one or more pooling layers to obtain a representation of the features, for example, in the form of one or more feature maps. The CNN can also include one or more upper pooling layers and one or more transposed convolutional layers. By using an up-pooling layer, the CNN can upsample the features extracted from the input medical scan image and further process the upsampled features through one or more transposed convolutional layers (e.g., via multiple deconvolution operations) to derive one or more magnified or dense feature maps. These dense feature maps can then be used to predict the location of objects of interest in the input medical scan images 402a, 402b, etc. For example, the predicted location or region can be indicated using a corresponding heatmap or Gaussian response map (e.g., 408a, 408b, etc.) of the input medical scan image.

[0027] Due to the similarity between the object of interest and certain background regions in the input medical scan images 402a, 402b, etc., the object location or region (e.g., 404a-404d) can include true positives (e.g., regions 404a and 404c, which may genuinely contain the object of interest) and false positives (e.g., background regions 404b and 404d, which may have been incorrectly identified as the object of interest). To distinguish between true and false positives, a graph representation (e.g., 410) of the predicted object location or region (e.g., 404a-d) can be created, and the graph representation can be processed by a graph neural network so that both true and false object regions can be labeled by the graph neural network. The graph representation 410 (e.g., a data structure representing a graph) can include nodes and edges. Each node of the graph representation 410 may correspond to a data structure configured to store information (e.g., features) about the corresponding predicted object (e.g., a balloon label pair), and each edge of the graph representation 410 may correspond to a data structure configured to store information (e.g., similarity) between two predicted objects (e.g., represented by corresponding graph nodes) associated with the graph edge. For example, in Figure 4A In the example shown, the nodes of graph representation 410 can correspond to data structures that store information (e.g., features) about the predicted object regions 404a, 404b, 404c, and 404d, respectively, and the edges of graph representation 410 can correspond to data structures that store information about the corresponding similarities between regions 404a and 404c, between regions 404a and 404d, between regions 404b and 404c, and between regions 404b and 404d, respectively. By representing the geometric and / or spatial features of the predicted object regions 404a-404d with the nodes and edges of graph representation 410, the object detection problem can be transformed into a graph node labeling problem and solved using a graph neural network (GNN) such as a message-passing graph neural network.

[0028] Figure 4BAn example of a GNN (e.g., 412) is illustrated. The GNN can be used to determine which detection regions 404a-404d are true or false object regions and / or which pairs of detection regions can be associated with the same object (e.g., confidence scores or values ​​can be used to indicate the likelihood that certain detection regions are associated with the same object). In the example, GNN 412 can include an encoding network, a core network, and / or a decoding network. The encoding network can be trained to receive a graph representation 410 and extract corresponding features from the nodes and edges of the graph representation. The core network can be trained to estimate the corresponding states (e.g., current features) of the nodes and edges of the graph representation 410 based on the features extracted by the encoding network, while the decoding network can be trained to indicate (e.g., using node labels 414) which nodes of the graph representation 410 (e.g., detected objects) truly represent the object of interest and which nodes of the graph representation 410 falsely represent the object of interest. The decoding network can also be configured to indicate (e.g., using edge labels 416) which edges of the graph representation 410 can connect nodes associated with the same object. For example, a decoding network can use a value of 1 (e.g., or a value of "true" or a high confidence score) to label the connection between two nodes representing the same object (e.g., ...). Figure 3 302a and 304a or Figure 3 The edges (302b and 304b in the code) can be used to decode the network, and the decoding network can use a value of 0 (e.g., or a value of "false" or low confidence score) to mark edges that do not connect two nodes representing the same object.

[0029] The encoding network of GNN 412 may include a multilayer perceptron (MLP) neural network (e.g., a fully connected neural network) comprising an input layer, an output layer, and one or more hidden layers having corresponding linear or non-linear activation nodes (e.g., perceptrons), which can be trained to encode extracted features into feature representations (e.g., feature vectors). The core network of GNN 412 may include a recurrent neural network (e.g., an RNN) or a cascaded neural network, which can be trained to receive the feature representations encoded by the encoding network and determine (e.g., iteratively) the corresponding states (e.g., current features) of each graph node and each graph edge of graph representation 410 based on features of other graph nodes and graph edges. The decoding network of GNN 412 may include a fully connected neural network, which is trained to determine (e.g., label) the true / false states of graph nodes based on the corresponding final states (e.g., final features) of graph nodes and graph edges after the above iterative update process. For example, if a graph node possesses a first set of features, the decoding network can label the graph node as a true object region (e.g., truly including a pair of balloon markers), and if a graph node possesses a second set of features, the decoding network can label the graph node as a false object region (e.g., not including a pair of balloon markers). The decoding network can learn these features and / or the true / false labels associated with these features through a training process.

[0030] The graph neural networks described in this paper can be used to facilitate the clustering of objects detected in multiple medical scan images (e.g., with...). Figure 2 (The operations associated with 206). For example, the graph neural network described herein (e.g., GNN 412) can take graph representation 410 as input and can label edges of the graph representation connecting nodes associated with the same object with a first value (e.g., 1, which may indicate that an edge or a node associated with an edge is connected). The graph neural network can label other edges of the graph representation with a second value (e.g., 0, which may indicate that an edge or a node associated with an edge is disconnected). The objects can then be clustered by finding groups of connected components (e.g., one or more groups of nodes), where each group of such connected components (e.g., nodes) can represent a cluster. For example, a first cluster (e.g., a first group of nodes or a first subset of nodes) associated with a first object of interest can be identified by identifying a first set of one or more edges, which indicates that nodes connected by one or more edges belong to the first object of interest. Similarly, a second cluster (e.g., a second group of nodes or a second subset of nodes) associated with a second object of interest can be identified by identifying a second set of one or more edges, which indicates that nodes connected by one or more edges belong to the second object of interest.

[0031] In the example, the graph neural network described herein can be configured to generate edge labels (e.g., consecutive edge labels) representing the similarity of graph nodes connected by edges. The clustering described herein can be performed based on the path length associated with a set of graph nodes (e.g., representing a corresponding image region or block containing a support). This edge path length can be calculated, for example, as the sum of the corresponding labels (e.g., label values) of the edges connecting the set of graph nodes, and the edge path length can represent the similarity between sets of graph nodes. Thus, a longer edge path length (e.g., a larger number of graph edges) can indicate a higher probability that nodes on the path belong to the same object (e.g., the same support). In the example, the graph neural network described herein can be used to determine the longest edge path (e.g., among multiple candidate edge paths), and the nodes on the edge path can be grouped into a first cluster (e.g., belonging to the first support). These nodes can then be removed from the graph representation, and another longest edge path can be determined from the updated graph representation (e.g., among multiple candidate edge paths), and this longest edge path can be used to identify nodes belonging to a second cluster (e.g., the second support). This process can be repeated iteratively until a predetermined number of objects (e.g., supports) have been identified or a preset minimum path length has been reached. As mentioned above, this number and / or length can be preset by the user.

[0032] In the example, the features of nodes in graph representation 410 may include a dimension indicating whether a node is a seed node for clustering, and this dimension can be used to facilitate clustering. Seed nodes may represent nodes of interest, which can be nodes with the highest probability of being true objects of interest (e.g., confidence scores). Seed nodes may also represent objects selected by the user. Seed nodes (e.g., corresponding to a pair of scaffold or balloon markers) may be labeled as one (e.g., true / positive) in the graph representation, while other nodes in the graph representation may be labeled as zero (e.g., false / negative).

[0033] The graph neural network described herein can be trained to receive a graph representation as input and determine, based on the corresponding features of the graph nodes and / or edges in the graph representation, which other graph nodes can be associated with the same objects as a certain graph node in the input (e.g., a positive graph node representing an object of interest). The graph neural network can then classify (e.g., label) these other graph nodes as belonging to the same cluster as the positive graph node in the input. Classification can be performed iteratively for all objects of interest. For example, after identifying nodes belonging to the first scaffold (e.g., a first subset of nodes in the graph representation) in the first phase of the process, these nodes can be removed from the graph representation. Then (e.g., during the second phase of the iterative process), another node (e.g., representing a second scaffold) can be selected (e.g., labeled as a positive node) from the remaining graph nodes, and the graph neural network can repeat the above operation to identify all graph nodes (e.g., a second subset of nodes in the graph representation) that can be associated with the same scaffold (e.g., the second scaffold) as the newly selected graph node. In the example, the process can be repeated iteratively until a predetermined number of objects (e.g., scaffolds) have been identified or a predetermined cluster size (e.g., a minimum cluster size) has been reached. The quantity and / or size (e.g., M) can be preset by the user (e.g., via a user interface provided by the object detection device described herein), and the object detection device described herein can automatically select the top M objects or clusters from a sorted list of identified objects or clusters to present to the user. The sorting can be performed, for example, in descending order based on the probability that the identified objects are true classes (e.g., confidence scores).

[0034] In the example, the clustering described herein can be performed based on the edge path length (e.g., multiple graph edges) associated with a set of graph nodes (e.g., representing a corresponding image region or block containing a support). This edge path length can represent the similarity between sets of graph nodes, and thus, a longer edge path length (e.g., a larger number of graph edges) can indicate that nodes on the path have a higher probability of belonging to the same object (e.g., the same support). Therefore, the graph neural network described herein can be used to determine the longest edge path (e.g., among multiple candidate edge paths), and the nodes on the edge path can be grouped into a first cluster (e.g., belonging to the first support). Next, these nodes can be removed from the graph representation, and another longest edge path (e.g., among multiple candidate edge paths) can be determined from the updated graph representation, and this longest edge path can be used to identify nodes belonging to a second cluster (e.g., the second support). This process can be repeated iteratively until a predetermined number of objects (e.g., supports) have been identified or a preset minimum path length has been reached. As mentioned above, this number and / or length can be preset by the user.

[0035] Figure 5Example operations that can be associated with training a neural network (e.g., the various classification neural networks and / or graph neural networks described herein) to perform one or more object detection tasks described herein are illustrated. As shown, the training operation may include initializing the parameters of the neural network (e.g., weights associated with the individual filters or kernels of the neural network) at 502. The parameters may be initialized, for example, based on samples collected from one or more probability distributions or parameter values ​​from another neural network with a similar architecture. The training operation may also include feeding training data to the neural network at 504 (e.g., medical scan images including multiple objects of interest such as multiple stents) and causing the neural network to predict the outcome at 506. Depending on the function the neural network is designed to perform, the prediction outcome may be, for example, a set of image features associated with the object of interest or graph nodes, a set of graph edges representing the relationship (e.g., similarity) between two graph nodes, true / false labels of the graph nodes, etc. At 508, the prediction outcome may be compared with a gold standard to determine the loss associated with the prediction. The loss may be determined using an appropriate loss function, such as mean squared error (MSE), L1 norm, L2 norm, etc. Once the loss is determined, it may be evaluated at 510 to determine whether one or more training termination criteria have been met. For example, if the loss is below a predetermined threshold, or if the change in loss between two training iterations (e.g., between consecutive training iterations) falls below a predetermined threshold, then the training termination criterion can be considered satisfied. If it is determined at step 510 that the training termination criterion has been satisfied, then training can end. Otherwise, before training returns to step 506, the loss can be backpropagated through the neural network at step 512 (e.g., based on gradient descent associated with the loss).

[0036] For the sake of simplicity, the training steps are depicted and described in a specific order herein. However, it should be understood that training operations can occur in various orders, simultaneously, and / or with other operations not presented or described herein. Furthermore, it should be noted that not all operations that may be included in the training process are depicted and described herein, and not all exemplified operations need to be performed.

[0037] The systems, methods, and / or apparatuses described herein may be implemented using one or more processors, one or more storage devices, and / or other suitable auxiliary devices (such as display devices, communication devices, input / output devices, etc.). Figure 6This is a block diagram illustrating an example device 600 that can be configured to perform the object detection tasks described herein. As shown, device 600 may include a processor (e.g., one or more processors) 602, which may be a central processing unit (CPU), graphics processing unit (GPU), microcontroller, reduced instruction set computer (RISC) processor, application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), physical processing unit (PPU), digital signal processor (DSP), field-programmable gate array (FPGA), or any other circuitry or processor capable of performing the functions described herein. Device 600 may also include communication circuitry 604, memory 606, mass storage device 608, input device 610, and / or communication link 612 (e.g., communication bus) through which one or more components shown in the figure may exchange information.

[0038] Communication circuitry 604 can be configured to send and receive information using one or more communication protocols (e.g., TCP / IP) and one or more communication networks, including local area networks (LANs), wide area networks (WANs), the Internet, and wireless data networks (e.g., Wi-Fi, 3G, 4G / LTE, or 5G networks). Memory 606 may include a storage medium (e.g., a non-transitory storage medium) configured to store machine-readable instructions that, when executed, cause processor 602 to perform one or more functions described herein. Examples of machine-readable media may include volatile or non-volatile memory, including but not limited to semiconductor memory (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)), flash memory, etc.). Mass storage device 608 may include one or more disks, such as one or more internal hard disks, one or more removable disks, one or more magneto-optical disks, one or more CD-ROMs or DVD-ROMs, etc., on which instructions and / or data may be stored for operation of processor 602. Input device 610 may include a keyboard, mouse, voice-controlled input device, touch-sensitive input device (e.g., touch screen), etc., for receiving user input from device 600.

[0039] It should be noted that device 600 can operate as a standalone device or can be connected to other computing devices (e.g., networked or grouped) to perform the functions described herein. And even in Figure 6 Only one example of each component is shown in the figure, and those skilled in the art will understand that device 600 may include multiple instances of one or more components shown in the figure.

[0040] Although this disclosure has been described according to certain embodiments and generally associated methods, changes and variations of the embodiments and methods will be apparent to those skilled in the art. Therefore, the above description of exemplary embodiments does not limit this disclosure. Other changes, substitutions, and modifications are possible without departing from the spirit and scope of this disclosure. Furthermore, unless specifically stated otherwise, discussions using terms such as “analyze,” “determine,” “enable,” “identify,” and “modify” refer to the actions and processes of a computer system or similar electronic computing device that manipulate and transform data representing physical (e.g., electronic) quantities within the registers and memories of the computer system into other data representing physical quantities within the computer system's memory or other such information storage, transmission, or display devices.

[0041] It should be understood that the above description is intended to be illustrative and not restrictive. Many other embodiments will become apparent to those skilled in the art upon reading and understanding the above description. Therefore, the scope of this disclosure should be determined by reference to the appended claims and the full scope of their equivalents.

Claims

1. An apparatus for processing medical scan images, comprising: One or more processors, which are configured as follows: Receive multiple medical scan images, wherein each medical scan image depicts at least a first object of interest and a second object of interest; One or more artificial neural networks are used to identify a corresponding first region and a corresponding second region in each of the plurality of medical scan images, wherein the first region corresponds to the first object of interest and the second region corresponds to the second object of interest; A first cluster is determined by clustering the corresponding first regions in each of the plurality of medical scan images; a second cluster is determined by clustering the corresponding second regions in each of the plurality of medical scan images; and Generate an output medical scan image that depicts at least one of the first object of interest based on the first cluster or the second object of interest based on the second cluster.

2. The device according to claim 1, wherein, The one or more processors are configured to identify the corresponding first region in each of the plurality of medical scan images, comprising: the one or more processors being configured to determine one or more first candidate regions in each of the plurality of medical scan images and select the first region from the one or more first candidate regions; and wherein the one or more processors are configured to identify the corresponding second region in each of the plurality of medical scan images, comprising: the one or more processors being configured to determine one or more second candidate regions in each of the plurality of medical scan images and select the second region from the one or more second candidate regions.

3. The device according to claim 1, wherein, The one or more processors are configured to determine a first feature associated with the first object of interest and a second feature associated with the second object of interest based on the plurality of medical scan images, and the one or more processors are further configured to determine the first cluster based on the first feature and the second cluster based on the second feature.

4. The device according to claim 1, wherein, The one or more processors are configured to determine graph representations of the corresponding first and second regions in each of the plurality of medical scan images, and the one or more processors are further configured to determine the first cluster and the second cluster by processing the graph representations with the aid of a graph neural network (GNN).

5. The device according to claim 4, wherein, The corresponding first region and the corresponding second region in each of the plurality of medical scan images are represented as nodes in the graph representation, wherein the first cluster is determined by using the GNN to identify a first subset of nodes corresponding to the first object of interest, and wherein the second cluster is determined by using the GNN to identify a second subset of nodes corresponding to the second object of interest.

6. The device according to claim 5, wherein, The GNN is trained to extract corresponding features of the nodes in the graph representation, and to identify the first subset of nodes and the second subset of nodes based on the extracted features.

7. The device according to claim 5, wherein, The graph representation also includes multiple edges, each edge connecting corresponding pairs of nodes and indicating whether the pairs of nodes belong to the same object of interest. The first subset of nodes is identified based on a first set of edges indicating that the first subset of nodes belongs to the first object of interest, and the second subset of nodes is identified based on a second set of edges indicating that the second subset of nodes belongs to the second object of interest.

8. The device according to claim 5, wherein, The graph representation also includes multiple edges, each edge connecting corresponding pairs of nodes and representing the similarity between the pairs of nodes, wherein the one or more processors are configured to identify the first subset of nodes by identifying a first path including one or more edges connecting the first subset of nodes, and wherein the one or more processors are configured to identify the second subset of nodes by identifying a second path including one or more edges connecting the second subset of nodes.

9. The device according to claim 8, wherein, The first path is identified as the longest path among a plurality of first candidate paths associated with the first node subset, and the second path is identified as the longest path among a plurality of second candidate paths associated with the second node subset.

10. The device according to claim 5, wherein, The one or more processors are configured to identify the first subset of nodes during a first phase of the iterative process, remove the first subset of nodes from the graph representation, and identify the second subset of nodes during a second phase of the iterative process.

11. The device according to claim 1, wherein, The one or more processors are configured to generate the output medical scan image depicting at least one of the first object of interest or the second object of interest, including: the one or more processors are configured to enhance the at least one of the first object of interest or the second object of interest based on the first cluster or the second cluster.

12. The device according to claim 1, wherein, The one or more processors are configured to enhance at least one of the first object of interest or the second object of interest in the output medical scan image, including: the one or more processors are configured to: Aligning at least a subset of the plurality of medical scan images based on the first cluster or the second cluster; and The output medical scan image is generated based on the average value of the subset of the plurality of medical scan images.

13. The device according to claim 1, wherein, The plurality of medical scan images include one or more fluorescence fluoroscopic images, wherein at least one of the first object of interest or the second object of interest includes a scaffold.

14. A method for processing medical scan images, the method comprising: Receive multiple medical scan images, wherein each medical scan image depicts at least a first object of interest and a second object of interest; One or more artificial neural networks are used to identify a corresponding first region and a corresponding second region in each of the plurality of medical scan images, wherein the first region corresponds to the first object of interest and the second region corresponds to the second object of interest; A first cluster is determined by clustering the corresponding first regions in each of the plurality of medical scan images; a second cluster is determined by clustering the corresponding second regions in each of the plurality of medical scan images; and Generate an output medical scan image that depicts at least one of the first object of interest based on the first cluster or the second object of interest based on the second cluster.

Images