Image segmentation confidence determination

Abstract

Examples are disclosed for determining a confidence level associated with image segmentation. A confidence level associated with an overall image segmentation result can be determined by generating multiple individual segmentation results, each from the same image data. These examples can then aggregate the individual segmentation results to form an overall image segmentation result and measure the spread of each individual segmentation result from the overall image segmentation result. The measured spread of each individual segmentation result can then be used to determine a confidence level associated with the overall image segmentation result. This can enable a confidence level associated with the overall image segmentation result to be determined. This confidence level can be determined without requiring ground truth for comparison with the overall image segmentation result.

Description

[Technical field]

[0001] [Related Applications] This application claims priority to U.S. Application No. 16 / 581,577, filed September 24, 2019, and claims the benefit of U.S. Provisional Application No. 62 / 869,793, filed July 2, 2019, the contents of which are hereby incorporated by reference.

[0002] The present disclosure relates generally to image segmentation, and more particularly to devices, systems, and methods for determining, and in some cases indicating, a confidence level associated with image segmentation. Examples disclosed herein may be applied, for example, in medical imaging. [Background technology]

[0003] Medical imaging techniques can be used to collect image data and generate in vivo visualization of anatomical regions of interest. One such example is intravascular imaging, which can image vascular structures and lumens. For example, intravascular imaging can be used to generate images of one or more of the coronary artery lumen, coronary artery wall tissue, and devices, such as stents, at or near the coronary artery wall. These images can be useful for diagnostic purposes, such as identifying diagnostically significant characteristics of blood vessels.

[0004] However, the information collected during medical imaging is likely to include a large amount of data, some of which may be irrelevant to the purpose for which the imaging is being performed. Thus, it may be useful to further analyze the image data collected during imaging to simplify image interpretation and automate the identification of certain anatomical features and / or measurements of medically useful parameters.

[0005] As one example, image segmentation can be used to simplify interpretation and automate the identification and / or measurement of important features in image data. Image segmentation can be used to divide an image, or a set of images, into one or more regions of interest to facilitate the identification of relevant information in the images. Summary of the Invention

[0006] Exemplary examples are described herein for determining a confidence level, such as a probability of error, associated with an image segmentation result. In many applications, it is important that the image segmentation results are reliable, since the image segmentation results can be used to facilitate meaningful decisions. In a medical imaging example, the image segmentation results can be used to facilitate identification of medically relevant information, and therefore the reliability of the segmentation results can be important in accurately identifying the medically relevant information. Thus, examples disclosed herein can estimate and provide a confidence level for the segmentation results, such that a user can take the estimated confidence level into account when using the segmentation results.

[0007] An estimation of the confidence level of the segmentation result may be particularly useful in autonomous segmentation. Because manual segmentation of image data by a user may be tedious and likely time consuming, autonomous segmentation, which performs segmentation operations on image data with no or minimal user input during execution, may be desirable. However, autonomous segmentation involves a level of uncertainty. Providing an estimated confidence level associated with an automated segmentation result may enable a user to be better informed when relying on an automated segmentation result.

[0008] Notably, the examples disclosed herein can determine a confidence level associated with an image segmentation result in applications where there is no ground truth (information known to be correct) to compare the segmentation result to. In many applications, image data is generated and segmented to determine one or more unknown pieces of information. As such, the lack of a known comparison point for the segmentation result can present a difficult problem in determining a confidence level associated with the segmentation result, since the segmentation result can be a means to locate the unknown information. The examples disclosed herein can solve this problem by determining a confidence level associated with the segmentation result without needing a ground truth to compare the segmentation result to. For example, in intravascular imaging, image data of a vessel wall, a vessel lumen, and plaque can be generated and segmented to locate a previously unknown boundary between the vessel wall and the vessel lumen. Because the boundary between the vessel wall and the lumen is unknown, the examples disclosed herein can provide a segmentation result for such image data, including a confidence level associated with the segmented boundary between the vessel wall and the lumen, without using a known boundary between the vessel wall and the lumen to determine the confidence level.

[0009] In some examples disclosed herein, a confidence level associated with an image segmentation result can be determined by generating multiple individual segmentation results, each from the same image data, and measuring the spread of each individual segmentation result (e.g., from an overall segmentation result formed from the multiple segmentation results) from a result common to all segmentation results (hereafter referred to as the consensus). The measured spread of each individual segmentation result can then be used to determine a confidence level associated with a consensus, or overall, image segmentation result that is presented to a user. This can allow a confidence level associated with a consensus, or overall, image segmentation result to be determined without requiring a ground truth to compare the consensus, or overall, segmentation result.

[0010] In one example, a method includes receiving, by a processor, an image comprising a cross-sectional view of an object. The method further includes generating, by the processor, for each of a plurality of models and using the respective models, a boundary prediction for the object in the image, each boundary prediction comprising a plurality of boundary prediction portions, each model of the plurality of models comprising a unique model. The method also includes generating, by the processor, a clustered boundary prediction for the object in the image based on the boundary prediction for each model of the plurality of models, the clustered boundary prediction comprising a plurality of clustered boundary prediction portions, each corresponding to one of the respective plurality of boundary prediction portions of the boundary prediction. The method further includes, for each of the plurality of clustered boundary prediction portions, determining, by the processor, a confidence level for the respective clustered boundary prediction portion based on a comparison between the respective clustered boundary prediction portion and a corresponding boundary prediction portion of the boundary prediction. The method also includes outputting, by the processor, a graphical indicator for each of the plurality of clustered boundary prediction portions for display on the display device, each clustered boundary prediction portion having a visual characteristic indicative of a confidence level for the respective clustered boundary prediction portion.

[0011] In another example, a computing device includes at least one processor and a storage component configured to store instructions. The instructions are executable by the at least one processor to receive an image comprising a cross-sectional view of the object. The instructions are further executable by the at least one processor to generate, for each of a plurality of models, a boundary prediction for the object in the image using a respective model, each boundary prediction comprising a plurality of boundary prediction portions, each model of the plurality of models comprising a unique model. The instructions are also executable by the at least one processor to generate a clustered boundary prediction for the object in the image based on the boundary prediction for each model of the plurality of models, the clustered boundary prediction comprising a plurality of clustered boundary prediction portions, each clustered boundary prediction portion corresponding to one of the respective plurality of boundary prediction portions of the boundary prediction. The instructions are further executable by the at least one processor to determine, for each of the plurality of clustered boundary prediction portions, a confidence level for the respective clustered boundary prediction portion based on a comparison between the respective clustered boundary prediction portion and a corresponding boundary prediction portion of the boundary prediction. The instructions are further executable by the at least one processor to output a graphical indicator for each of the plurality of clustered boundary prediction portions for display on a display device, each clustered boundary prediction portion having a visual characteristic indicative of a confidence level for the respective clustered boundary prediction portion.

[0012] In another example, a non-transitory computer-readable storage medium is described having instructions stored thereon that, when executed, cause one or more processors of a computing device to receive an image comprising a cross-sectional view of an object. The instructions further cause the one or more processors to generate, for each of a plurality of models, a boundary prediction for the object in the image using a respective model, each boundary prediction comprising a plurality of boundary prediction portions, each model of the plurality of models comprising a unique model. The instructions also cause the one or more processors to generate clustered boundary predictions for the object in the image based on the boundary prediction for each model of the plurality of models, the clustered boundary predictions comprising a plurality of clustered boundary prediction portions, each clustered boundary prediction portion corresponding to one of the respective plurality of boundary prediction portions of the boundary prediction. The instructions further cause the one or more processors, for each of the plurality of clustered boundary prediction portions, to determine a confidence level for the respective clustered boundary prediction portion based on a comparison between the respective clustered boundary prediction portion and a corresponding boundary prediction portion of the boundary prediction. The instructions also cause the one or more processors to output a graphical indicator for each of the plurality of clustered boundary prediction portions for display on a display device, each clustered boundary prediction portion having a visual characteristic indicative of a confidence level for the respective clustered boundary prediction portion.

[0013] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will become apparent from the description and drawings.

[0014] The following drawings illustrate specific examples of the present invention and therefore do not limit the scope of the invention. The drawings are intended to be used in conjunction with the explanations in the following description. Examples of the invention are described below in conjunction with the accompanying drawings, in which like numerals indicate like elements. [Brief description of the drawings]

[0015] [Figure 1]FIG. 1 illustrates an example system including a camera configured to capture an image and a computing device configured to analyze the image in accordance with one or more of the techniques disclosed herein. [Diagram 2] 1 is an example cross-sectional view of a blood vessel showing a number of exemplary individual segmentation results, along with an exemplary overall segmentation result formed from the multiple individual segmentation results, in accordance with one or more of the techniques disclosed herein. [Diagram 3] 3 is an example cross-sectional view of the vessel in FIG. 2 along with an exemplary overall segmentation result with associated confidence level according to one or more of the techniques disclosed herein. [Figure 4] 4 is a diagram showing an example of how clustering of individual segmentation results can be used to determine confidence levels in accordance with one or more of the techniques disclosed herein. The diagram in FIG. 4 illustrates relatively strong clustering around a particular value. [Diagram 5] FIG. 5 is another diagram showing an example of how clustering of individual segmentation results can be used to determine confidence levels. The diagram in FIG. 5 illustrates that there is no strong clustering around any particular value. [Figure 6] FIG. 1 is a flow diagram illustrating an example process for predicting clustered boundaries of objects in an image according to one or more of the techniques described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] The following detailed description is exemplary in nature and provides some illustrative examples and examples. Those skilled in the art will recognize that many of the examples described have a variety of suitable alternatives.

[0017] Examples disclosed herein can determine a confidence level associated with an image segmentation result. In particular, examples disclosed herein can use multiple individual segmentation results, each from the same image data. Such examples can apply the statistical properties of the integration process to estimate the confidence of the segmentation performance by measuring the spread of each individual segmentation result from the consensus of all segmentation results (i.e., from the overall segmentation result formed from the multiple individual segmentation results). This estimated confidence of the image segmentation result can provide a user with additional information on how to use the segmentation result without knowing the ground truth.

[0018] Figure 1 illustrates an example system including a camera configured to capture images and a computing device configured to analyze the images in accordance with one or more of the techniques disclosed herein. Although Figure 1 illustrates only one particular example of a computing device 2, many other examples of computing device 2 may be used in other examples and may include a subset of the components included in the example computing device 2, as well as additional components not illustrated in Figure 1. For example, rather than being a separate component, camera 20 may be incorporated into computing device 2 or may be operatively connected in some manner other than via a wireless connection.

[0019] As shown in the example of FIG. 1, computing device 2 includes one or more processors 4, one or more input / output (I / O) components 6, one or more communication units 14, and one or more storage components 8. The storage components 8 of computing device 2 include a clustering module 10 and a predictive model 12. For purposes of this disclosure, the predictive models can be analogous to partitioning operations, with each model describing a different partitioning operation. A communication channel (not shown) can interconnect (physically, communicatively, and / or functionally) each of components 4, 6, 8, and 14 for inter-component communication. In some examples, the communication channel can include a system bus, a network connection, an inter-process communication data structure, or any other method for communicating data.

[0020] One or more communication units 14 of the computing device 2 can communicate with external devices over one or more wired and / or wireless networks by transmitting and / or receiving network signals on one or more networks. Examples of communication units 14 include network interface cards (such as, for example, Ethernet cards, etc.), optical transceivers, radio frequency transceivers, GPS receivers, or any other type of device capable of sending and / or receiving information. Other examples of communication units 14 can include shortwave radios, cellular data radios, wireless network radios, as well as universal serial bus (USB) controllers. The communication units 14 can include wireless interfaces configured to operate according to wireless protocols such as Wi-Fi, Bluetooth, or LTE.

[0021] The one or more I / O components 6 can include input components capable of receiving input. Examples of input include tactile, audio, and video input. In one example, the I / O components 6 of the computing device 2 include an intravascular ultrasound catheter, a presence sensing input device (e.g., a touch-sensitive screen, PSD), a mouse, a keyboard, a voice response system, a video camera, a microphone, or any other type of device for detecting input from a human or machine. In some examples, the I / O components 6 can include one or more sensor components, one or more position sensors (GPS components, Wi-Fi components, cellular components), one or more temperature sensors, one or more motion sensors (e.g., accelerometers, gyros), one or more pressure sensors (e.g., barometers), one or more ambient light sensors, and one or more other sensors (e.g., microphones, cameras, infrared proximity sensors, hygrometers, etc.).

[0022] The I / O components 6 may also include one or more output components that generate output. Examples of output include tactile, audible, and video output. In one example, the I / O components 6 of the computing device 2 include a PSD, a sound card, a video graphics adapter card, a speaker, a cathode ray tube (CRT) monitor, a liquid crystal display (LCD), or any other type of device for generating output to a human or machine.

[0023] One or more processors 4 may implement functions and / or execute instructions associated with computing device 2. Examples of processors 4 include application processors, display controllers, auxiliary processors, one or more sensor hubs, and any other hardware configured to function as a processor, processing unit, or processing device. Modules 10 and models 12 may be operable by processor 4 to perform various actions, operations, or functions of computing device 2. For example, processor 4 of computing device 2 may retrieve and execute instructions stored in storage component 8 that cause processor 4 to perform operations of modules 10 and models 12. The instructions, when executed by processor 4, may cause computing device 2 to store information in storage component 8.

[0024] One or more storage components 8 within computing device 2 can store information for processing during operation of computing device 2 (e.g., computing device 2 can store data accessed by modules 10 and predictive models 12 during execution on computing device 2). In some examples, storage components 8 are temporary memory, meaning that the primary purpose of storage components 8 is not long-term storage. Storage components 8 on computing device 2 can be configured as volatile memory for short-term storage of information and thus do not retain stored contents when power is removed. Examples of volatile memory include random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), and other forms of volatile memory known in the art.

[0025] In some examples, the storage component 8 also includes one or more computer readable storage media. In some examples, the storage component 8 includes one or more non-transitory computer readable storage media. The storage component 8 can be configured to store a larger amount of information than is typically stored by volatile memory. The storage component 8 can also be configured to store information for long periods of time as non-volatile memory space and retain information even after power on / off cycles. Examples of non-volatile memory include magnetic hard disks, optical disks, floppy disks, flash memory, or in the form of electrically programmable memory (EPROM) or electrically erasable programmable memory (EEPROM). The storage component 8 can store program instructions and / or information (e.g., data) associated with the module 10 and the predictive model 12. The storage component 8 can include a memory configured to store data or other information associated with the module 10 and the predictive model 12.

[0026] The camera 20 may be any image capture device capable of capturing an image of an object. In some examples, the camera 20 may be an intravascular ultrasound catheter configured to capture an image including, for example, a cross-sectional view of a coronary artery of the patient 70. However, in other examples, the camera 20 may be any other type of camera configured to capture a cross-sectional image of an object. Thus, although throughout this disclosure, the measurement of the boundary of an object within a blood vessel is described after capturing an intravascular ultrasound image, the same techniques can be applied to any image in which the boundary of an object within the image is to be defined. For example, similar techniques can be applied to natural images (e.g., color images, grayscale images), different medical modality images (e.g., images captured through magnetic resonance imaging (MRI), x-ray, computed tomography (CT), positron emission tomography (PET), single photon emission computed tomography (SPECT), and ultrasound), and non-natural images (e.g., thermal images, ultraviolet images). Furthermore, although the techniques described herein are described with respect to cross-sectional regions, the same techniques can be applied to any object within an image where the boundaries of such object can be located.

[0027] In accordance with the techniques of the present disclosure, the clustering module 10 can receive images including cross-sectional views of objects. The clustering module 10 can receive images from a camera, such as the camera 20. The camera 20 can be located external to the clustering module 10, in which case the clustering module 10 receives the images via the communication unit 14. In other examples, the camera 20 can be incorporated within the computing device 2. The images can be any images including objects with known or unknown boundaries within the images, including cross-sections of the objects. For example, the camera 20 can be an intravascular ultrasound device configured to capture images of veins or arteries of the patient 70.

[0028] For each of the multiple models, the clustering module 10 can generate a boundary prediction for an object in the image using the respective model. The boundary prediction can also be referred to as an image segmentation or an individual segmentation result. Each model of the multiple models can be a unique model. Furthermore, each boundary prediction can include multiple boundary prediction portions. For example, the multiple models can include any type of machine learning or deep learning model, pattern recognition model, computer vision model, or parameter modification model, among other models. For example, the multiple models can include a support vector machine (SVM) model, a UNet model, a random forest model, and an edge detection computer vision model such as Sobel filtering, and variations thereof with different parameters. Finally, the multiple models can include more than nine or more separate models, where each of the separate models can be used to predict the boundary of an object in the image.

[0029] The clustering module 10 can generate a clustered boundary prediction of the object in the image based on the boundary prediction for each model of the multiple models. The clustered boundary prediction can also be referred to as an overall segmentation result. The clustering module 10 can determine the clustered boundary prediction using a merging technique along with a clustering metric. In some examples, the clustering metric for the clustered boundary prediction can be a median of the individual image segmentation results (or corresponding boundary prediction portions of the boundary predictions). In some examples, the clustering metric for the clustered boundary prediction can be an average boundary as averaged from each of the corresponding boundary prediction portions of the boundary predictions of the multiple models. In other examples, the clustered boundary prediction can be a weighted clustered boundary prediction for the object in the image. For example, the clustering module 10 can determine a weight for each model for each of the multiple models. The clustering module 10 can then apply a weight to each of the boundary predictions based on the corresponding boundary prediction portion of the boundary prediction and a weight for each model of the multiple models, and average the corresponding boundary prediction portion of the boundary prediction together with the weight to generate weighted clustered boundary predictions.

[0030] The clustered boundary prediction may include a plurality of clustered boundary prediction portions, each corresponding to one of the plurality of boundary prediction portions of the boundary prediction. For example, if each boundary prediction is divided into 100 boundary prediction portions, the clustered boundary prediction may be similarly divided into 100 clustered boundary prediction portions, with corresponding portions having similar positions with respect to the object in the image. For each of the plurality of clustered boundary prediction portions of the clustered boundary prediction, the clustering module 10 may determine a confidence level for the respective clustered boundary prediction portion based on a comparison between the respective clustered boundary prediction portion and the corresponding boundary prediction portion of the boundary prediction.

[0031] In some examples, when determining the confidence level, the clustering module 10 can determine a spread for each model based on a comparison between the clustered boundary prediction portion and the corresponding boundary prediction portion of the boundary prediction for each of the multiple models. One example of such a spread can be a z-score, although any measure of deviation between two values ​​can be used for purposes of the techniques disclosed herein. The clustering module 10 can then determine a confidence level for each clustered boundary prediction portion based on the spread for each of the corresponding boundary prediction portions of the boundary prediction.

[0032] As statistically defined, a z-score is equivalent to the number of standard deviations a particular value is away from the mean. The clustering module 10 can determine a z-score for each model by first using the clustered boundary prediction portion and the corresponding boundary prediction portion of the boundary prediction to create a prediction distribution, such as the prediction distributions of FIGS. 4 and 5, where the clustered boundary prediction portion is the mean of the prediction distribution. This prediction distribution can be as simple as a dot plot along a line, or a more complex distribution, such as a bell curve. The clustering module 10 can then determine a standard deviation for the prediction distribution. The standard deviation can be calculated according to typical rules in mathematics, or can be calculated by unique adjustments to typical algorithms for calculating standard deviations. The clustering module 10 can then determine a z-score for each corresponding boundary prediction portion of the boundary prediction as the number of standard deviations that each corresponding boundary prediction portion is away from the clustered boundary prediction portion.

[0033] In some examples, one or more of the models may generate outlier-like results where the model's prediction is clearly an error for at least that clustered boundary prediction portion of the clustered boundary prediction. In practice, the clustering module 10 may determine that the spread of the first boundary prediction for the first corresponding boundary prediction portion exceeds a deviation threshold for the prediction distribution, where the first boundary prediction is associated with a first model of the plurality of models. In some examples, the deviation threshold may be a user-defined threshold, where other examples may include a default or predefined threshold. In response to making this determination, the clustering module 10 may remove the first model from the plurality of models to generate a subset of models that includes all other models from the plurality of models except the first model (e.g., the subset of models includes clustered boundary prediction portions that are not outliers of other individual segmentation results). The clustering module 10 may then recalculate at least each clustered boundary prediction portion of the clustered boundary prediction using only the subset of models, without allowing the error model to change the results of the algorithm.

[0034] The clustering module 10 can output a representation including a graphical indicator for each of the plurality of clustered boundary prediction portions of the clustered boundary prediction of at least the entire segmentation result for display on a display device such as the input / output component 6. Each clustered boundary prediction portion of the plurality of clustered boundary prediction portions has a visual feature or visual indicator indicating a confidence level for the respective clustered boundary prediction portion. The visual feature can be any distinct visual feature that can indicate a confidence level of the particular clustered boundary prediction portion at issue. Examples of visual features include line color, fill pattern, line thickness, line division, text indicator, graphical icon, or any combination thereof. For example, when the visual feature includes line color, a green indicator for a clustered boundary prediction portion of the clustered boundary prediction can indicate a high confidence for that clustered boundary prediction portion of the clustered boundary prediction. Conversely, clustered boundary prediction portions of the clustered boundary prediction with a low confidence level can be shown as red in the output, and portions of the clustered boundary prediction with a medium confidence level can be shown as yellow. This can be a spectrum of colors, varying from a green spectrum to a red spectrum as the confidence level decreases.

[0035] FIG. 2 illustrates an example cross-section of a blood vessel 100. The cross-section of the blood vessel 100 includes a number of exemplary individual segmentation results 105A-105E along with an exemplary overall segmentation result 110. The overall segmentation result 110 can be formed from one or more of the individual segmentation results 105A-105E. For example, one or more of the individual segmentation results 105A-105E can be combined to form the overall segmentation result 110. In the illustrated example, the overall segmentation result 110 is formed from each of the individual segmentation results 105A-105E. Although the illustrated example is described with respect to five individual segmentation results 105A-105E, other numbers of individual segmentation results (e.g., six or more, or four or less) can be used in other examples within the scope of the present disclosure.

[0036] 2, a cross-sectional view of a blood vessel 100 includes a blood vessel wall 115 that defines a blood vessel lumen 120. Multiple, independent segmentation operations are performed on the same image data. In this example, multiple, independent segmentation operations are performed on image data corresponding to the cross-sectional view of the blood vessel 100. In the example shown, multiple, independent segmentation operations (or models) are performed to identify / predict an interface or boundary between the blood vessel wall 115 and the blood vessel lumen 120.

[0037] Each of the multiple, independent segmentation operations may result in an individual segmentation result. In the illustrated example, each independent segmentation operation may produce a corresponding individual segmentation result 105A-105E. For example, a first segmentation operation may produce a first individual segmentation result 105A, a second segmentation operation may produce a second individual segmentation result 105B, a third segmentation operation may produce a third individual segmentation result 105C, a fourth segmentation operation may produce a fourth individual segmentation result 105D, and a fifth segmentation operation may produce a fifth individual segmentation result 105E. Each segmentation operation may be different. For example, in the illustrated example, each segmentation operation may be a different segmentation algorithm for identifying an interface between the vessel wall 115 and the vessel lumen 120. And, each of the different segmentation algorithms may be performed on the same image data corresponding to a cross-sectional view of the blood vessel 100. Thus, each of the individual segmentation results 105A-105E generated by these different segmentation operations can include one or more regions whose locations are similar to the locations of other regions and one or more regions whose locations are substantially different from the locations of other regions in approximating the interface between the vascular wall 115 and the vascular lumen 120.

[0038] 2, the cross-sectional view of blood vessel 100 can include one or more regions 125A, 125B, 125C, in which individual segmentation results 105A-105E identify similar locations as interfaces between vessel wall 115 and vessel lumen 120. In regions 125A, 125B, and 125C, two or more of the individual segmentation results 105A-105E generally overlap, and thus identify similar locations as interfaces between vessel wall 115 and vessel lumen 120. For example, as shown in FIG. 2, in each of regions 125A, 125B, and 125C, two or more (e.g., each) of the individual segmentation results 105A-105E are considered to be within a predetermined range of each other, and thus generally overlap, and identify similar locations as interfaces between vessel wall 115 and vessel lumen 120.

[0039] 2, the cross-sectional view of blood vessel 100 can include one or more regions 130A, 130B, 130C, in which individual segmentation results 105A-105E identify substantially different locations as interfaces between vessel wall 115 and vessel lumen 120. In regions 130A, 130B, and 130C, two or more of the individual segmentation results 105A-105E are generally separate and therefore identify substantially different locations as interfaces between vessel wall 115 and vessel lumen 120. For example, as shown in FIG. 2, in each of regions 130A, 130B, and 130C, two or more (e.g., each) of the individual segmentation results 105A-105E are outside a predetermined range of each other and therefore are generally separate and would be considered to identify substantially different locations as interfaces between vessel wall 115 and vessel lumen 120.

[0040] As noted, two or more of the individual segmentation results 105A-105E can be used to form the overall segmentation result 110. For example, two or more of the individual segmentation results 105A-105E can be combined to form the overall segmentation result 110. In the example shown, the median of the combined individual segmentation results 105A-105E was used to form the overall segmentation result 110 shown in FIG. 2. In some examples, each of the individual segmentation results 105A-105E can be used to form the overall segmentation result 110. In other examples, one or more outlier individual segmentation results 105A-105E can be discarded, and the remaining individual segmentation results 105A-105E can be used to form the overall segmentation result 110. For example, if one individual segmentation result differs from each of the other individual segmentation results by more than a predetermined outlier threshold, that individual segmentation result can be excluded from use in the overall segmentation result 110 information. In some examples, when a portion of an individual segmentation result differs from corresponding portions of other individual segmentation results by more than a predetermined outlier threshold, this portion of the individual segmentation result can be excluded from use in the information of the overall segmentation result 110 (e.g., such that the corresponding portion of the overall segmentation result 110 is formed from corresponding non-outlier portions of the other individual segmentation results).

[0041] 3 illustrates an example of a cross-section of a blood vessel 100 including an overall segmentation result 110 presented with an associated confidence level. As illustrated in FIG. 3, the overall segmentation result 110 is presented with an associated confidence level by including different indicia in areas of the overall segmentation result 110 where there are different probabilities of error at the location of the overall segmentation result 110. In the illustrated example, visual indicia are associated with the overall segmentation result 110 to convey areas of the overall segmentation result 110 where there are different probabilities of error at the location of the overall segmentation result 110. For example, the overall segmentation result 110 may include one type of visual indicia associated with portions of the overall segmentation result 110 where there is a relatively low probability of error at the location of the overall segmentation result 110 and another different type of visual indicia associated with portions of the overall segmentation result 110 where there is a relatively high probability of error at the location of the overall segmentation result 110.

[0042] In the illustrated example, the portions of the overall segmentation result 110 include distinct indices 205 , 210 , and 215 .

[0043] The indicator 205 is associated with a portion of the overall segmentation result 110 where the probability of error at the location of the overall segmentation result 110 is relatively low. A confidence level can be considered relatively high where the probability of error at the location of the overall segmentation result 110 is relatively low. In the illustrated example, the portion of the overall segmentation result 110 where the probability of error at the location of the overall segmentation result 110 is relatively low and therefore associated with the indicator 205 corresponds to regions 125A, 125B, and 125C where the individual segmentation results 105A-105E identify similar locations as interfaces between the vessel wall 115 and the vessel lumen 120.

[0044] The indicator 215 is associated with a portion of the overall segmentation result 110 where there is a relatively high probability of error at the location of the overall segmentation result 110. Where the probability of error at the location of the overall segmentation result 110 is relatively high, the confidence level can be considered relatively low. In the illustrated example, the portion of the overall segmentation result 110 where the probability of error at the location of the overall segmentation result 110 is relatively high and therefore associated with the indicator 215 corresponds to regions 130A and 130C where the individual segmentation results 105A-105E identify substantially different locations as the interface between the vessel wall 115 and the vessel lumen 120.

[0045] In some examples, the indicator 210 may also be included in and associated with a portion of the overall segmentation result 110 where the probability of error at the location of the overall segmentation result 110 is relatively medium. Where the probability of error at the location of the overall segmentation result 110 is relatively medium, the confidence level may be considered relatively medium. For example, the indicator 210 may be included in and associated with a portion of the overall segmentation result 110 where the probability of error at the location of the overall segmentation result 110 transitions between a relatively high probability of error and a relatively low probability of error. In the illustrated example, the portion of the overall segmentation result 110 where the probability of error at the location of the overall segmentation result 110 is relatively medium and thus associated with the indicator 210 may correspond to the region in FIG. 2 where the individual segmentation results 105A-105E transition between identifying a substantially different location as an interface between the vascular wall 115 and the vascular lumen 120 and identifying a substantially similar location as an interface between the vascular wall 115 and the vascular lumen 120.

[0046] As noted, in the illustrated example, the indices 205, 210, 215 can be visual indices presented on a displayed image of a cross-section of the blood vessel 100. As shown, the indices 205, 210, 215 can be located on a line representing the overall segmentation result 110. The indices 205, 210, 215 can be different visual indices presenting different visual characteristics so that a user can visually distinguish portions of the overall segmentation result 110 that are associated with different confidence levels (e.g., probability of error). For example, the different visual indices 205, 210, 215 can each be a different color. For example, the visual indicators 205, 210, 215 may take the form of a color gradient, such that the visual indicator 205 is displayed in a first portion of the color gradient (e.g., an overall green spectrum of colors), the visual indicator 210 is displayed in a second subsequent portion of the color gradient (e.g., an overall yellow spectrum of colors), and the visual indicator 215 is displayed in a third subsequent portion of the color gradient (e.g., an overall red spectrum of colors). In another example, the visual indicators 205, 210, 215 may take the form of different line patterns in the line representing the overall segmentation result 110. For example, the different visual indicators 205, 210, 215 may each be different dotted line patterns or a combination of different solid and dotted line patterns such that a user can visually distinguish portions of the overall segmentation result 110 that are associated with different confidence levels (e.g., probabilities of error). The different visual indicators 205, 210, 215 may differ with respect to other visual features.

[0047] Various techniques can be used to determine a confidence level associated with the overall segmentation result 110. Below is described one particular example for determining a confidence level associated with the overall segmentation result 110. Other techniques can also be used within the scope of this disclosure to determine a confidence level associated with the overall segmentation result 110.

[0048] One typical technique runs multiple, different partitioning algorithms to obtain corresponding individual partitioning results, then applies a clustering metric to these individual partitioning results, and converts the clustering metric into a problem-specific probability measure.

[0049] In this exemplary technique, a combined prediction function can be exploited to obtain clusters. Combined generally means integrating the results (e.g., individual segmentation results) of several models on the same data (e.g., image data corresponding to cross-sections of blood vessels). Once integrated, a metric is used to obtain a final single result (e.g., an overall segmentation result).

[0050] In this exemplary technique, several possible boundary locations (e.g., individual segmentation results) can be obtained by merging the results together with a final calculated boundary (e.g., overall segmentation result). The more results with a single boundary or nearby boundaries, the higher the confidence that the selected boundary is correct. If most results are different, it can be concluded that the boundaries are unclear from the image data, and therefore the confidence of selecting the correct boundary at the corresponding location is low.

[0051] In a typical technology integration process, the image segmentation confidence level can be determined by measuring the spread of each algorithm's results from the consensus of all algorithms (e.g., by measuring the spread of each individual segmentation result from the overall segmentation result). Any type of measuring statistical information including the results of each algorithm and any type that defines the consensus of all algorithms' results can be used to estimate the confidence level.

[0052] For example, in this exemplary technique, a measure of clustering can be determined using the standard deviation of the predictions (e.g., individual split results). A high standard deviation can be correlated with low confidence, while a low standard deviation can be correlated with high confidence. In some cases, an outlier point, or multiple outlier points, such as from a single individual split result, can be removed to avoid skewing the results due to a small number of low-level predictions.

[0053] In this exemplary technique, once the standard deviation of the prediction (e.g., individual split results) is determined, the standard deviation can be converted into a confidence measure. For example, to do so, a minimum and maximum standard deviation can be empirically determined to define a range. For example, a standard deviation less than 1 can be considered a perfect prediction, and a standard deviation greater than 10 can be considered pure guesswork. After defining the range with the minimum and maximum standard deviation, the determined standard deviation can be normalized to provide a probability of error. The inverse of this probability of error value can be considered as a confidence level for the correct final calculated boundary (e.g., overall split result).

[0054] Such techniques can be useful because they allow one to determine the confidence of a segmentation result (e.g., an overall segmentation result) without comparing each prediction (e.g., each segmentation result) to ground truth (e.g., to the actual boundary between two items in the image data). Although this disclosure describes this technique in a typical intravascular imaging application, this disclosure can be applied to other imaging applications. These other imaging applications can include, for example, magnetic resonance imaging (MRI) image data, computed tomography (CT) image data, visible light image data, and thermal image data.

[0055] Figures 4 and 5 show schematic diagrams of an example of a method for determining a confidence level using clustering of individual split results. The diagram in Figure 4 illustrates a relatively strong clustering around a particular value, while the diagram in Figure 5 illustrates a broader distribution with no strong clustering around any particular value. In the example shown in Figures 4 and 5, the method attempts to predict a value located at 2, and each point plotted in Figures 4 and 5 represents an independent prediction (e.g., an individual split result) from the combined one.

[0056] As shown in FIG. 4, most of the predictions (e.g., individual split results) are relatively accurate, but the single prediction shown at approximately a value of 5 is relatively inaccurate because it is farther away from a value of 2 than any of the other predictions. Thus, in the example of FIG. 4, there is a relatively strong clustering around the value of 2. In one example, the single prediction shown at approximately a value of 5 can be considered an outlier because it falls outside a predetermined range from the other predictions and can be removed from further use. FIG. 4 visually appears to indicate that after consolidation, there is a relatively high confidence level in the prediction of the final value (e.g., the overall split result).

[0057] As shown in Figure 5, the individual predictions (e.g., individual segmentation results) are relatively spread out, so the confidence level in any single prediction is relatively low. In this example, there is no strong clustering around any particular value. Therefore, after consolidation, it can be determined that there is a relatively low confidence value associated with each individual prediction shown here.

[0058] As shown in the examples shown in Figures 4 and 5, evaluating predictive clustering can have a strong correlation with the confidence level of the prediction, without the presence of ground truth (e.g., in these examples, without knowing that the true value is 2). Predictive clustering can be evaluated using any common clustering metric, such as standard deviation as described elsewhere herein.

[0059] To translate the clustering metric into a confidence level, a mapping can be determined between the clustering metric and the probability. In many instances, the relationship between the clustering metric and the error can be systematically estimated as appropriate for a particular application. Using the example of standard deviation, an appropriate standard deviation can be empirically selected to explain most of the clustering errors. After establishing such a relationship, the mapping can be approximated as appropriate for various applications. For example, one example mapping can be a simple normalization metric after empirically determining the maximum acceptable standard deviation, as detailed herein above. Such a mapping can be non-linear if applicable to a particular application.

[0060] 6 is a flow diagram illustrating an example process for predicting clustered boundaries of objects in an image in accordance with one or more of the techniques described herein. The operations of FIG. 6 may be performed by one or more processors of a computing device, such as computing device 2 of FIG. 1. For purposes of example only, FIG. 6 is described below in the context of computing device 2 of FIG. 1.

[0061] In accordance with the techniques of this disclosure, the computing device 2 receives an image comprising a cross-sectional view of an object (602). For each of the multiple models, the computing device 2 generates a boundary prediction for the object in the image using the respective model (604). Each model of the multiple models is a unique model. Furthermore, each boundary prediction has a plurality of boundary prediction portions. The computing device 2 generates a clustered boundary prediction for the object in the image based on the boundary prediction for each model of the multiple models. The clustered boundary prediction includes a plurality of clustered boundary prediction portions, each corresponding to one of the plurality of boundary prediction portions of each of the boundary predictions. For each of the plurality of clustered boundary prediction portions, the computing device 2 determines a confidence level for the respective clustered boundary prediction portion based on a comparison between the respective clustered boundary prediction portion and a corresponding boundary prediction portion of the boundary prediction (608). The computing device 2 outputs a graphical indicator for each of the plurality of clustered boundary prediction portions for display on the display device (610). Each clustered boundary prediction portion has a visual feature indicative of a confidence level for the respective clustered boundary prediction portion.

[0062] It should be appreciated that, in some examples, any certain acts or events of the techniques described herein can be performed in a different sequence, can be added, combined, or omitted altogether (e.g., not all of the acts or events described are necessary to practice the techniques.) Furthermore, in some examples, acts or events can be performed simultaneously rather than sequentially, for example, through multithreading, interrupt handling, or multiple processors.

[0063] In one or more examples, the functions described can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored on or transmitted over a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. A computer-readable medium can include a computer-readable storage medium corresponding to a tangible medium, such as a data storage medium, or a communication medium including any medium that facilitates transfer of a computer program from one place to another, for example according to a communication protocol. In this manner, a computer-readable medium can generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium, such as a signal or carrier wave. A data storage medium can be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described in this disclosure. A computer program product can include a computer-readable medium.

[0064] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM, or other optical disk storage devices, magnetic disk storage devices, or other magnetic storage components, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection can be properly referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microphone waves, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microphone waves, are included in the definition of media. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, and computer-readable storage media refers to non-transitory, tangible storage media. As used herein, disk and disc include compact discs (CDs), laser discs, optical disks, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically and discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer readable media.

[0065] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Thus, the term "processor" as used herein may refer to any of the foregoing structures, or any other structure suitable for implementation of the techniques described herein. In addition, in some aspects, the functionality described herein may be provided in dedicated hardware and / or software modules configured for encoding and decoding, or for incorporation in a combined codec (encoder / decoder). Also, the techniques may be implemented entirely in one or more circuit or logic elements.

[0066] The techniques of this disclosure can be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units are described in this disclosure to emphasize functional aspects of an apparatus configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. Rather, the various units can be combined in a codec hardware unit, as described above, or can be provided by a collection of interoperating hardware units, including one or more processors in cooperation with appropriate software and / or firmware, as described above.

[0067] Various examples have been described with respect to certain disclosed embodiments. The examples are presented for purposes of illustration and not limitation. Those skilled in the art will recognize that various changes, adaptations, and modifications may be made without departing from the scope of the invention. Some aspects of the invention are described below. [Aspect 1] 1. A method comprising: receiving, by a processor, an image comprising a cross-sectional view of an object; generating, by the processor and using the respective model, a boundary prediction for the object in the image for each of a plurality of models, wherein each boundary prediction comprises a plurality of boundary prediction portions, and each model of the plurality of models comprises a unique model; generating, by the processor and based on the boundary prediction for each model of the plurality of models, a clustered boundary prediction for the object in the image, wherein the clustered boundary prediction comprises a plurality of clustered boundary prediction portions, each corresponding to one of the plurality of boundary prediction portions of a respective one of the boundary predictions; for each of the plurality of clustered boundary prediction portions, determining, by the processor, a confidence level for the respective clustered boundary prediction portion based on a comparison between the respective clustered boundary prediction portion and the corresponding boundary prediction portion of the boundary prediction; outputting, by the processor and for display on a display device, a graphical indicator for each of the plurality of clustered boundary prediction portions, each of the plurality of clustered boundary prediction portions having a visual characteristic indicative of the confidence level for the respective clustered boundary prediction portion. [Aspect 2] 2. The method of claim 1, wherein the plurality of models includes one or more of a machine learning model, a deep learning model, a pattern recognition model, a computer vision model, and a parameter modification model. [Aspect 3] Determining the confidence level comprises: determining, for each of the plurality of models, by the processor and based on a comparison between the respective clustered boundary prediction portions and the corresponding boundary prediction portions of the boundary prediction, a spread for the respective model; 2. The method of claim 1, further comprising: determining, by the processor, the confidence level for each of the clustered boundary prediction portions based on the spread for each of the corresponding boundary prediction portions of the boundary prediction. [Aspect 4] Determining the spread for each of the corresponding boundary prediction portions of the boundary prediction includes: creating, by the processor, a prediction distribution using each of the clustered boundary prediction portions and the corresponding boundary prediction portions of the boundary prediction, the prediction distribution being the mean of the clustered boundary prediction portion; determining, by the processor, a standard deviation for each of the corresponding boundary prediction portions of the boundary prediction; 4. The method of claim 3, further comprising: determining, by the processor, a z-score for each of the corresponding boundary prediction portions of the boundary predictions as the number of standard deviations that each corresponding boundary prediction portion is away from the clustered boundary prediction. [Aspect 5] determining, by the processor, that the spread for a first corresponding boundary prediction portion of a first boundary prediction exceeds a deviation threshold for the prediction distribution, where the first boundary prediction is associated with a first model of the plurality of models; removing, by the processor, the first model from the plurality of models to generate a subset of models; 4. The method of embodiment 3, further comprising recalculating, by the processor, at least each of the clustered boundary prediction portions using only a subset of the models. [Aspect 6] 6. The method of embodiment 5, wherein the deviation threshold comprises one of a user defined threshold or a default threshold. [Aspect 7] 2. The method of claim 1, wherein the visual characteristics comprise one or more of a line color, a fill pattern, a line thickness, a line division, a text indicator, or a graphic icon. [Aspect 8] for each of the plurality of models, determining, by the processor, a weight for the respective model; 2. The method of claim 1, further comprising: generating, by the processor, the clustered boundary prediction as a weighted clustered boundary prediction for the object in the image based on the corresponding boundary prediction portion of the boundary prediction and the weight for each model of the plurality of models. [Aspect 9] 2. The method of embodiment 1, wherein the plurality of models comprises three or more separate models. [Aspect 10] 1. A computing device, comprising: At least one processor; a storage component configured to store instructions, the instructions comprising: receiving an image comprising a cross-sectional view of an object; for each of a plurality of models, generating a boundary prediction for the object in the image using the respective model, wherein each boundary prediction comprises a plurality of boundary prediction portions, and each model of the plurality of models comprises a unique model; generating a clustered boundary prediction for the object in the image based on the boundary prediction for each model of the plurality of models, wherein the clustered boundary prediction comprises a plurality of clustered boundary prediction portions, each corresponding to one of the plurality of boundary prediction portions of a respective one of the boundary predictions; for each of the plurality of clustered boundary prediction portions, determining a confidence level for the respective clustered boundary prediction portion based on a comparison between the respective clustered boundary prediction portion and the corresponding boundary prediction portion of the boundary prediction; 11. A computing device executable by the at least one processor to output, for display on a display device, a graphical indicator for each of the plurality of clustered boundary prediction portions, each of the clustered boundary prediction portions having a visual characteristic indicative of the confidence level for the respective clustered boundary prediction portion. [Aspect 11] 11. The computing device of embodiment 10, wherein the plurality of models includes one or more of a machine learning model, a deep learning model, a pattern recognition model, a computer vision model, and a parameter modification model. [Aspect 12] The instructions executable by the one or more processors to determine the trust level include: determining, for each of the plurality of models, a spread for the respective model based on a comparison between the respective clustered boundary prediction portion and the corresponding boundary prediction portion of the boundary prediction; 11. The computing device of claim 10, further comprising instructions executable by the one or more processors to determine the confidence level for each of the clustered boundary prediction portions based on the spread for each of the corresponding boundary prediction portions of the boundary prediction. [Aspect 13] The instructions executable by the one or more processors to determine the spread for each of the corresponding boundary prediction portions of the boundary prediction include: using each of the clustered boundary prediction portions and the corresponding boundary prediction portions of the boundary prediction to create a prediction distribution in which the clustered boundary prediction portion is the mean of the prediction distribution; determining a standard deviation for each of the corresponding boundary prediction portions of the boundary prediction; 13. The computing device of claim 12, further comprising instructions executable by the one or more processors to determine a z-score for each of the corresponding boundary prediction portions of the boundary prediction as the number of standard deviations that each corresponding boundary prediction portion is away from the clustered boundary prediction portion. [Aspect 14] The instructions may further include: determining that the spread for a first corresponding boundary prediction portion of a first boundary prediction exceeds a deviation threshold for the prediction distribution, where the first boundary prediction is associated with a first model of the plurality of models; removing the first model from the plurality of models to generate a subset of models; 13. The computing device of embodiment 12, wherein the computing device is executable by the one or more processors to recalculate at least each of the clustered boundary prediction portions using only a subset of the models. [Aspect 15] 15. The computing device of embodiment 14, wherein the deviation threshold comprises one of a user defined threshold or a default threshold. [Aspect 16] 11. The computing device of aspect 10, wherein the visual characteristics comprise one or more of a line color, a fill pattern, a line thickness, a line division, a text indicator, or a graphic icon. [Aspect 17] The instructions may further include: For each of the plurality of models, determining a weight for the respective model; 11. The computing device of claim 10, wherein the computing device is executable by the one or more processors to generate the clustered boundary prediction as a weighted clustered boundary prediction for the object in the image based on the corresponding boundary prediction portion of the boundary prediction and the weight for each model of the plurality of models. [Aspect 18] 11. The computing device of embodiment 10, wherein the plurality of models comprises three or more separate models. [Aspect 19] 11. The computing device of embodiment 10, further comprising a camera configured to capture the image. [Aspect 20] A non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors of a computing device to: receiving an image comprising a cross-sectional view of an object; for each of a plurality of models, generating a boundary prediction for the object in the image using the respective model, wherein each boundary prediction comprises a plurality of boundary prediction portions, and each model of the plurality of models comprises a unique model; generating a clustered boundary prediction for the object in the image based on the boundary prediction for each model of the plurality of models, wherein the clustered boundary prediction comprises a plurality of clustered boundary prediction portions, each corresponding to one of the plurality of boundary prediction portions of a respective one of the boundary predictions; for each of the plurality of clustered boundary prediction portions, determining a confidence level for the respective clustered boundary prediction portion based on a comparison between the respective clustered boundary prediction portion and the corresponding boundary prediction portion of the boundary prediction; 11. A non-transitory computer-readable storage medium comprising: a processor for generating a plurality of clustered boundary prediction portions each having a visual characteristic indicative of the confidence level for said respective clustered boundary prediction portions, said plurality of clustered boundary prediction portions being output for display on a display device;

Claims

1. 1. A method comprising: receiving, by a processor, an image comprising a cross-sectional view of an object; generating, by the processor, for each of a plurality of models and using each of the plurality of models, a boundary prediction for the object in the image, wherein each boundary prediction comprises a plurality of boundary prediction portions, each of the plurality of models comprising a unique model; generating, by the processor and based on the boundary prediction for each of the plurality of models, a clustered boundary prediction for the object in the image, wherein the clustered boundary prediction comprises a plurality of clustered boundary prediction portions, each corresponding to one of the plurality of boundary prediction portions of the boundary prediction; determining, for each of the plurality of clustered boundary prediction portions, by the processor and based on a comparison between each of the plurality of clustered boundary prediction portions and the corresponding boundary prediction portion of the boundary prediction, a confidence level for each of the plurality of clustered boundary prediction portions; and outputting, by the processor and for display on a display device, a graphical indicator for each of the plurality of clustered boundary prediction portions having a visual feature indicative of the confidence level for each of the plurality of clustered boundary prediction portions.

2. 2. The method of claim 1, wherein the plurality of models includes one or more of a machine learning model, a deep learning model, a pattern recognition model, a computer vision model, and a parameter modification model.

3. Determining the confidence level comprises: determining, for each of the plurality of models, by the processor and based on a comparison between each of the plurality of clustered boundary prediction portions and the corresponding boundary prediction portion of the boundary prediction, a spread for each of the plurality of models; 2. The method of claim 1, further comprising: determining, by the processor, the confidence level for each of the plurality of clustered boundary prediction portions based on the spread of the boundary prediction for each of the corresponding boundary prediction portions.

4. Determining the spread for each of the corresponding boundary prediction portions of the boundary prediction includes: generating, by the processor, a prediction distribution using each of the plurality of clustered boundary prediction portions and each of the corresponding boundary prediction portions of the boundary prediction, the clustered boundary prediction portion being a mean of the prediction distribution; determining, by the processor, a standard deviation for each of the corresponding boundary prediction portions of the boundary prediction; 4. The method of claim 3, further comprising: determining, by the processor, a z-score for each of the corresponding boundary prediction portions of the boundary prediction as the number of standard deviations that each of the corresponding boundary prediction portions is away from the clustered boundary prediction.

5. determining, by the processor, that the spread for a first corresponding boundary prediction portion of a first boundary prediction exceeds a deviation threshold for the prediction distribution, where the first boundary prediction is associated with a first model of the plurality of models; removing, by the processor, the first model from the plurality of models to generate a subset of models; 5. The method of claim 4, further comprising: recalculating, by the processor, at least each of the plurality of clustered boundary prediction portions using only a subset of the models.

6. 6. The method of claim 5, wherein the deviation threshold comprises one of a user defined threshold or a default threshold.

7. 2. The method of claim 1, wherein the visual characteristics comprise one or more of a line color, a fill pattern, a line thickness, a line division, a text indicator, or a graphic icon.

8. for each of the plurality of models, determining, by the processor, a weight for each of the plurality of models; 2. The method of claim 1 , further comprising generating, by the processor, each of the plurality of clustered boundary predictions as a weighted clustered boundary prediction for the object in the image based on the corresponding boundary prediction portion of the boundary prediction and the weight for each of the plurality of models.

9. 2. The method of claim 1, wherein the plurality of models comprises three or more separate models.

10. 1. A computing device, comprising: At least one processor; a storage component configured to store instructions, the instructions comprising: receiving an image comprising a cross-sectional view of an object; for each of a plurality of models, generating a boundary prediction for the object in the image using each of the plurality of models, where each boundary prediction comprises a plurality of boundary prediction portions, and where each model of the plurality of models comprises a unique model; generating a clustered boundary prediction for the object in the image based on the boundary prediction for each model of the plurality of models, wherein the clustered boundary prediction comprises a plurality of clustered boundary prediction portions, each corresponding to one of the plurality of boundary prediction portions of the boundary prediction; for each of the plurality of clustered boundary prediction portions, determining a confidence level for each of the plurality of clustered boundary prediction portions based on a comparison between each of the plurality of clustered boundary prediction portions and the corresponding boundary prediction portion of the boundary prediction; 11. A computing device executable by the at least one processor to output, for display on a display device, a graphical indicator for each of the plurality of clustered boundary prediction portions, each of the plurality of clustered boundary prediction portions having a visual characteristic indicative of the confidence level for each of the plurality of clustered boundary prediction portions.

11. 11. The computing device of claim 10, wherein the plurality of models includes one or more of a machine learning model, a deep learning model, a pattern recognition model, a computer vision model, and a parameter modification model.

12. The instructions executable by the at least one processor to determine the trust level include: determining a spread for each of the plurality of models based on a comparison between each of the plurality of clustered boundary prediction portions and the corresponding boundary prediction portion of the boundary prediction for each of the plurality of models; 11. The computing device of claim 10, further comprising instructions executable by the at least one processor to determine the confidence level for each of the plurality of clustered boundary prediction portions based on the spread for each of the corresponding boundary prediction portions of the boundary prediction.

13. The instructions executable by the at least one processor to determine the spread for each of the corresponding boundary prediction portions of the boundary prediction include: creating a prediction distribution using each of the clustered boundary prediction portions and the corresponding boundary prediction portions of the boundary prediction, the prediction distribution being the mean of the clustered boundary prediction portion; determining a standard deviation for each of the corresponding boundary prediction portions of the boundary prediction; 13. The computing device of claim 12, further comprising instructions executable by the at least one processor to determine a z-score for each of the corresponding boundary prediction portions of the boundary prediction as the number of standard deviations that each of the corresponding boundary prediction portions is away from the clustered boundary prediction portions.

14. The instructions may further include: determining that the spread for a first corresponding boundary prediction portion of a first boundary prediction exceeds a deviation threshold for the prediction distribution, where the first boundary prediction is associated with a first model of each of the plurality of models; removing the first model from each of the plurality of models to generate a subset of models; 14. The computing device of claim 13, executable by the at least one processor to recalculate at least each of the plurality of clustered boundary prediction portions using only a subset of the models.

15. 15. The computing device of claim 14, wherein the deviation threshold comprises one of a user defined threshold or a default threshold.

16. 11. The computing device of claim 10, wherein the visual characteristics comprise one or more of a line color, a fill pattern, a line thickness, a line division, a text indicator, or a graphic icon.

17. The instructions may further include: For each of the plurality of models, determining a weight for each of the plurality of models; 11. The computing device of claim 10, executable by the at least one processor to generate the clustered boundary predictions as weighted clustered boundary predictions for the object in the image based on the corresponding boundary prediction portions of the boundary predictions and the weights for each of the plurality of models.

18. 11. The computing device of claim 10, wherein the plurality of models comprises three or more separate models.

19. 11. The computing device of claim 10, further comprising a camera configured to capture the image.

20. A non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors of a computing device to: receiving an image comprising a cross-sectional view of an object; for each of a plurality of models, generating a boundary prediction for the object in the image using each of the plurality of models, where each boundary prediction comprises a plurality of boundary prediction portions, and where each of the plurality of models comprises a unique model; generating a clustered boundary prediction for the object in the image based on the boundary prediction for each of the plurality of models, wherein the clustered boundary prediction comprises a plurality of clustered boundary prediction portions, each corresponding to one of the plurality of boundary prediction portions of the boundary prediction; for each of the plurality of clustered boundary prediction portions, determining a confidence level for each of the plurality of clustered boundary prediction portions based on a comparison between each of the plurality of clustered boundary prediction portions and the corresponding boundary prediction portion of the boundary prediction; 11. A non-transitory computer-readable storage medium comprising: a processor for generating a plurality of clustered boundary prediction portions, the plurality of clustered boundary prediction portions being processed by a processor for generating a plurality of clustered boundary prediction portions;

Images