Anchor point based image segmentation for medical imaging

Abstract

A method implemented by one or more computer devices includes receiving an image related to a portion of a target anatomical structure. Boundary points are extracted for an initial boundary associated with a region of corresponding pixels in the image. The boundary points are evaluated in sequential order to select an anchor point. The evaluating includes determining that the current boundary point being evaluated is the next anchor point when at least one perpendicular distance to a line extending between the previous anchor point and the current boundary point, calculated for a portion of the initial boundary located between the previous anchor point and the current boundary point, is greater than a selected threshold. An anchor point image is generated for display on a graphical user interface of a display device. The anchor point image includes an anchor indicator representing the anchor point.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is related to and claims the benefit of the priority date of U.S. Provisional Application No. 63 / 427,639, filed November 23, 2022, entitled "Anchor Points-Based Image Segmentation for Medical Imaging," which is incorporated herein by reference in its entirety.

[0002] Technical Field The present application relates generally to analyzing images, and more particularly to methods and systems that involve image analysis to generate anchor indicators in a graphical user interface that enable efficient automatic and / or semi-automatic adjustment to the boundaries of anatomical regions of interest identified using image segmentation. [Background technology]

[0003] background Medical imaging typically involves visualizing and analyzing digitized images to determine whether variations in tissue are due to disease, toxicity, and / or natural processes. Specifically, medical imaging may rely on one or more image analysis tasks to transform raw image data into a qualitative understanding of the tissue. For example, the one or more image analysis tasks generally include image enhancement, image segmentation, image feature extraction, and finally image classification. Some examples of image analysis may include identifying specific regions of tissue that appear normal, diseased, or correspond to one or more other similar regions of interest. For example, by identifying regions of tissue that appear normal or diseased and quantifying the area, shape, or texture of these regions of tissue, computational-based methods can perform in minutes what would otherwise require hours of tedious work performed by a pathologist, multiple graders, or other scientific or medical experts.

[0004] Image segmentation generally involves dividing a digitized image into different regions of pixels. In some cases, a class label may be assigned to each of the different regions of pixels (e.g., normal region, disease region, region of interest, or region of concern). In some cases, the class label may be predicted utilizing a machine learning model trained to recognize characteristics (e.g., features) of regions of pixels in a digitized image. For example, in one example, as part of the drug discovery and development process, machine learning model-based image segmentation may be relied upon to accurately classify regions of pixels in a tissue scan corresponding to, e.g., a tumor bed, stroma, or healthy tissue for the purpose of identifying appropriate treatments. In another example, machine learning model-based image segmentation may be relied upon to accurately classify regions of pixels in a patient's retina corresponding to, e.g., individual retinal layers and / or one or more fluid pockets in the patient's retina. After identifying appropriate treatments, such as through new drug discovery, these new drugs may be subject to regulatory approval requirements by government or regulatory agencies. For example, the approval procedure may involve pathologists, human graders, central reading centers (CRCs), or other medical or scientific experts reviewing not only the efficacy of the new drug itself, but also the effectiveness and accuracy of the means and methods by which the drug was validated and / or the new drug was developed. However, because the image segmentation task alone may not render sharp, clear boundaries between different pixel regions in a medical scan, for example, the class labels corresponding to different pixel regions may be less precise or inaccurate. Summary of the Invention

[0005] overview In one or more embodiments, a method includes receiving an image related to a portion of a target anatomical structure. The method includes extracting a plurality of boundary points for an initial boundary associated with corresponding regions of pixels in the image, the plurality of boundary points being associated in a sequential order for a selected two-dimensional plane corresponding to the image. The method includes evaluating the plurality of boundary points according to the sequential order to select a plurality of anchor points from the plurality of boundary points. The evaluating includes determining that a first boundary point of the plurality of boundary points is a first anchor point. The evaluating includes determining that a current boundary point of the plurality of boundary points being evaluated is a next anchor point of the plurality of anchor points when at least one perpendicular distance to a line extending between the previous anchor point and the current boundary point, calculated for a portion of the initial boundary located between the previous anchor point and the current boundary point, is greater than a selected threshold. The method includes generating an anchor point image for display in a graphical user interface of a display device, the anchor point image including a plurality of anchor indicators representing the plurality of anchor points.

[0006] In one or more embodiments, a method includes receiving a plurality of images related to a portion of an anatomical structure of a subject. The method includes performing segmentation using the plurality of images and a segmentation model to generate a plurality of segmented images. The method includes, for each segmented image of the plurality of segmented images, extracting a plurality of boundary points for each initial boundary of a set of initial boundaries associated with each segmented image. The plurality of boundary points are associated with a sequential order of a selected two-dimensional plane corresponding to the plurality of images. The method includes, for each segmented image of the plurality of segmented images, evaluating the plurality of boundary points in a sequential order for each initial boundary of the set of initial boundaries associated with each segmented image to select a plurality of anchor points from the plurality of boundary points for each initial boundary of the set of initial boundaries identified in each image. The evaluating includes determining a first boundary point of the plurality of boundary points to be a first anchor point, determining a last boundary point of the plurality of boundary points to be a last anchor point, and determining a current boundary point of the plurality of boundary points being evaluated to be a next anchor point of the plurality of anchor points when at least one perpendicular distance of a set of perpendicular distances calculated for a corresponding set of intermediate boundary points to a line extending between the previous anchor point and the current boundary point is greater than a selected threshold. The method includes generating an anchor point image for display in a graphical user interface of a display device, the anchor point image including a plurality of anchor indicators representing a plurality of line segments connecting the plurality of anchor points and the plurality of anchor indicators.

[0007] In one or more embodiments, a system includes one or more computing devices including one or more non-transitory computer-readable storage media containing instructions and one or more processors coupled to the one or more storage media. The one or more processors are configured to execute the instructions to receive an image related to a portion of a target anatomical structure. The one or more processors are configured to execute the instructions for extracting a plurality of boundary points for an initial boundary associated with corresponding regions of pixels in the image, the plurality of boundary points being associated in a sequential order for a selected two-dimensional plane corresponding to the image. The one or more processors are configured to execute the instructions for evaluating the plurality of boundary points according to the sequential order to select a plurality of anchor points from the plurality of boundary points. The evaluating includes determining that a first boundary point of the plurality of boundary points is the first anchor point. The evaluating includes determining that a current boundary point of the plurality of boundary points being evaluated is the next anchor point of the plurality of anchor points when at least one perpendicular distance to a line extending between the previous anchor point and the current boundary point, calculated for a portion of the initial boundary located between the previous anchor point and the current boundary point, is greater than a selected threshold. The one or more processors are configured to execute instructions for generating an anchor point image for display in a graphical user interface of a display device, the anchor point image including a plurality of anchor indicators representing a plurality of anchor points. [Brief explanation of the drawings]

[0008] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate certain aspects of the subject matter disclosed herein and, together with the description, serve to explain some of the principles associated with the disclosed implementations.

[0009] [Figure 1A] FIG. 1 is a block diagram of an image analysis system in accordance with one or more exemplary embodiments.

[0010] [Figure 1B] 1B is a more detailed block diagram of the image processor of FIG. 1A in accordance with one or more exemplary embodiments.

[0011] [Figure 2A] 1 is a flowchart of a process for generating calibration data in accordance with one or more exemplary embodiments. [Figure 2B] 1 is a flowchart of a process for generating calibration data in accordance with one or more exemplary embodiments.

[0012] [Figure 3A] 1 is a flowchart of a process for evaluating boundary points in accordance with one or more exemplary embodiments. [Figure 3B] 1 is a flowchart of a process for evaluating boundary points in accordance with one or more exemplary embodiments.

[0013] [Figure 4] FIG. 10 is a diagram of a workflow of a process for evaluating multiple boundary points that may be included in multiple anchor points, according to one or more embodiments.

[0014] [Figure 5] FIG. 10 is a diagram illustrating how perpendicular distance is calculated in accordance with one or more exemplary embodiments.

[0015] [Figure 6] 1 is a diagram of an image displayed on a graphical user interface in accordance with one or more exemplary embodiments.

[0016] [Figure 7] 1 is an illustration of a segmented image displayed in a graphical user interface in accordance with one or more exemplary embodiments.

[0017] [Figure 8]1 is a diagram of an image with identified boundaries displayed in a graphical user interface in accordance with one or more exemplary embodiments.

[0018] [Figure 9] 1 is a diagram of an image displayed with boundary indicators in a graphical user interface in accordance with one or more exemplary embodiments.

[0019] [Figure 10] 1 is a diagram of an image including an anchor indicator displayed in a graphical user interface in accordance with one or more exemplary embodiments.

[0020] [Figure 11] 1 is a diagram of an image displayed with boundary indicators in a graphical user interface in accordance with one or more exemplary embodiments.

[0021] [Figure 12] 1 is a diagram of an image including an anchor indicator displayed in a graphical user interface in accordance with one or more exemplary embodiments.

[0022] [Figure 13] 1 is a diagram of an image displayed with boundary indicators in a graphical user interface in accordance with one or more exemplary embodiments.

[0023] [Figure 14] 1 is a diagram of an image including anchor points displayed in a graphical user interface in accordance with one or more exemplary embodiments.

[0024] [Figure 15] 1 is a diagram of an image displayed with boundary indicators in a graphical user interface in accordance with one or more exemplary embodiments.

[0025] [Figure 16] 1 is a diagram of an image including anchor points displayed in a graphical user interface in accordance with one or more exemplary embodiments.

[0026] [Figure 17] 10 is an illustration of a graphical user interface that converts a display of a boundary into an anchor indicator in accordance with one or more exemplary embodiments.

[0027] [Figure 18] 10 is an illustration of a graphical user interface that converts a display of a boundary into an anchor indicator in accordance with one or more exemplary embodiments.

[0028] [Figure 19] 10 is an illustration of a graphical user interface that converts a display of a boundary into an anchor indicator in accordance with one or more exemplary embodiments.

[0029] [Figure 20] 20 is a diagram of the graphical user interface from FIG. 19 illustrating varying anchor point density in accordance with one or more exemplary embodiments.

[0030] [Figure 21] FIG. 1 is a block diagram illustrating an example of a computing system, in accordance with one or more exemplary embodiments.

[0031] It should be understood that the figures are not necessarily drawn to scale, and that objects in the figures are not necessarily drawn to scale relative to each other. The figures are intended to provide clarity and understanding of various embodiments of the devices, systems, and methods disclosed herein. Wherever possible, the same reference numerals will be used throughout the figures to refer to the same or like parts. Furthermore, it should be appreciated that the figures are not intended to limit the scope of the present teachings in any way. DETAILED DESCRIPTION OF THE INVENTION

[0032] Detailed Description I. Overview The embodiments described herein take into account that, in certain cases, currently available image analysis technologies may not provide the desired level of accuracy for use in disease screening, diagnosis, and / or treatment management. Furthermore, the embodiments described herein take into account that certain rules, regulations, and / or requirements associated with clinical trials, government agencies, and / or regulatory bodies may mandate some level of human intervention to correct inaccurate data generated using automated techniques. For example, complete reliance on machine learning models to diagnose or analyze disease treatment based on medical images may not be fully accepted by certain members or groups of medical or regulatory bodies, and in some cases, may not be permitted by regulatory bodies. Therefore, it is desirable to improve the accuracy of information generated using machine learning models.

[0033] For example, machine learning models may be used to segment various images (e.g., medical images including OCT images of the retina or MRI images of the brain). For example, AI-based semantic segmentation may be used to classify each pixel as belonging to one of various selected categories. The selected category may be, for example, retinal fluid layer, brain tissue, lung tissue, or some other anatomical structure or feature. This process may be used to generate a segmented image that identifies regions of pixels belonging to different categories. The boundaries of these regions may also be identified.

[0034] However, in some cases, the identification of these regions and their boundaries may not have a desired level of accuracy. In some cases, it may be desirable for a human user to review the segmented regions and / or their boundaries to adjust the boundaries to more accurately reflect the actual anatomical regions. Therefore, it may be desirable to have a method and system that allows for fast, efficient, and accurate adjustment of these boundaries based on user input. New images generated with the newly adjusted boundaries may be used as training data to retrain a machine learning model to more accurately segment regions and boundaries.

[0035] Thus, the embodiments described herein provide automated and semi-automated methods and computer-readable media for improving data accuracy using machine learning models. For example, the embodiments described herein provide image analysis systems that can be used to align images or generate alignment data including aligned images in which boundaries of particular anatomical regions of interest identified in those images more accurately reflect the actual anatomical regions of interest.

[0036] In one or more embodiments, the segmented image may identify regions of pixels, each of which is classified as belonging to a selected category. The segmented image may be associated with a portion of the subject's anatomy (e.g., eye, retina, brain, lung, chest, leg, etc.). The regions of pixels may correspond to a fluid layer, a pathological element, a tissue type, a lobe, or some other type of anatomical structure or feature.

[0037] A set of initial boundaries associated with a region of pixels may be identified using the segmented image. In some cases, these boundaries are identified in the segmented image. In other cases, these boundaries may be calculated from the segmented image. The initial boundaries associated with a region of pixels may be, for example, but not limited to, an outer boundary, an inner boundary, a lower boundary, an upper boundary, a lateral boundary, a combination thereof, or some other type of boundary. Multiple boundary points may be extracted for the initial boundaries. The multiple boundary points are associated with a sequential order of a reference two-dimensional (2D) plane corresponding to the image. The multiple boundary points may be evaluated according to the sequential order to select multiple anchor points from the multiple boundary points. This evaluation may include, for example, automatically determining a first boundary point of the multiple boundary points to be a first anchor point. This evaluation may include, for example, automatically determining a last boundary point of the multiple boundary points to be a last anchor point. This evaluation may include, for example, automatically determining that the current boundary point of the plurality of boundary points being evaluated is the next anchor point of the plurality of anchor points if at least one perpendicular distance to a line extending between the previous anchor point and the current boundary point, calculated for a portion of the initial boundary located between the previous anchor point and the current boundary point, is greater than a selected threshold.

[0038] The selected threshold may be selected to result in a desired density of anchor points with a desired level of complexity of curvature, grooves, texture, and / or contours in the boundaries represented by the anchor points. A higher threshold may be selected to reduce the density (number) of anchor points and reduce the complexity of curvature, grooves, texture, and / or contours in the captured boundaries. Conversely, a lower threshold may be selected to increase the density (number) of anchor points and increase the complexity of curvature, grooves, texture, and / or contours in the captured boundaries.

[0039] A graphical user interface is generated in which an anchor point image is displayed. The anchor point image may include, for example, the original image on which the segmentation was performed and a plurality of anchor indicators representing the plurality of anchor points. The plurality of anchor indicators may be controllable graphical indicators. For example, a human user (e.g., a medical professional or expert, a human pathologist, a human grader, a reading center, or other type of human user) may move an anchor indicator to change the position of the corresponding anchor point and, therefore, the corresponding boundary.

[0040] By viewing the anchor indicators in the original image, a human user can make decisions about where the boundaries generated via machine learning techniques can be adjusted to more accurately reflect the actual corresponding boundaries associated with the anatomy of interest. In this manner, the direction and / or curvature of the boundaries may be controlled based on user input.

[0041] The anchor points are selected to reduce overall computing resources expended by giving the user the option to adjust all pixels that would otherwise form the boundary of the region of interest. By using anchor indicators to represent the anchor points, the user can focus on those critical points along the boundary that affect curvature, contour, texture, etc., allowing the system to more easily and efficiently update the boundary based on less input without sacrificing accuracy. In some embodiments, the segmented image may be adjusted to reclassify (or reallocate) various pixels based on the new / modified boundary.

[0042] The image analysis system may process the user's input and automatically adjust the positions of the corresponding anchor points, thereby automatically adjusting previously identified boundaries. The image analysis system may generate adjusted images with new boundaries that can be used, for example, as new labeled data for supervised and / or semi-supervised training and / or retraining of machine learning models. In this way, machine learning models used to segment images may be better trained to perform segmentation more accurately.

[0043] The use of anchor points and anchor indicators representing those anchor points in the manner described herein may enable adjustments / corrections of segmented images in a manner that reduces the overall time and resources required to make these adjustments / corrections without sacrificing accuracy. Because the anchor indicators represent only a subset of the boundary points extracted for a given boundary, adjustments can be made more efficiently to the entire boundary without requiring the user to adjust each boundary point (pixel) of that boundary. Furthermore, fewer anchor indicators means fewer overall adjustment calculations need to be made without sacrificing overall accuracy. Therefore, generating anchor points and a graphical user interface that displays controllable / movable anchor indicators representing the anchor points may improve the overall functionality of the image analysis system described herein and reduce the overall consumption of computing resources.

[0044] II. Exemplary Image Analysis System Architecture 1A is a block diagram of an image analysis system 100 according to one or more exemplary embodiments. Image analysis system 100 may be used to analyze multiple images, such as, for example, but not limited to, image 101. Each image of multiple images 101 may be an image of a portion of a subject's anatomical structure. For example, image 101 may be a medical image. Thus, the image analysis system may also be referred to as a medical imaging analysis system. Furthermore, image analysis system 100 may be used to analyze images using image segmentation and therefore may also be referred to as an image segmentation analysis system or a medical image segmentation analysis system.

[0045] Image analysis system 100 may be used to train and use machine learning models to more accurately and efficiently segment images (e.g., OCT images) to provide assessment, detection, diagnosis, and / or treatment of patients with disease, or a combination thereof. Image analysis system 100 may include a computing platform 102, data storage 104, an input system 105, and a display system 106. In one or more embodiments, data storage 104, input system 105, display system 106, or any combination thereof, may be part of a remote system 107 located remotely relative to computing platform 102.

[0046] Computing platform 102 may take a variety of forms. In one or more embodiments, computing platform 102 includes a single computer (e.g., a server or computer system), multiple computers (e.g., servers) communicating with each other (e.g., via one or more wired, one or more wireless, and / or one or more optical communication links), a smart television, and / or any combination thereof. In other examples, computing platform 102 takes the form of a cloud computing platform, a mobile computing platform (e.g., a smartphone, tablet, laptop, etc.), or a combination thereof.

[0047] In some embodiments, computing platform 102 may include, but is not limited to, a cloud-based computing architecture suitable for hosting, serving, and / or interfacing one or more hardware, software, and / or firmware modules or tools executing on remote system 107. In one or more embodiments, computing platform 102 may include a Platform as a Service (PaaS) architecture, a Software as a Service (SaaS) architecture, an Infrastructure as a Service (IaaS) architecture, a Compute as a Service (CaaS) architecture, a Data as a Service (DaaS) architecture, a Database as a Service (DBaaS) architecture, or other similar cloud-based computing architecture (e.g., "X as a Service" (XaaS)).

[0048] The hardware of computing platform 102 may include, for example, but not limited to, a general-purpose processor, a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a system-on-chip (SoC), a microcontroller, a field programmable gate array (FPGA), a central processing unit (CPU), an application processor (AP), a visual processing unit (VPU), a neural processing unit (NPU), a neural decision processor (NDP), a deep learning processor (DLP), a tensor processing unit (TPU), a neuromorphic processing unit (NPU), or any other processing device that may be suitable for processing various medical profile data and making one or more decisions based thereon. Computing platform 102 may include software (e.g., instructions running / executing on one or more processors), firmware (e.g., microcode), or some combination thereof.

[0049] Data storage 104, input system 105, and / or display system 106 may each communicate with computing platform 102 (e.g., using one or more wired, wireless, optical, and / or other types of communication links). In some examples, data storage 104, input system 105, display system 106, or any combination thereof, may be considered part of computing platform 102 or may be otherwise integrated with computing platform 102. Thus, in some examples, computing platform 102, data storage 104, and display system 106 may be separate components that communicate with each other, while in other examples, some combination of these components may be integrated together. Data storage 104 may include one or more data stores, such as, for example, one or more databases.

[0050] Input system 105 may include one or more input devices for receiving user input. Input system 105 may include, for example, without limitation, a mouse, touchpad, keyboard, touchscreen, keypad, joystick, virtual reality input device, multiple arrow keys, or combinations thereof that allow a user to enter user input. Display system 106 may include one or more display devices, such as, without limitation, a monitor, television, screen, or some other type of display device. A device such as a touchscreen may function as both an input device for input system 105 and a display device for display system 106.

[0051] Image analysis system 100 includes image processor 108, which may be implemented using hardware, software, firmware, or a combination thereof. In one or more embodiments, image processor 108 is implemented on computing platform 102. In some embodiments, a first portion (e.g., hardware, firmware, and / or software) of image processor 108 is implemented on computing platform 102, and a second portion (e.g., hardware, firmware, and / or software) of image processor 108 is implemented on remote system 107.

[0052] Image processor 108 may include, for example, segmentation tool 110 and alignment tool 112, each of which may be implemented using hardware, software, firmware, or a combination thereof. In one or more embodiments, image processor 108 receives image 101 for processing. In some embodiments, image 101 may be received from data storage 104 and / or some other type of data store (e.g., cloud storage) in communication with computing platform 102. Image 101 may capture a portion of a subject's (e.g., patient's) anatomical structure. For example, image 101 may capture the subject's retina, one or more bones, brain, one or more lungs, one or more kidneys, liver, bladder, heart, other tissue, or a combination thereof.

[0053] Additionally, image 101 may be a medical image (e.g., a medical scan, a medical diagnostic image) including an optical coherence tomography (OCT) image (e.g., an OCT volume, an OCT B-scan), a magnetic resonance imaging (MRI) image, a computed tomography (CT) image, an X-ray image, an ultrasound image, a positron emission tomography (PET) image, a single-photon emission computed tomography (SPECT) image, or any other type of 2D / 3D image capturing a portion of an anatomical structure. Image 114 may be an example of image 101. Image 114 may take the form of an OCT image, such as an OCT B-scan, which may be used to capture and render retinal layer depth. An OCT B-scan refers to a cross-sectional image in which the amplitude of reflections is represented on a grayscale or false color scale. In some cases, an OCT B-scan may be referred to as a luminance scan. In other embodiments, image 114 is an MRI image, a CT image, or some other type of medical imaging scan.

[0054] In one or more embodiments, image 101 is generated after processing of an initial image generated by a medical imaging system (e.g., a CT device, an MRI device, an OCT scanner, etc.). For example, the initial image generated by the medical imaging system may be filtered, scaled, resized, cropped, and / or pre-processed in some other manner to generate image 101.

[0055] After the image processor 108 receives the images 108, the segmentation tool 110 may process the received images 101 to generate multiple segmented images 116, which may be 2D or 3D images. In one or more embodiments, the segmentation tool 110 may include a segmentation model 118 that may receive each of the images 101 as input and generate a corresponding segmented image to form the segmented images 116. The segmentation model 118 may itself include any number of models or combinations of models (e.g., one or more neural networks), including, for example, but not limited to, machine learning models such as deep learning models. For example, a deep learning model may include any number of neural networks, including, for example, one or more convolutional neural networks (CNNs). In one or more embodiments, the segmentation model 118 includes a deep learning model that performs artificial intelligence (AI)-based semantic segmentation. Thus, the segmentation model 118 may sometimes be referred to as a semantic segmentation model, an AI segmentation model, or a deep learning (DL) segmentation model. Semantic segmentation is a pixel-by-pixel segmentation in which each pixel is classified (or labeled) as belonging to a category.

[0056] To perform AI-based semantic segmentation of an image such as image 114 to generate corresponding segmented image 120, segmentation model 118 groups pixels of image 114 into multiple categories. For example, segmentation model 118 may label or otherwise classify each pixel of image 114 as belonging to one of these categories (e.g., 2, 3, 4, 5, size, 7, 8, 9, 10, or some other number of categories).

[0057] In one or more embodiments, segmentation tool 110 or another portion of image processor 108 may display segmented image 120 in a graphical user interface (GUI) 122 of display system 106. Graphical user interface 122 may be generated by image processor 108 for display on display system 106 and may allow a user to interact with graphical user interface 122. Segmented image 120 may be displayed in graphical user interface 122 such that each group of pixels belonging to a different category may be represented with a different graphic indicator or feature to distinguish each group of pixels. For example, different colors, shading, highlighting, patterns, labels, text, and / or other types of graphical indicators or features may be used to distinguish different groups of pixels in different categories. As one specific example, each different category may be represented in graphical user interface 122 by corresponding pixels of each different category having a different color.

[0058] In one or more embodiments, each group of pixels belonging to a different category may be referred to as a "segment" in the segmented image 120. A segment may be a contiguous or discontinuous grouping of pixels. For example, a segment may be formed by one or more distinct regions of pixels that are each labeled as belonging to the same category. Alternatively, a segment may be an individual region of pixels, and a particular grouping of pixels may be represented by one or more segments in the segmented image 120. In this manner, the segmentation model 118 may perform one or more image segmentation processes (e.g., semantic image segmentation) to segment regions of pixels included in the image 114.

[0059] The adjustment tool 112 may receive the segmented images 116 from the segmentation tool 110 for processing. The adjustment tool 112 may process the segmented images 116 to generate adjustment data 124. The adjustment data 124 may be data that results in adjusted or corrected data that can be used to adjust (or correct) the segmented images 116 so that the adjusted segmented images 116 can be used as input for a model. For example, the adjustment data 124 may include multiple adjusted images 126 that can be used to retrain the segmentation model 118. The adjusted images 126 may be 2D or 3D images and may be used to train / retrain a different machine learning model. The adjusted images 126 may generate additional information about the image 101 and be used as input for another model or algorithm to generate an output based on the image 101 (e.g., output for screening, diagnosis, or treatment management). The adjustment may be to the definition or location / shape / size of the region of interest. Types of adjustments that can be made are further described with respect to FIG. 1B.

[0060] 1B is a more detailed block diagram of the image processor 108 of FIG. 1A, in accordance with one or more exemplary embodiments, which provides further details about the segmentation tool 110 and the adjustment tool 112 of the image processor 108.

[0061] As previously described, the segmentation model 118 groups pixels of each image in the images 101 into multiple selected categories 128. The selected categories 128 may include 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, or several other categories. The selected categories 128 may be selected based on one or more regions of interest. The segmentation model 118 may be used to identify a single anatomical structure or feature of interest, and the segmentation model 118 may classify each pixel of the image as representing or not representing that anatomical structure / feature of interest (e.g., background). The segmentation model 118 may also be used to identify multiple features of interest, such as a fluid layer. The segmentation model 118 may also be used to identify multiple features of interest, such as a fluid layer. The selected category 128 may include, for example, background, normal area, diseased area, area of ​​interest, fluid layer, pathological element, a particular type of tissue (e.g., brain tissue, lung tissue, bone tissue, etc.), a particular type of organ, some other type of anatomical structure or feature, or a combination thereof.

[0062] For example, the segmentation model 118 may label or otherwise classify each pixel in the image 114 as belonging to one of the selected categories 128. In one or more embodiments, each group of pixels belonging to different categories may be referred to as a “segment” in the segmented image 120. A segment may be a contiguous or discontinuous grouping of pixels. For example, a segment may be formed by one or more distinct regions of pixels that are each labeled as belonging to the same category. Alternatively, a segment may be an individual region of pixels, and a particular grouping of pixels (e.g., all pixels of a selected category) may be represented by one or more segments in the segmented image 120. In this manner, the segmentation model 118 may perform one or more image segmentation processes (e.g., semantic image segmentation) to segment regions of pixels included in the image 114.

[0063] For example, segmentation tool 110 may process image 114 to generate segmented image 120, which includes a region of pixels 130 where all pixels in a selected region of pixels 130 (e.g., region of pixels 132) belong to the same selected category of selected category 128. One region of pixels may form an entire segment of segmented image 120, or multiple regions of pixels may form a segment of segmented image 120.

[0064] Adjustment tool 112 may receive segmented images 116 (including segmented image 120) from segmentation tool 110 for processing. For each of segmented images 116, adjustment tool 112 identifies one or more boundaries for one or more regions of interest. The region of interest may be, for example, one of the regions 130 of pixels. In one or more embodiments, segmented image 120 received by adjustment tool 112 includes an identification or indication of a set of initial boundaries 134. For example, segmented image 120 may be annotated to identify the set of initial boundaries 134. In other embodiments, adjustment tool 112 determines the set of initial boundaries 134 based on segments of segmented image 120.

[0065] For example, for segmented image 120, adjustment tool 112 processes segmented image 120 to identify set of initial boundaries 134. Set of initial boundaries 134 includes one or more initial boundaries, such as initial boundary 136. Initial boundary 136 is associated with a corresponding region of pixels, which may be region of pixels 132. For example, initial boundary 124 may include an outer boundary of region of pixels 132, an inner boundary of region of pixels 132, an upper boundary of region of pixels 132, a lower boundary of region of pixels 132, a selected boundary between region of pixels 132 and another adjacent region of pixels, or some other type of boundary that relates to or provides information about the anatomical structure or feature represented by region of pixels 132.

[0066] The adjustment tool 112 extracts a plurality of boundary points for each boundary in the set of initial boundaries 134. For example, the adjustment tool 112 extracts a plurality of boundary points 138 for the initial boundary 136. In one or more embodiments, the boundary points 138 may be pixels forming the initial boundary 136. For example, the boundary points 138 may be the X, Y position or the X, Y, Z position (e.g., in pixel units or some other unit used to identify the location of pixels) of each pixel forming the initial boundary 136. In other embodiments, the boundary points 138 include every nth pixel forming the initial boundary 136, where n may be (e.g., 1, 2, 3, 4, 5, etc.).

[0067] Boundary points 138 may be associated with a sequential order. In one or more embodiments, the sequential order is associated with a selected two-dimensional (2D) plane or selected direction in an X, Y, Z coordinate system. For example, if image 114 and / or segmented image 120 are 2D images, the 2D plane may be a 2D plane corresponding to either or both of these images. In this example, the sequential order proceeds from left to right across the 2D plane, such that the sequential order begins with boundary points (pixels) from the leftmost column of pixels and continues to the rightmost column.

[0068] If image 114 and / or segmented image 120 are 3D images, the sequential order may be as described above, except for each 2D plane parallel to the selected direction or reference 2D plane in the X, Y, Z coordinate system. Thus, for a 3D image, boundary points 138 may include a different group of boundary points for each selected plane / slice parallel to the reference 2D plane. The sequential order may proceed from left to right across each of these individual selected planes / slices.

[0069] Next, the adjustment tool 112 evaluates the boundary points 138 in sequential order to select a plurality of anchor points 140 from the plurality of boundary points. In other words, the anchor points 140 comprise a subset of the boundary points 138. The anchor points 140 are selected such that a line segment connecting the anchor points approximates the curvature of the initial boundary 136 within a selected tolerance. The selected tolerance is selected such that a line segment connecting any pair of anchor points does not overly linearize the curvature of the corresponding portion of the initial boundary 136. For example, the linearization of the curved portion of the initial boundary 136 by a line extending between the pair of anchor points may be measured by the maximum perpendicular distance between the line segment extending between the pair of anchor points and the initial boundary 136. A description of how the boundary points 138 are evaluated to select a subset to form the anchor points 140 is further described below in Section III (Exemplary Image Analysis Methods).

[0070] Once anchor points 140 are determined, adjustment tool 112 may generate anchor point image 142 for display in graphical user interface 122. Anchor point image 142 may include, for example, original image 114 from which segmented image 120 was generated, with multiple anchor indicators 144 superimposed on original image 114 at locations corresponding to original locations 146 of anchor points 140.

[0071] In one or more embodiments, each anchor indicator of the anchor indicators 144 may be a graphical indicator that is movable or controllable via user input 148. In other words, the anchor indicators 144 may be manipulated by a user. For example, the adjustment tool 112 may receive user input 148 (e.g., entered by a user via the input system 105 described in FIG. 1A ) via the graphical user interface 122 that changes the position of one or more of the anchor indicators 144 on the anchor point image 142. The user input 148 may include the user moving an anchor indicator (e.g., dragging the anchor indicator from its current position to a new position). The user input 148 may include the user selecting (e.g., clicking, etc.) a new position for a particular anchor indicator. The user entering the user input 148 may be, for example, but not limited to, a medical professional or medical expert, such as a human pathologist, a human grader, a reading center, or other suitable user.

[0072] In one or more embodiments, if the anchor point image 142 is a 3D image displayed in 3D format, the graphical user interface 122 may display the anchor point image 142 in a manner that allows the user to control which cross sections or slices of the 3D image are displayed at a given time. These cross sections or slices may be associated with the boundary points 138 and aligned with the reference 2D plane described above as determining the sequential order associated with the boundary points 138.

[0073] Adjustment tool 112 processes user input 148 and updates the positions of one or more of anchor points 140 so that anchor points 140 have final positions 150. Final positions 150 include at least one position that is different (e.g., adjusted) from each of original positions 146 of anchor points 140.

[0074] Adjustment tool 112 may generate adjustment data 124 using final positions 150 of anchor points 140. Adjustment data 124 may include, for example, final positions 150 of anchor points, new positions relative to others of boundary points 138 calculated based on final positions 150, adjusted image 152, or a combination thereof. Adjusted image 152 may be an example of an adjusted image of multiple adjusted images 126. Adjusted image 152 may be, for example, image 114 in which initial boundary 136 has been adjusted to form a new boundary represented in the adjusted image (e.g., via lines, colored lines, patterned lines, text, highlighting, etc.).

[0075] In one or more embodiments, adjusted image 152 may be an adjusted version of segmented image 120 in which pixels have been reclassified based on final positions 150 of anchor points 140. For example, final positions 150 of anchor points 140 may be used to identify a new set of boundaries. The new set of boundaries may be used to reclassify (or reassign) one or more pixels of segmented image 120. For example, the new boundaries may be used to reclassify (or reassign) one or more pixels that currently belong to a first category of selected categories 128 as belonging to a second category. Adjusted image 152 may be referred to as an adjusted segmented image and may be used to form training inputs for segmentation model 118 for training / retraining segmentation model 118 (or another type of segmentation model).

[0076] While the above process has been described with respect to adjustment tool 112 generating anchor points 140 and adjustment data 124 for initial boundary 136, the same process may be used to adjust any other initial boundaries present in set of initial boundaries 134 so that adjustment data 124 accounts for these one or more other initial boundaries. For example, adjustment image 152 may identify new boundaries for each of set of initial boundaries 134.

[0077] Also, while the process above has been described with respect to adjustment tool 112 processing segmented images 120 to generate adjusted image 152, the same process may be used to process each segmented image 116 to generate additional adjustment data 124 (e.g., adjusted image 126). For example, the embodiments described herein may be used to generate adjusted image 126 in the form of an adjusted segmented image.

[0078] In one or more embodiments, the training data 124 may be used to create newly labeled inputs (e.g., labeled image inputs) for supervised or semi-supervised retraining of the segmentation model 118. In other embodiments, the training data 124 may be used to create labeled image inputs for training / retraining of some other type of machine learning algorithm. In still other embodiments, the training data 124 may be used as input or to create inputs for another algorithm or model. This other algorithm or model may be used, for example, without limitation, to screen, diagnose, or manage treatment (e.g., predict treatment response, select treatments with the highest predicted efficacy, etc.) of a selected disease, disorder, or group of diseases / disorders.

[0079] Using anchor points and anchor indicators representing identified boundaries in the segmented image 116 in the above manner to allow a user to correct those anchor points reduces the overall time and resources required to adjust / correct the segmented image 116 to form the adjusted image 126. Because the anchor indicators represent only a subset of the boundary points extracted for a given boundary, adjustments to the entire boundary may be made more efficiently without the user having to adjust each boundary point (pixel) of that boundary. Furthermore, fewer anchor indicators mean fewer overall adjustment calculations need to be performed without sacrificing overall accuracy. Thus, generating anchor points and a graphical user interface 122 that displays controllable / movable anchor indicators representing the anchor points may improve the overall functionality of the image analysis system 100 and reduce the overall consumption of computing resources.

[0080] 1A and 1B , various data and images generated by segmentation tool 110 and adjustment tool 112 may be stored in data storage 104 for future use. For example, image processor 108 may store in data storage 104 the final positions of the anchor points determined for segmented image 120 (e.g., the final positions of the anchor points of each initial boundary 134, including the final positions 150 of anchor points 140 of initial boundary 136). In some embodiments, image processor 108 may store in data storage 104 an anchor point image 142 having anchor indicators 144 at positions corresponding to the final positions 150 of anchor points 140. In some embodiments, image processor 108 may store adjustment data 124 in data storage 104. In some embodiments, image processor 108 may store segmented image 116 in data storage 104. In some embodiments, image processor 108 may store segmented images 116 with the final locations of anchor points identified in each segmented image 116 (e.g., final locations 150 of anchor points 140 identified in segmented image 120), images 101 with the final locations of anchor points identified in each image 101 (e.g., final locations 150 of anchor points 140 identified in image 114), or both. Similarly, in some embodiments, image processor 108 may store a set of initial boundaries and / or their corresponding boundary points associated with each of segmented images 116, images 101, or both.

[0081] 1A and 1B, in one or more embodiments, the images 101 processed by the image processor 108 may take the form of OCT images of a subject's retina. In these examples, a segmentation tool 110 may be used to perform retinal segmentation on these OCT images. Retinal segmentation involves detecting and identifying one or more retinal (e.g., retina-related) elements of the retinal image. The retinal elements may be comprised of at least one of a retinal layer element or a retinal pathology element. The detection and identification of the one or more retinal layer elements may be referred to as segmenting a layer element (or retinal layer element). The detection and identification of the one or more retinal pathology elements may be referred to as segmenting a pathology element (or retinal pathology element).

[0082] A retinal layer element may be, for example, a retinal layer or a boundary associated with a retinal layer. Examples of retinal layers include, but are not limited to, the inner limiting membrane (ILM) layer, retinal nerve fiber layer, ganglion cell layer, inner plexiform layer, inner nuclear layer, outer plexiform layer, outer nuclear layer, outer limiting membrane (ELM) layer, photoreceptor layer, retinal pigment epithelium (RPE) layer, RPE detachment layer, Bruch's membrane (BM) layer, choriocapillaris layer, choroidal stromal layer, ellipsoid zone (EZ), and other types of retinal layers. In some cases, a retinal layer may be composed of one or more layers. As an example, a retinal layer may be the outer plexiform layer-Henle fiber layer (OPL-HFL). A boundary associated with a retinal layer may be, for example, the inner boundary of a retinal layer, the outer boundary of a retinal layer, a boundary associated with a pathological feature of a retinal layer (e.g., the inner or outer boundary of a retinal layer detachment), or some other type of boundary. For example, the boundary can be the inner boundary of the RPE (IB-RPE) peeling layer, the outer boundary of the RPE (OB-RPE) peeling layer, or another type of boundary.

[0083] Retinal pathology elements may include, for example, fluid (e.g., fluid pockets), cells, solid material, or a combination thereof, indicative of retinal pathology (e.g., a disease or condition such as AMD or DME). For example, the presence of specific retinal fluid may be a sign of nAMD or DME. Examples of retinal pathology elements include, but are not limited to, intraretinal fluid (IRF), subretinal fluid (SRF), fluid associated with pigment epithelial detachment (PED), hyperreflective material (HRM), subretinal hyperreflective material (SHRM), intraretinal hyperreflective material (IHRM), hyperreflective lesions (HRF), retinal fluid pockets, drusen, fibrotic development, and disruption. In some cases, retinal pathology elements may be disruption (e.g., discontinuity, delamination, loss, etc.) of a retinal layer or zone. For example, the disruption may be disruption of the ellipsoid region, ELM, RPE, or another layer or region. The disruption may represent damage or loss of cells (e.g., photoreceptors) in the area of ​​disruption.

[0084] Furthermore, the retinal pathology may include a characteristic or subtype of one of the following: fluid (e.g., IRF, SRF, fluid associated with PED), material (e.g., HRM, SHRM, IHRM), lesion (e.g., HRF, SHRM lesion), or disruption. In particular, examples of retinal pathology may include the characteristics and / or subtypes of the different types of elements and disruptions described above that can be detected and identified through retinal segmentation. For example, whether the retinal fluid is clear or cloudy may be a detectable and identifiable characteristic of the retinal fluid. Thus, in some examples, the retinal pathology may be clear IRF, cloudy IRF, clear SRF, cloudy SRF, some other type of clear retinal fluid, some other type of cloudy retinal fluid, or a combination thereof. In some cases, with respect to SHRM, shape characteristics (e.g., tall SHRM, dome-shaped SHRM at the fovea, flat SHRM near the fovea, irregular shape, etc.), boundary characteristics (e.g., unclear SHRM, clear SHRM), reflectance (e.g., increased reflectance or other levels of reflectance), layering characteristics (e.g., hyperreflective bands in SHRM lesions), and lesion characteristics (e.g., height, width, and / or area of ​​SHRM lesions) may be examples of retinal pathological elements that may be detected and identified via retinal segmentation.

[0085] In other embodiments, the image 101 processed by the image processor 108 may take the form of a CT image, and the segmentation tool 118 may be used to segment bone (e.g., the spine), lung tissue, or some other type of tissue. In some embodiments, the image 101 may be an MRI image of the brain, and the segmentation tool 118 is used to segment one or more ventricles of the brain.

[0086] In this manner, image processor 108 may be used to process different types of images (e.g., medical images / imaging) and identify regions of pixels that correspond to anatomical structures or features of interest (e.g., tissue, bone, organ, fluid layer, etc.). Additionally, image processor 108 may generate adjusted boundary definitions that take into account corrections received via user input. The adjustment data 126 generated by image processor 108 may be used in a variety of scenarios, including, for example, forming training inputs for supervised and / or semi-supervised training / retraining of machine learning models.

[0087] Thus, the set of anchor points can be utilized to reassign class labels for segmented regions of tissue. In this manner, the present technology can correct or adjust the machine learning model-based image segmentation or other image segmentation to improve accuracy. Furthermore, the corrected segmentation (e.g., adjusted image 126) can be used to retrain the segmentation model 118 and / or other models to further improve the performance and accuracy of these models. This can further ensure that the machine learning model-based image segmentation or other image segmentation, and any new pharmaceuticals or other treatments developed thereon, are reliable and suitable for clinical trials and approval for use by governmental or regulatory agencies.

[0088] III. Exemplary Image Analysis Methods 2A-2B form a flowchart of a process for generating calibration data according to one or more exemplary embodiments.

[0089] 2A is a flowchart of a process 200 for generating adjustment data according to one or more exemplary embodiments. Process 200 may be implemented using, for example, but not limited to, image analysis system 100 described with respect to FIGS. 1A-1B. For example, process 200 includes various operations (steps) that may be performed using image processor 108 of FIGS. 1A-1B.

[0090] Process 200 includes step 202 of receiving an image related to a portion of a target anatomical structure. The portion of the anatomical structure may be, for example, but not limited to, the retina of an eye, the brain, the lungs, the heart, or some other type of organ or anatomical structure, or portion thereof. The image received in operation 202 may be a 2D or 3D image. In one or more embodiments, the image is an MRI image, a CT image, an X-ray image, an ultrasound image, a PET image, a SPECT image, an OCT image (e.g., an OCT volume, an OCT B-scan), or some other type of 2D / 3D image capturing a portion of the anatomical structure. The image received in step 202 may be, for example, image 114 or another of images 101 described with respect to FIGS. 1A-1B.

[0091] In some embodiments, the image is a segmented image generated from an initial image processed using a machine learning model (e.g., a deep learning model). For example, the image received in step 202 may be another of segmented image 120 or segmented image 116 described with respect to FIGS. 1A-1B. The segmented image may have been generated using, for example, an artificial intelligence-based semantic segmentation model (e.g., segmentation model 118 described with respect to FIGS. 1A-1B).

[0092] In one or more embodiments, each group of pixels in the initial image may be classified as belonging to one of a plurality of selected categories (e.g., selected categories 128 of FIG. 1B) to form a “segment” in the segmented image. A segment may be a contiguous or discontinuous grouping of pixels. For example, a segment may be formed by one or more distinct regions of pixels that are each labeled as belonging to the same category. Alternatively, a segment may be a region of individual pixels, such that a particular grouping of pixels may be represented in the segmented image by one or more segments. In this manner, the segmented image includes a plurality of regions of pixels (e.g., region of pixels 130 of FIG. 1B).

[0093] The process 200 includes a step 204 that includes extracting a plurality of boundary points for an initial boundary associated with a corresponding region of pixels of the image.

[0094] If the image received in step 202 is a segmented image, extracting the plurality of boundary points may include identifying an initial boundary associated with at least one segment of the segmented image and then identifying pixels forming the initial boundary as boundary points. In some embodiments, extracting the plurality of boundary points includes identifying pixels forming the initial boundary as boundary points, such that identification of the initial boundary and the plurality of boundary points occurs simultaneously or substantially simultaneously. In other embodiments, the segmented image may include a curve identifying the initial boundary, such that step 204 includes identifying pixels forming the initial boundary as boundary points.

[0095] If the image received in step 202 is an image that has not yet been segmented, such as, for example, an OCT image, an MRI image, a CT image, an X-ray image, an ultrasound image, a PET image, a SPECT image, or some other type of image, process 200 may optionally include operation 204a, which includes segmenting the image to identify initial boundaries based on segments of the segmented image.

[0096] The multiple boundary points are associated with a sequential order. The sequential order may be defined with respect to a reference two-dimensional (2D) plane. If the image received in step 202 is a 2D image, the 2D plane may be the same X, Y plane of the image, which may be formed by the XY pixel coordinate system of the image itself. If the image is a 3D image, the sequential order may be defined with respect to multiple 2D planes (or slices) parallel to the reference 2D plane. The initial boundary described in step 204 may be, for example, the initial boundary 136 of FIG. 1B. The multiple boundary points described in step 204 may be, for example, the boundary point 138 of FIG. 1B.

[0097] Process 200 includes step 206, which includes evaluating a plurality of boundary points according to a sequential order to select a plurality of anchor points from the plurality of boundary points. The evaluation of the plurality of boundary points in step 206 may be performed via sub-operations (sub-steps). Examples of sub-operations that may be performed as part of step 206 are described in further detail below with respect to FIG. 2B.

[0098] Process 200 further includes step 208, which includes generating an anchor point image for display in a graphical user interface of a display device, the anchor point image including a plurality of anchor indicators representing the plurality of anchor points. The anchor point image may be, for example, anchor point image 142 including a plurality of anchor indicators 144 described in FIG. 1B.

[0099] In one or more embodiments, the anchor point image includes the image received in step 202, and the anchor indicators are overlaid or otherwise displayed on this image. As previously mentioned, this image may be the original image (e.g., an OCT image, an MRI image, a CT image, etc.) from which the segmented image was then generated. The anchor indicators may be controllable, movable, or otherwise manipulable graphical indicators (e.g., a graphic shape or icon indicating that the corresponding location has been determined to be an anchor point).

[0100] Process 200 may optionally include step 210, which includes receiving user input to adjust the position of at least one of the plurality of anchor indicators. For example, a user may enter user input that moves the anchor indicator to a different position that more accurately aligns with the boundary of the actual object of interest. The movement of the anchor indicator causes the corresponding anchor point to change from its original position to a different final position.

[0101] Process 200 may optionally include step 212, which includes generating training data based on user input. The training data may be, for example, training data 124 of FIGS. 1A-1B. The training data may be used to generate labeled images that may be used to train or retrain a machine learning model. For example, the labeled images may be used to train or retrain a segmentation model (e.g., segmentation model 118) used to generate the segmented images described above. In one or more embodiments, the training data includes a trained version of the segmented image in which one or more pixels of the segmented image are reclassified from a first category (or label) to a second category (or label) based on the final positions of the anchor points. This trained version of the segmented image (e.g., the trained segmented image) may be used to form training input for the segmentation model or a different model. Using the trained version of the segmented image may improve the accuracy and efficiency of the segmentation model in classifying pixels.

[0102] Process 200 may optionally include step 214, which includes performing accuracy improvement operations based on the training data. The accuracy improvement operations may include any number of operations themselves. The accuracy improvement operations may include generated adjusted images, which may label the images for use in retraining the segmentation model as described above. If the training data includes labeled images, the accuracy improvement operations may include retraining the segmentation model based on the labeled images to improve the performance of the segmentation model. The accuracy improvement operations may include using the training data as input images for, or to form, another algorithm or model to improve the accuracy of the model output compared to using the images 101 or the segmented images 116 as input.

[0103] 2B is a more detailed flowchart of step 206 of process 200 according to one or more exemplary embodiments. Performing step 206 may include performing various sub-operations, including, for example, operation 216, operation 218, and operation 220.

[0104] Act 216 includes determining that a first boundary point among the plurality of boundary points in sequential order is a first anchor point among the plurality of anchor points.

[0105] Operation 218 includes determining that the last boundary point of the plurality of boundary points in sequential order is the last anchor point of the plurality of anchor points. In some embodiments, operation 216 and operation 218 are performed as part of the same step.

[0106] Operation 220 includes, for each current boundary point of the plurality of boundary points being evaluated, determining that the current boundary point is a next anchor point of the plurality of anchor points when at least one perpendicular distance to a line extending between the previous anchor point and the current boundary point, calculated for a portion of the initial boundary located between the previous anchor point and the current boundary point, is greater than a selected threshold. Thus, each boundary point in the sequential order may be evaluated for inclusion in the plurality of anchor points based on one or more boundary points that come before and after the current boundary point in the sequential order being evaluated. The current boundary point may also be referred to as a checkpoint.

[0107] In operation 220, for example, one or more boundary points located between a previous anchor point and a current boundary point may be referred to as a set of intermediate boundary points. Operation 220 may include calculating a perpendicular distance from each intermediate boundary point of the set of intermediate boundary points to a line extending between the previous anchor point and the current boundary point to form a set of perpendicular distances. If at least one of these perpendicular distances is greater than a selected threshold, the current boundary point (i.e., check point) is determined to be the next anchor point. The selected threshold may be, for example, a distance between 1 and 15 in units of the reference 2D plane. The units may be in units of pixels.

[0108] The selected threshold may be selected to control the density of anchor points selected from the boundary points. In other words, the threshold may be selected to control the complexity of the curvature, grooves, texture, and / or contours in the boundary represented by the anchor points. A higher threshold may be selected to reduce the density (number) of anchor points and reduce the complexity of the curvature, grooves, texture, and / or contours in the captured boundary. In contrast, a lower threshold may be selected to increase the density (number) of anchor points and increase the complexity of the curvature, grooves, texture, and / or contours in the captured boundary.

[0109] If there are no intermediate boundary points located between the current boundary point being evaluated and the previous anchor point, and the current boundary point is not the last boundary point, then the next boundary point is evaluated.

[0110] 3A-3B together are a flowchart of a process for evaluating boundary points according to one or more exemplary embodiments. Process 300 may be implemented using, for example, but not limited to, image analysis system 100 described with respect to FIGS. 1A-1B. For example, process 300 includes various operations (steps) that may be performed using image processor 108 of FIGS. 1A-1B. Process 300 may be an example of implementing step 206 of FIGS. 2A-2B. Process 300 is specific to the selected 2D plane of boundary points being evaluated and may be performed for each 2D plane of boundary points being evaluated.

[0111] Step 302 includes determining that a first boundary point of a plurality of boundary points in a sequential order associated with the plurality of boundary points is a first anchor point.

[0112] Step 304 includes selecting the next boundary point of the plurality of boundary points according to sequential order as the current boundary point for evaluation.

[0113] Step 306 includes determining whether the current boundary point is the last boundary point of the plurality of boundary points in sequential order. If the current boundary point is the last boundary point of the plurality of boundary points in sequential order, process 300 performs step 308.

[0114] Step 308 involves determining that the last boundary point is the last anchor point, which is the last boundary point of the initial boundary, after which process 300 ends.

[0115] Referring again to step 308, if the current boundary point is not the last boundary point of the plurality of boundary points in sequential order, then process 300 performs step 310.

[0116] Step 310 involves determining whether the previous boundary point was selected as an anchor point. If the previous boundary point was selected as an anchor point, process 300 returns to step 304 described above. However, if the previous boundary point was not selected as an anchor point, process 300 performs step 312.

[0117] Step 312 includes calculating, for a portion of the initial boundary located between the previous anchor point and the current boundary point, a set of perpendicular distances to a line extending between the previous anchor point and the current boundary point. For example, the multiple boundary points of the initial boundary may include one or more boundary points between the previous anchor point and the current boundary point. This one or more boundary points may be referred to as a "set of intermediate boundary points." A perpendicular distance is calculated for each intermediate boundary point of this set of intermediate boundary points.

[0118] Step 314 includes determining whether at least one vertical distance of the set of vertical distances is calculated to be greater than a selected threshold, which may be, for example, a threshold selected between 1 and 10, where the threshold is in pixels. If no vertical distance of the set of vertical distances is greater than the selected threshold, process 300 returns to step 304 described above.

[0119] The threshold value may be selected to control the density of anchor points selected from the boundary points. In other words, the threshold value may be selected to control the complexity of the curvature, grooves, texture, and / or contours in the boundary represented by the anchor points. A higher threshold value may be selected to reduce the density (number) of anchor points and reduce the complexity of the curvature, grooves, texture, and / or contours in the captured boundary. In contrast, a lower threshold value may be selected to increase the density (number) of anchor points and increase the complexity of the curvature, grooves, texture, and / or contours in the captured boundary.

[0120] Referring back to step 314, if at least one vertical distance in the set of vertical distances is greater than the selected threshold, then process 300 performs step 316.

[0121] Step 316 involves determining that the current boundary point is the next anchor point, and process 300 then returns to operation 304 as described above.

[0122] 4 is a diagram of a workflow of a process for evaluating multiple boundary points that may be included in multiple anchor points, according to one or more embodiments. Workflow 400 may be an example of an implementation of step 206 of FIG. 2. Workflow 400 may be implemented using image processor 108 of FIGS. 1A-1B. For example, workflow 400 may be implemented using adjustment tool 112 of image processor 108 of FIGS. 1A-1B. Workflow 400 illustrates one manner in which at least a portion of process 300 described with reference to FIGS. 3A-3B may be implemented.

[0123] Initial boundary 402 is an example of an implementation of initial boundary 136 of Figure 1B. Initial boundary 402 may be a boundary identified from a segmented image (e.g., segmented image 120 of Figure 1). Image processor 108 extracts a plurality of boundary points 404 from initial boundary 402. The plurality of boundary points 404 may be an example of an implementation of plurality of boundary points 138 of Figure 1B.

[0124] The plurality of boundary points 404 includes, for example, boundary points 406, 408, 410, 412, 414, 416, and other points along the initial boundary 402 of Figure 4. Each boundary point of the plurality of boundary points 404 corresponds to a pixel in the segmented image and has an X,Y position (e.g., in pixels). The plurality of boundary points 404 are evaluated in sequential order (which is shown in Figure 4 as a left-to-right order). In workflow 400, anchor points are drawn using solid black, intermediate boundary points are drawn using a dotted pattern, and the current boundary being evaluated at a given step (i.e., checkpoint) is drawn using a vertical stripe pattern.

[0125] Since boundary point 306 is the first boundary point in sequential order from left to right, boundary point 406 is determined to be the first anchor point, and the next boundary point is selected for evaluation.

[0126] In step 418, boundary point 408 is the next to be evaluated. Since boundary point 406 immediately preceding boundary point 408 is the previous anchor point (e.g., the first anchor point), the next boundary point is selected for evaluation.

[0127] In step 420, boundary point 410 is the next to be evaluated. Here, boundary point 408 is the midpoint boundary point between the previous anchor point (boundary point 406) and the current boundary point (boundary point 410). Line 421 extends between the previous anchor point (boundary point 406) and the current boundary point (boundary point 410). A determination is made as to whether the vertical distance from boundary point 408 to line 421 is greater than a selected threshold (e.g., 1, 1.25, 1.5, 2, 2.5, 3, 3.5, 4, 5, 6, 7, 8, 9, 10, etc., or some other number of pixel units between 1 and 15). Here, the calculated vertical distance is less than or equal to the threshold, so the next boundary point is selected for evaluation.

[0128] In step 422, boundary point 412 is evaluated. Here, boundary points 408 and 410 are intermediate boundary points between the previous anchor point (boundary point 406) and the current boundary point (boundary point 412). Line 423 extends between the previous anchor point (boundary point 406) and the current boundary point (boundary point 412). The perpendicular distance from boundary point 408 to line 423 is calculated. The perpendicular distance from boundary point 410 to line 423 is calculated. It is determined whether any one of these perpendicular distances is greater than a selected threshold. Here, since none of the calculated perpendicular distances is greater than the selected threshold, the next boundary point is selected for evaluation.

[0129] In step 424, boundary point 414 is evaluated. Here, boundary points 408, 410, and 412 are intermediate boundary points between the previous anchor point (boundary point 406) and the current boundary point (boundary point 414). A line 425 extends between the previous anchor point (boundary point 406) and the current boundary point (boundary point 414). The perpendicular distance from boundary point 408 to line 425 is calculated. The perpendicular distance from boundary point 410 to line 425 is calculated. The perpendicular distance from boundary point 412 to line 425 is calculated. A determination is made as to whether at least one of these perpendicular distances is greater than a selected threshold. Here, it is determined that the calculated perpendicular distance from boundary point 410 to line 425 is greater than the threshold, so the current boundary point (boundary point 414) is determined to be the next anchor point, and the next boundary point is selected for evaluation.

[0130] In step 426, boundary point 416 is evaluated. In step 426, boundary point 414 is now the previously identified anchor point. In step 426, the next boundary point is selected for evaluation because boundary point 414 immediately preceding boundary point 416 is the previous anchor point.

[0131] The above-described evaluation process is completed for each of the remaining boundary points, with the last of the boundary points being selected as the last (or final) anchor point of the initial boundary 402 .

[0132] 5 is a diagram illustrating how perpendicular distance is calculated in accordance with one or more example embodiments. In FIG. 5, previous anchor point 500 is a previously identified anchor point, such as, for example, but not limited to, boundary point 406 in FIG. 4. Current boundary point 502 (sometimes referred to as a checkpoint) is the current boundary point being evaluated. Set of intermediate boundary points 503 may include all boundary points between previous anchor point 500 and current boundary point 502. Here, set of intermediate boundary points 503 includes intermediate boundary point 504, intermediate boundary point 506, and intermediate boundary point 508.

[0133] The line 510 may be a line (e.g., a theoretical / abstract / imaginary / virtual / calculated line) extending between the previous anchor point 500 and the current boundary point 502. A perpendicular distance is calculated for each intermediate boundary point in the set of intermediate boundary points 503. The perpendicular distance may be measured as the length of a line extending perpendicularly between the location of the point of interest (e.g., the intermediate boundary point) and a reference line (e.g., the line extending between the previous anchor point and the current boundary point).

[0134] Here, vertical distance 512, vertical distance 514, and vertical distance 516 are calculated for boundary point 504, boundary point 506, and boundary point 508, respectively, with respect to line 510. If any one of these vertical distances is determined to be greater than a selected threshold, then current boundary point 502 is determined to be the next anchor point.

[0135] The threshold value may be selected to control the density of anchor points selected from the boundary points. In other words, the threshold value may be selected to control the complexity of the curvature, grooves, texture, and / or contours in the boundary represented by the anchor points. A higher threshold value may be selected to reduce the density (number) of anchor points and reduce the complexity of the curvature, grooves, texture, and / or contours in the captured boundary. In contrast, a lower threshold value may be selected to increase the density (number) of anchor points and increase the complexity of the curvature, grooves, texture, and / or contours in the captured boundary.

[0136] Enabling a user to correct identified boundaries in a segmented image using anchor points and anchor indicators representing those anchor points in the manner described above with respect to Figures 2A, 2B, 3A, 3B, 4, and 5 reduces the overall time and resources required to adjust / correct a segmented image to form an adjusted image. Because the anchor indicators represent only a subset of the boundary points extracted for a given boundary, adjustments to the entire boundary may be made more efficiently without the user having to adjust each boundary point (pixel) of that boundary. Furthermore, fewer anchor indicators mean fewer overall adjustment calculations need to be performed without sacrificing overall accuracy. Thus, generating anchor points and a graphical user interface displaying controllable / movable anchor indicators representing the anchor points may improve the overall functionality of an image analysis system performing the methods described herein and reduce the overall consumption of computing resources.

[0137] IV. Graphical User Interface Display Examples 6-16 illustrate types of images that may be displayed in a graphical user interface according to one or more exemplary embodiments. The graphical user interface may be, for example, the graphical user interface 122 of FIGS. 1A-1B. The different images shown in FIGS. 6-16 illustrate how an image (e.g., an OCT B-scan) may be processed using the image analysis system 100 described with reference to FIGS. 1A-1B and the methods for identifying boundaries of an object of interest, corresponding boundary points, and corresponding anchor points described in connection with FIGS. 2A, 2B, 3A, 3B, 4, and 5. These anchor points may be displayed on the image using user-operable anchor indicators to allow the user to make adjustments to more accurately identify the boundary of the object of interest.

[0138] 6 is a diagram of an image displayed on a graphical user interface in accordance with one or more exemplary embodiments. Image 600 is an example of an implementation of image 114 described with respect to FIGS. 1A-1B. Image 600 may be a retinal image capturing a subject's retina. For example, image 600 may be an OCT-B scan.

[0139] FIG. 7 is an illustration of a segmented image displayed on a graphical user interface, according to one or more exemplary embodiments. Segmented image 700 is an example of an implementation of segmented image 120 described with respect to FIGS. 1A-1B. Segmented image 700 may have been generated, for example, using segmentation model 118 of segmentation tool 110 of image processor 108 of FIGS. 1A-1B. Segmented image 700 identifies multiple pixel regions 702. In this example, each pixel region is a "segment" corresponding to a different retinal element. For example, image 600 of FIG. 6 may be segmented to detect and identify one or more retinal (e.g., retina-related) elements. The retinal elements may consist of at least one of retinal layer elements or retinal pathology elements.

[0140] 8 is a diagram of an image with identified boundaries displayed in a graphical user interface in accordance with one or more exemplary embodiments. Image 600 from FIG. 6 is displayed with initial boundary set 802 superimposed on image 600. Initial boundary set 802 includes initial boundary 804, initial boundary 806, initial boundary 808, and initial boundary 810. Initial boundary set 802 is an example of an implementation of initial boundary set 134 of FIG. 1B.

[0141] 9 is an illustration of an image displayed with boundary indicators in a graphical user interface, according to one or more exemplary embodiments. Image 600 from FIG. 6 is displayed with boundary indicators 900 that represent corresponding boundary points extracted for initial boundary 804 from FIG. 8. The boundary points represented by boundary indicators 900 may be an example of an implementation of boundary points 138 of FIG. 1B.

[0142] 10 is a diagram of an image including anchor indicators displayed in a graphical user interface in accordance with one or more exemplary embodiments. Image 600 of FIG. 6 is displayed with anchor indicators 1000 representing anchor points determined for initial boundary 804 of FIG. 8. Anchor indicators 1000 may be an example of an implementation of anchor indicator 144 of FIG. 1B. Anchor indicators 1000 are connected by line segments 1002 that form new boundary 1004. New boundary 1004 approximates initial boundary 804 from FIG. 8 and reduces the curvature, grooves, texture, and / or contour complexity of initial boundary 804 within a selected tolerance.

[0143] 11 is a diagram of an image displayed with boundary indicators in a graphical user interface according to one or more exemplary embodiments. Image 600 of FIG. 6 is displayed with boundary indicators 1100 that represent boundary points extracted for initial boundary 806 of FIG. 8. The boundary points represented by boundary indicators 1100 may be an example of an implementation of boundary points 138 of FIG. 1B.

[0144] 12 is a diagram of an image including anchor indicators displayed in a graphical user interface in accordance with one or more exemplary embodiments. Image 600 from FIG. 6 is displayed with anchor indicators 1200 representing anchor points determined for initial boundary 806 from FIG. 8. Anchor indicators 1200 may be an example of an implementation of anchor indicator 144 of FIG. 1B. Anchor indicators 1200 are connected by line segments 1202 that form new boundary 1204. New boundary 1204 approximates initial boundary 806 from FIG. 8 and reduces the curvature, grooves, texture, and / or contour complexity of initial boundary 804 within a selected tolerance.

[0145] Figure 13 is an illustration of an image displayed with boundary indicators in a graphical user interface, according to one or more exemplary embodiments. Image 600 of Figure 6 is displayed with boundary indicators 1300 that represent boundary points extracted for initial boundary 808 of Figure 8. The boundary points represented by boundary indicators 1300 may be an example of an implementation of boundary points 138 of Figure 1B.

[0146] 14 is an illustration of an image including anchor points displayed in a graphical user interface, according to one or more exemplary embodiments. Image 600 from FIG. 6 is displayed with anchor indicators 1400 representing anchor points determined for initial boundary 808 from FIG. 8. Anchor indicators 1400 may be an example of an implementation of anchor indicator 144 of FIG. 1B. Anchor indicators 1400 are connected by line segments 1402 that form a new boundary 1404. New boundary 1404 approximates initial boundary 808 from FIG. 8 and reduces the curvature, grooves, texture, and / or contour complexity of initial boundary 804 within a selected tolerance.

[0147] 15 is an illustration of an image displayed with boundary indicators in a graphical user interface according to one or more exemplary embodiments. Image 600 of FIG. 6 is displayed with boundary indicators 1500 representing boundary points extracted for initial boundary 810 of FIG. 8. The boundary points represented by boundary indicators 1500 may be an example of an implementation of boundary points 138 of FIG. 1B.

[0148] 16 is an illustration of an image including anchor points displayed in a graphical user interface, according to one or more exemplary embodiments. Image 600 from FIG. 6 is displayed with anchor indicators 1600 representing anchor points determined for initial boundary 810 from FIG. 8. Anchor indicators 1600 may be an example of an implementation of anchor indicator 144 of FIG. 1B. Anchor indicators 1600 are connected by line segments 1602 that form new boundary 1604. New boundary 1604 approximates initial boundary 810 from FIG. 8 and reduces the curvature, grooves, texture, and / or contour complexity of initial boundary 804 within a selected tolerance.

[0149] As previously mentioned, the automatic selection of anchor points and the display of corresponding anchor indicators as shown in Figures 10, 12, 14, and 16 allows a user (e.g., a medical expert, medical specialist, human pathologist, human grader, reading center, etc.) to easily provide user input that can then be used to automatically adjust previously identified boundaries and form new boundaries that more accurately represent the boundaries of the subject of interest.

[0150] The use of anchor points, and anchor indicators representing those anchor points, in the manner described above with respect to Figures 6-16 may enable adjustments / corrections of the segmented image 700 in Figure 7 in a manner that reduces the overall time and resources required to make these adjustments / corrections without sacrificing accuracy. Because the anchor indicators represent only a subset of the boundary points extracted for a given boundary, adjustments may be made more efficiently to the entire boundary without requiring the user to adjust each boundary point (pixel) of that boundary. Furthermore, fewer anchor indicators means fewer overall adjustment calculations need to be performed without sacrificing overall accuracy. Therefore, generating anchor points and a graphical user interface that displays controllable / movable anchor indicators representing the anchor points may improve the overall functionality of the image analysis system described herein and reduce the overall consumption of computing resources.

[0151] Additionally, the image processor may generate adjustment data (e.g., adjusted images 126 as described with respect to FIGS. 1A-1B) that may include or be used to generate new segmented images in which pixel classifications more accurately reflect the subject's actual anatomy / lesion. In some cases, the new segmented images (adjusted images) may be used to better train and / or retrain the machine learning model to improve segmentation performance. This type of improvement may help ensure that machine learning model-based image segmentation and / or other image segmentations, and new pharmaceuticals or other treatments developed thereon, are reliable and may be suitable for clinical trials and approval for use by governmental or regulatory agencies.

[0152] For example, segmentation of retinal layers can be used to determine whether a lesion is present in the retina and track changes in one or more lesions over time. Thus, delineating the precise boundaries of each retinal region or layer allows for proper identification and tracking of retinal lesions. In certain embodiments, the set of anchor points can be configured to adjust the boundaries to accurately label one or more retinal lesions.

[0153] 17-20 are diagrams of various displays that may be presented in a graphical user interface, according to one or more exemplary embodiments. Graphical user interface 1700 is an example of one implementation of graphical user interface 122 of FIGS. 1A-1B.

[0154] 17 is an illustration of a graphical user interface that converts a display of boundaries into anchor indicators, according to one or more exemplary embodiments. On the left, graphical user interface 1700 displays image 1701. Image 1701 is a high-resolution image (e.g., an MRI scan of a subject's brain). Image 1701 is displayed with boundaries 1702 that identify the ventricles of the brain.

[0155] 1A-1B may be used to convert a representation of boundary 1702 into a representation of anchor indicators 1704 on image 1701, as shown on the right. Anchor indicators 1704, when connected via line segments as shown, approximate boundary 1702 within a selected tolerance. Anchor indicators 144, which may be one example of an implementation of anchor indicators 1704 in FIG. 1B, represent anchor points. Each of anchor indicators 1704 is a movable graphic indicator that can be moved via user input to adjust the position of the corresponding anchor point.

[0156] 18 is an illustration of a graphical user interface that converts a display of a boundary into an anchor indicator, according to one or more exemplary embodiments. On the left, graphical user interface 1800 displays image 1801. Image 1801 is a high-resolution image (e.g., a CT scan of a subject's chest). Image 1801 is displayed with boundary 1802 that identifies an area of ​​bony tissue (e.g., a vertebra).

[0157] 1A-1B may be used to convert a representation of boundary 1802 into a representation of anchor indicators 1804 on image 1801, as shown on the right. Anchor indicators 1804, when connected via line segments as shown, approximate boundary 1802 within a selected tolerance. Anchor indicators 144, which may be one example of an implementation of anchor indicators 1804 in FIG. 1B, represent anchor points. Each of anchor indicators 1804 is a movable graphic indicator that can be moved via user input to adjust the position of the corresponding anchor point.

[0158] 19 is an illustration of a graphical user interface that converts a display of a boundary into an anchor indicator, according to one or more exemplary embodiments. On the left, graphical user interface 1900 displays image 1901. Image 1901 is a high-resolution image (e.g., a CT scan of a subject's chest). Image 1901 is displayed with boundary 1902 that identifies a region of lung tissue (e.g., a lung lobe).

[0159] 1A-1B may be used to convert a representation of boundary 1902 into a representation of anchor indicators 1904 on image 1901, as shown on the right. Anchor indicators 1904, when connected via line segments as shown, approximate boundary 1902 within a selected tolerance. Anchor indicators 144, which may be one example of an implementation of anchor indicators 1904 in FIG. 1B, represent anchor points. Each of anchor indicators 1904 is a movable graphic indicator that can be moved via user input to adjust the position of the corresponding anchor point.

[0160] 20 is an illustration of the graphical user interface from FIG. 19 showing changes in anchor point density, according to one or more exemplary embodiments. The anchor point density changes in response to changes in the selected threshold used to evaluate vertical distance, as described with respect to embodiments herein (e.g., as described in FIGS. 2A, 2B, 3A, 3B). On the left side, anchor indicator 1904 from FIG. 19 is shown. Lowering the selected threshold used to calculate vertical distance as described herein reduces the density of selected anchor points, and therefore the number of displayed anchor indicators, as shown on the right side with anchor indicator 2000.

[0161] V. Exemplary Computing System 21 is a block diagram illustrating an example of a computing system 2100, according to one or more exemplary embodiments. The computing system 2100 may be used to implement the computing platform 102 and / or the remote system 107 of FIG. 1A and / or any components therein.

[0162] In one or more examples, computer system 2100 may include a bus 2102 or other communication mechanism for communicating information, and a processor 2104 coupled with bus 2102 for processing information. In various embodiments, computer system 2100 may also include memory, which may be a random access memory (RAM) 2106 or other dynamic storage device, coupled to bus 2102 for determining instructions to be executed by processor 2104. Memory may also be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 2104. In various embodiments, computer system 2100 may further include a read-only memory (ROM) 2108 or other static storage device coupled to bus 2102 for storing static information and instructions for processor 2104. A storage device 2110, such as a magnetic disk or optical disk, may be provided and coupled to bus 2102 for storing information and instructions.

[0163] In various embodiments, the computer system 2100 may be coupled via the bus 2102 to a display 2112, such as a cathode ray tube (CRT) or liquid crystal display (LCD), for displaying information to a computer user. An input device 2114, including alphanumeric and other keys, may be coupled to the bus 2102 for communicating information and command selections to the processor 2104. Another type of user input device is a cursor control device 2116, such as a mouse, joystick, trackball, gesture input device, eye-gaze-based input device, or cursor direction keys, for communicating directional information and command selections to the processor 2104 and for controlling cursor movement on the display 2112. This input device 2114 typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allow the device to specify a position in a plane. However, it should be understood that input devices 2114 that allow three-dimensional (e.g., x, y, and z) cursor movement are also contemplated herein.

[0164] Consistent with particular implementations of the present teachings, results may be produced by computer system 2100 in response to processor 2104 executing one or more sequences of one or more instructions contained in RAM 2106. Such instructions may be read into RAM 2106 from another computer-readable medium or computer-readable storage medium, such as storage device 2110. Execution of the sequences of instructions contained in RAM 2106 may cause processor 2104 to perform the processes described herein. Alternatively, hard-wired circuitry may be used in place of or in combination with software instructions to implement the present teachings. Thus, implementations of the present teachings are not limited to any specific combination of hardware circuitry and software.

[0165] As used herein, the terms “computer-readable medium” (e.g., data store, data storage, data storage device, etc.) or “computer-readable storage medium” refer to any medium that participates in providing instructions to processor 2104 for execution. Such a medium may take many forms, including, but not limited to, non-volatile media, volatile media, and transmission media. Examples of non-volatile media may include, but are not limited to, optical, solid-state, and magnetic disks, such as storage device(s) 2110. Examples of volatile media may include, but are not limited to, dynamic memory, such as RAM 2106. Examples of transmission media may include, but are not limited to, coaxial cables, copper wire, and fiber optics, including the wires that comprise bus 2102.

[0166] Common forms of computer-readable media include, for example, floppy disks, hard disks, magnetic tape or any other magnetic medium, CD-ROMs, any other optical medium, punch cards, paper tape, any other physical medium with a pattern of holes, RAM, PROMs, and EPROMs, flash EPROMs, any other memory chip or cartridge, or any other tangible medium from which a computer can read.

[0167] In addition to computer-readable media, instructions or data may be provided as signals on a transmission medium included in a communication device or system to provide one or more sequences of instructions to the processor 2104 of the computer system 2100 for execution. For example, a communication device may include a transceiver having signals indicative of instructions and data. The instructions and data are configured to cause one or more processors to implement the functions outlined in the disclosure herein. Representative examples of data communication transmission connections may include, but are not limited to, a telephone modem connection, a wide area network (WAN), a local area network (LAN), an infrared data connection, an NFC connection, an optical communication connection, etc.

[0168] The methodologies described herein may be implemented by various means depending on the application. For example, the methodologies may be implemented in hardware, firmware, software, or any combination thereof. In a hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processors (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, or combinations thereof.

[0169] In various embodiments, the methods of the present teachings may be implemented as firmware and / or software programs and applications written in conventional programming languages ​​such as C, C++, Python, etc. When implemented as firmware and / or software, the embodiments described herein may be implemented in a non-transitory computer-readable medium having stored thereon a program for causing a computer to perform the above-described methods. It should be understood that the various engines described herein may be provided on a computer system, such as computer system 2100, whereby processor 2104 performs the analyses and decisions provided by these engines in response to instructions provided by any one or combination of memory components RAM 2106, ROM 2108, or storage device 2110, and user input provided via input device 2114.

[0170] In some exemplary embodiments, the computing system 2100 may be used to execute various interactive computer software applications that may be used for organizing, analyzing, and / or storing various forms of data. Alternatively, the computing system 2100 may be used to execute any type of software application. These applications may be used to perform various functions, such as planning functions (e.g., creating, managing, editing spreadsheet documents, word processing documents, and / or other objects), computing functions, communication functions, etc. Applications may include various add-in functions or may be standalone computing products and / or functions. When active within an application, functionality may be used to generate a user interface that is provided via the input / output devices 2114. The user interface may be generated by the computing system 2100 and presented to a user (e.g., on a computer screen monitor, etc.).

[0171] One or more aspects or features of the subject matter described herein may be implemented in digital electronic circuitry, integrated circuits, specially designed ASICs, field programmable gate array (FPGA) computer hardware, firmware, software, and / or combinations thereof. These various aspects or features may include implementation in one or more computer programs executable and / or interpretable on a programmable system including at least one programmable processor, which may be special-purpose or general-purpose, coupled to receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device. The programmable system or computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communications network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0172] These computer programs, sometimes referred to as programs, software, software applications, applications, components, or code, contain machine instructions for a programmable processor and may be implemented in a high-level procedural and / or object-oriented programming language and / or in an assembly / machine language. As used herein, the term “machine-readable medium” refers to any computer program product, apparatus, and / or device used to provide machine instructions and / or data to a programmable processor, such as, for example, magnetic disks, optical disks, memory, and programmable logic devices (PLDs), including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor. A machine-readable medium may non-transitory store such machine instructions, such as, for example, a non-transitory solid-state memory, a magnetic hard drive, or any equivalent storage medium. Alternatively or additionally, a machine-readable medium may temporarily store such machine instructions, such as, for example, a processor cache or other random access memory associated with one or more physical processor cores.

[0173] VI. Enumeration of Exemplary Embodiments The present disclosure is not limited to these exemplary embodiments and applications or the manner in which the exemplary embodiments and applications operate or are described herein. Further, the figures may show simplified or partial views, and the dimensions of the elements in the figures may be exaggerated or out of proportion.

[0174] 1. A method comprising: receiving an image relating to a portion of a target anatomical structure; extracting a plurality of boundary points for an initial boundary associated with corresponding regions of pixels in the image, the plurality of boundary points being associated in a sequential order for a selected two-dimensional plane corresponding to the image; evaluating the plurality of boundary points according to the sequential order to select a plurality of anchor points from the plurality of boundary points; and generating an anchor point image for display in a graphical user interface of a display device, the anchor point image including a plurality of anchor indicators representing the plurality of anchor points. The evaluating includes determining that a first boundary point of the plurality of boundary points is a first anchor point; and determining that a current boundary point of the plurality of boundary points being evaluated is a next anchor point of the plurality of anchor points when at least one perpendicular distance to a line extending between the previous anchor point and the current boundary point, calculated for a portion of the initial boundary located between the previous anchor point and the current boundary point, is greater than a selected threshold.

[0175] Embodiment 2. The method of embodiment 1, wherein the evaluating further includes determining that the current boundary point being evaluated is not a next anchor point of the plurality of anchor points when a perpendicular distance calculated for a portion of the initial boundary located between the previous anchor point and the current boundary point relative to a line extending between the previous anchor point and the current boundary point is not greater than a selected threshold.

[0176] Embodiment 3. The method of embodiment 1 or embodiment 2, wherein the evaluating further includes determining that the current boundary point being evaluated is not a next anchor point among the plurality of anchor points when there is no boundary point in consecutive order between the previous anchor point and the current boundary point being evaluated.

[0177] Embodiment 4. The method of any one of embodiments 1 to 3, further comprising receiving a user input in a graphical user interface that moves at least one anchor indicator of the plurality of anchor indicators.

[0178] Embodiment 5. The method of embodiment 4, further comprising adjusting, based on user input, an original position of at least one anchor point of the plurality of anchor points corresponding to the at least one anchor indicator, such that the plurality of anchor points have a final position.

[0179] Embodiment 6. The method of embodiment 4 or embodiment 5, further comprising generating adjustment data based on user input, the adjustment data including an adjusted segmented image, and generating the adjustment data in which at least some pixels of the segmented image are reclassified to form the adjusted segmented image based on the final positions of the multiple anchor points.

[0180] Embodiment 7. The method of embodiment 6, further comprising retraining a segmentation model, including a machine learning model, using the adjusted segmented image.

[0181] Embodiment 8. The method of any one of embodiments 1 to 7, wherein evaluating further comprises determining that a last boundary point of the plurality of boundary points is a last anchor point.

[0182] Embodiment 9. The method of any one of embodiments 1 to 8, wherein determining that a current boundary point of the plurality of boundary points being evaluated is a next anchor point of the plurality of anchor points includes: calculating a perpendicular distance between each intermediate boundary point of the plurality of boundary points located between the previous anchor point and the current boundary point and a line extending between the previous anchor point and the current boundary point to form a set of perpendicular distances; determining that at least one perpendicular distance of the set of perpendicular distances is greater than a selected threshold; and determining that the current boundary point is to be the next anchor point in response to determining that at least one perpendicular distance of the set of perpendicular distances is greater than the selected threshold.

[0183] Embodiment 10. The method of any one of embodiments 1 to 9, wherein the selected threshold is selected to be between 1 and 10 pixel units.

[0184] Embodiment 11. A method according to any one of embodiments 1 to 10, wherein the anchor point image displayed in the graphical user interface further includes a new boundary formed by line segments connecting multiple anchor indicators representing multiple anchor points.

[0185] Embodiment 12. A method according to any one of embodiments 1 to 11, further comprising: receiving user input in a graphical user interface to move at least one anchor indicator of a plurality of anchor indicators; adjusting an initial boundary to form a new boundary based on the user input; generating an adjusted image in which the new boundary is overlaid on the image; and using the adjusted image to form input for an algorithm.

[0186] Embodiment 13. The method of any one of embodiments 1 to 12, wherein the image comprises a magnetic resonance imaging (MRI) image, a computed tomography (CT) image, an X-ray image, an ultrasound image, a positron emission tomography (PET) image, a single photon emission computed tomography (SPECT) image, or an optical coherence tomography (OCT) image.

[0187] Embodiment 14. The method of any one of embodiments 1 to 13, wherein the image is a two-dimensional image or a three-dimensional image.

[0188] Embodiment 15. A method comprising: receiving a plurality of images relating to a portion of an anatomical structure of a target; performing segmentation using the plurality of images and a segmentation model to generate a plurality of segmented images; extracting, for each segmented image among the plurality of segmented images, a plurality of boundary points for each initial boundary of a set of initial boundaries associated with each segmented image, the plurality of boundary points being associated in a sequential order for a selected two-dimensional plane corresponding to the plurality of images; evaluating, for each segmented image among the plurality of segmented images, the plurality of boundary points in a sequential order for each initial boundary of the set of initial boundaries associated with each segmented image to select a plurality of anchor points from the plurality of boundary points for each initial boundary of the set of initial boundaries identified in each image; and generating an anchor point image for display in a graphical user interface of a display device, the anchor point image including a plurality of anchor indicators representing a plurality of line segments connecting the plurality of anchor points and the plurality of anchor indicators. The evaluating includes determining a first boundary point of the plurality of boundary points to be a first anchor point, determining a last boundary point of the plurality of boundary points to be a last anchor point, and determining a current boundary point of the plurality of boundary points being evaluated to be a next anchor point of the plurality of anchor points when at least one perpendicular distance of a set of perpendicular distances calculated for a corresponding set of intermediate boundary points to a line extending between the previous anchor point and the current boundary point is greater than a selected threshold.

[0189] Embodiment 16. The method of embodiment 15, wherein the segmentation model includes a machine learning model, and further comprising: receiving user input to adjust the position of at least one of the plurality of anchor indicators; adjusting an original position of at least one anchor point of the plurality of anchor points corresponding to at least one of the plurality of anchor indicators to form a final position of the plurality of anchor points; and modifying a corresponding segmented image of the plurality of segmented images using the final positions of the plurality of anchor points to form an adjusted segmented image for use in retraining the segmentation model.

[0190] Embodiment 17. A system including one or more computing devices, the system comprising: one or more non-transitory computer-readable storage media containing instructions; and one or more processors coupled to the one or more storage media, the one or more processors configured to execute instructions for: receiving an image related to a portion of an anatomical structure of a target; extracting a plurality of boundary points for an initial boundary associated with a region of corresponding pixels in the image, the plurality of boundary points being associated in a sequential order for a selected two-dimensional plane corresponding to the image; evaluating the plurality of boundary points according to the sequential order to select a plurality of anchor points from the plurality of boundary points; and generating an anchor point image for display in a graphical user interface of a display device, the anchor point image including a plurality of anchor indicators representing the plurality of anchor points. The evaluating includes determining that a first boundary point of the plurality of boundary points is a first anchor point, and determining that a current boundary point of the plurality of boundary points being evaluated is a next anchor point of the plurality of anchor points when at least one perpendicular distance to a line extending between the previous anchor point and the current boundary point, calculated for a portion of the initial boundary located between the previous anchor point and the current boundary point, is greater than a selected threshold.

[0191] Embodiment 18. The system of embodiment 17, wherein the one or more processors are configured to execute instructions to determine that the current boundary point being evaluated is not a next anchor point of the plurality of anchor points when there is no boundary point along the initial boundary between the previous anchor point and the current boundary point being evaluated.

[0192] Embodiment 19. The system described in embodiment 17 or embodiment 18, wherein the one or more processors are further configured to execute instructions for receiving user input that moves at least one anchor indicator of the plurality of anchor indicators in the graphical user interface.

[0193] Embodiment 20. The system described in embodiment 19, wherein the one or more processors are further configured to execute instructions for adjusting an original position of at least one anchor point among the plurality of anchor points corresponding to at least one anchor indicator based on user input to form a final position for the plurality of anchor points.

[0194] Embodiment 21. The system described in embodiment 19 or embodiment 20, wherein one or more processors are further configured to execute instructions for generating adjustment data based on user input, the adjustment data including an adjusted segmented image, and wherein at least some pixels of the segmented image are reclassified to form the adjusted segmented image based on the final positions of the multiple anchor points.

[0195] Embodiment 22. The system described in embodiment 21, wherein the one or more processors are further configured to execute instructions for retraining a segmentation model, including a machine learning model, using the adjusted segmented image.

[0196] Embodiment 23. A system described in any one of embodiments 16 to 22, wherein the one or more processors are further configured to execute instructions for determining that the last boundary point of the plurality of boundary points is the last anchor point.

[0197] Embodiment 24. The system described in any one of embodiments 16 to 23, wherein the one or more processors are further configured to execute instructions to calculate a vertical distance between each intermediate boundary point of a plurality of boundary points located between the previous anchor point and the current boundary point and a line extending between the previous anchor point and the current boundary point to form a set of vertical distances, determine that at least one vertical distance of the set of vertical distances is greater than a selected threshold, and determine that the current boundary point will be the next anchor point in response to determining that at least one vertical distance of the set of vertical distances is greater than the selected threshold.

[0198] Embodiment 25. A system described in any one of embodiments 16 to 24, wherein the selected threshold is selected to be in the range of 1 to 10 pixels.

[0199] Embodiment 26. A system described in any one of embodiments 16 to 25, wherein the anchor point image displayed in the graphical user interface further includes a new boundary formed by line segments connecting multiple anchor indicators representing multiple anchor points.

[0200] Embodiment 27. The system of any one of embodiments 16 to 26, wherein the image comprises a magnetic resonance imaging (MRI) image, a computed tomography (CT) image, an X-ray image, an ultrasound image, a positron emission tomography (PET) image, a single photon emission computed tomography (SPECT) image, or an optical coherence tomography (OCT) image.

[0201] Embodiment 28. A system described in any one of embodiments 16 to 27, wherein the image is a two-dimensional image or a three-dimensional image.

[0202] Additional Embodiments Embodiment A1: A method comprising: receiving, by one or more computing devices, a medical image of a patient; segmenting the medical image into regions of pixels, the regions of pixels including at least a first region of pixels and a second region of pixels; extracting a set of boundary points along an identified boundary between the first region of pixels and the second region of pixels; determining consecutive pairs of anchor points, the consecutive pairs of anchor points being configured to be utilized to adjust the identified boundary between the first region of pixels and the second region of pixels, wherein determining the consecutive pairs of anchor points comprises identifying a starting anchor point, where for a first one of the identified pairs of boundary points, the starting anchor point is a first boundary point of the set of boundary points and for a subsequent one of the identified pairs of boundary points, the starting anchor point is an ending anchor point of a previously identified pair of boundary points; and identifying an ending anchor point, where a curvature between the starting anchor point and the ending anchor point satisfies a predetermined criterion.

[0203] Embodiment A2: The method of embodiment A1, further comprising: displaying a second image on a display of one or more other computing devices based on the consecutive pairs of anchor points; and adjusting an identified boundary between the first region of pixels and the second region of pixels in response to determining one or more user inputs corresponding to manipulation of at least one of the consecutive pairs of anchor points.

[0204] Embodiment A3: The method of embodiment A1, wherein adjusting the identified boundary between the first region of pixels and the second region of pixels comprises adjusting a contour of the identified boundary.

[0205] Embodiment A4: The method described in embodiment A1, wherein adjusting the identified boundary between the first pixel region and the second pixel region comprises correcting the class label corresponding to the first pixel region or the second pixel region.

[0206] Embodiment A5: The method described in embodiment A4, wherein correcting the class label corresponding to the first pixel region or the second pixel region is performed in response to receiving one or more user inputs from a human pathologist.

[0207] Embodiment A6: The method of embodiment A1, wherein the curvature between the start anchor point and the end anchor point comprises the perpendicular distance between the start anchor point and the end anchor point.

[0208] Embodiment A7: The method of embodiment A1, wherein the curvature between the start anchor point and the end anchor point meets a predetermined criterion when the curvature exceeds a perpendicular distance threshold.

[0209] Embodiment A8: The method of embodiment A7, wherein the vertical distance threshold comprises a vertical distance of approximately 1 pixel.

[0210] Embodiment A9: The method of embodiment A7, wherein the vertical distance threshold comprises a vertical distance of approximately 2 pixels.

[0211] Embodiment A10: The method of embodiment A7, wherein the vertical distance threshold comprises a user-configurable threshold.

[0212] Embodiment A11: The method of embodiment A10, wherein the user-configurable threshold is configured to be adjusted to adjust the identified boundary between the first pixel region and the second pixel region according to a desired accuracy.

[0213] Embodiment A12: The method of embodiment A11, wherein the desired accuracy varies based on the total number of consecutive pairs of anchor points.

[0214] Embodiment A13: The method of embodiment A1, wherein for the last one of the identified pairs of boundary points, the ending anchor point is the last boundary point of the set of boundary points.

[0215] Embodiment A14: The method of embodiment A1, wherein determining consecutive pairs of anchor points further includes ceasing to identify the end anchor point if the curvature between the start anchor point and the end anchor point does not meet a predetermined criterion.

[0216] Embodiment A15: The method of embodiment A1, wherein identifying regions of pixels in the image comprises segmenting regions of pixels indicative of normal regions and regions of pixels indicative of disease regions.

[0217] Embodiment A16: The method described in embodiment A1, wherein identifying regions of multiple pixels in the image further includes inputting the image to a machine learning model trained to segment regions of multiple pixels in the image, and utilizing the machine learning model to segment at least a first region of pixels and a second region of pixels, and outputting a predicted class label for each of the first region of pixels and the second region of pixels.

[0218] Embodiment A17: The method described in embodiment A16, further comprising: displaying a second image based on a predicted class label for each of the first pixel region and the second pixel region; receiving one or more user inputs corresponding to a request to update the predicted class label for at least one of the first pixel region or the second pixel region; and displaying a third image based on the updated predicted class label.

[0219] Embodiment A18: The method of embodiment A17, further comprising inputting a third image into the machine learning model to retrain the machine learning model.

[0220] Embodiment A19: The method of embodiment A1, further comprising determining another one of the consecutive pairs of anchor points, wherein determining the other one of the consecutive pairs of anchor points comprises identifying another starting anchor point, wherein the other starting anchor point is an ending anchor point of a first boundary point of the identified pair of boundary points, and identifying another ending anchor point when the curvature between the other starting anchor point and the other ending anchor point meets a predetermined criterion.

[0221] Embodiment A20: The method described in embodiment A19, wherein determining the other one of the consecutive pairs of anchor points further includes ceasing to identify the other end anchor point if the curvature between the other start anchor point and the other end anchor point does not meet a predetermined criterion.

[0222] Embodiment A21: The method of embodiment A1, wherein the image includes images of one or more lesions, and wherein consecutive pairs of anchor points are configured to be utilized to adjust the identified boundaries to accurately label the one or more lesions.

[0223] Embodiment A22: The method described in embodiment A1, wherein adjusting the identified boundary utilizing consecutive pairs of anchor points makes the medical image suitable for use in clinical trials of one or more pharmaceutical agents.

[0224] Embodiment A23: The method of embodiment A1, wherein the image comprises a magnetic resonance imaging (MRI) image, a computed tomography (CT) image, an X-ray image, an ultrasound image, a positron emission tomography (PET) image, a single photon emission computed tomography (SPECT) image, or an optical coherence tomography (OCT) image.

[0225] Embodiment A24: A system including one or more computing devices, comprising: one or more non-transitory computer-readable storage media including instructions; and one or more processors coupled to the one or more storage media, wherein the one or more processors receive an image, the image including a medical image of a patient; identify a region of a plurality of pixels in the image, the region of the plurality of pixels including at least a first region of pixels and a second region of pixels; extract a set of boundary points along the identified boundary between the first region of pixels and the second region of pixels; and determine consecutive pairs of anchor points, the consecutive pairs of anchor points being located between the first region of pixels and the second region of pixels. and a second region of pixels, wherein the instructions for determining one of the consecutive pairs of anchor points further include instructions for identifying a starting anchor point, where for a first one of the identified pairs of boundary points, the starting anchor point is a first boundary point of the set of boundary points and for a subsequent one of the identified pairs of boundary points, the starting anchor point is an ending anchor point of a most recently identified pair of boundary points; and identifying an ending anchor point, where a curvature between the starting anchor point and the ending anchor point satisfies a predetermined criterion.

[0226] Embodiment A25: The system described in embodiment A24, wherein the instructions further include instructions for: causing a display of one or more other computing devices to display a second image based on the consecutive pairs of anchor points; and adjusting an identified boundary between the first region of pixels and the second region of pixels in response to determining one or more user inputs corresponding to manipulation of at least one of the consecutive pairs of anchor points.

[0227] Embodiment A26: The system described in embodiment A24, wherein adjusting the identified boundary between the first region of pixels and the second region of pixels comprises adjusting a contour of the identified boundary.

[0228] Embodiment A27: A system described in embodiment A24, wherein adjusting the identified boundary between the first pixel region and the second pixel region comprises correcting a class label corresponding to the first pixel region or the second pixel region.

[0229] Embodiment A28: A system described in embodiment A27, wherein correcting the class label corresponding to the first pixel region or the second pixel region is performed in response to receiving one or more user inputs from a human pathologist.

[0230] Embodiment A29: The system described in embodiment A24, wherein the curvature between the start anchor point and the end anchor point comprises a perpendicular distance between the start anchor point and the end anchor point.

[0231] Embodiment A30: The system described in embodiment A24, wherein the curvature between the start anchor point and the end anchor point meets a predetermined criterion when the curvature exceeds a perpendicular distance threshold.

[0232] Embodiment A31: The system described in embodiment A30, wherein the vertical distance threshold comprises a vertical distance of approximately 1 pixel.

[0233] Embodiment A32: The system described in embodiment A30, wherein the vertical distance threshold comprises a vertical distance of approximately 2 pixels.

[0234] Embodiment A33: The system described in embodiment A30, wherein the vertical distance threshold comprises a user-configurable threshold.

[0235] Embodiment A34: The system described in embodiment A33, wherein the user-settable threshold is configured to be adjusted to adjust the identified boundary between the first region of pixels and the second region of pixels according to a desired accuracy.

[0236] Embodiment A35: The system of embodiment A34, wherein the desired accuracy varies based on the total number of consecutive pairs of anchor points.

[0237] Embodiment A36: The system of embodiment A24, wherein for the last one of the identified pairs of boundary points, the ending anchor point is the last boundary point of the set of boundary points.

[0238] Embodiment A37: The system described in embodiment A24, wherein the instructions for determining one of the consecutive pairs of anchor points further include instructions for ceasing to identify the end anchor point if the curvature between the start anchor point and the end anchor point does not meet a predetermined criterion.

[0239] Embodiment A38: The system described in embodiment A24, wherein the instructions for identifying regions of pixels in the image include instructions for segmenting regions of pixels indicative of normal regions and regions of pixels indicative of disease regions.

[0240] Embodiment A39: The system described in embodiment A24, wherein the instructions for identifying regions of multiple pixels in an image further include instructions for inputting the image to a machine learning model trained to segment regions of multiple pixels in an image, and using the machine learning model to segment at least a first region of pixels and a second region of pixels, and outputting a predicted class label for each of the first region of pixels and the second region of pixels.

[0241] Embodiment A40: The system described in embodiment A39, wherein the instructions further include instructions for displaying a second image based on a predicted class label for each of the first region of pixels and the second region of pixels, receiving one or more user inputs corresponding to a request to update the predicted class label for at least one of the first region of pixels or the second region of pixels, and displaying a third image based on the updated predicted class label.

[0242] Embodiment A41: The system described in embodiment A40, wherein the instructions further include instructions for inputting a third image into the machine learning model to retrain the machine learning model.

[0243] Embodiment A42: The system described in embodiment A24, further including instructions for determining another one of the consecutive pairs of anchor points, wherein the instructions for determining the other one of the consecutive pairs of anchor points further include instructions for identifying another starting anchor point, where the other starting anchor point is an ending anchor point of a first boundary point of the identified pair of boundary points, and identifying another ending anchor point when the curvature between the other starting anchor point and the other ending anchor point meets a predetermined criterion.

[0244] Embodiment A43: The system described in embodiment A42, wherein the instructions for determining the other one of the consecutive pairs of anchor points further include instructions for ceasing to identify the other end anchor point if the curvature between the other start anchor point and the other end anchor point does not meet a predetermined criterion.

[0245] Embodiment A44: A system described in embodiment A24, wherein the image includes images of one or more lesions, and wherein consecutive pairs of anchor points are configured to be utilized to adjust the identified boundaries to accurately label the one or more lesions.

[0246] Embodiment A45: The system described in embodiment A24, wherein adjusting the identified boundary utilizing consecutive pairs of anchor points makes the medical image suitable for use in clinical trials of one or more pharmaceuticals.

[0247] Embodiment A46: The system of embodiment A24, wherein the image comprises a magnetic resonance imaging (MRI) image, a computed tomography (CT) image, an X-ray image, an ultrasound image, a positron emission tomography (PET) image, a single photon emission computed tomography (SPECT) image, or an optical coherence tomography (OCT) image.

[0248] Embodiment A47: When executed by one or more processors of one or more computing devices, a method includes: receiving an image, the image including a medical image of a patient; identifying a region of a plurality of pixels in the image, the region of the plurality of pixels including at least a first region of pixels and a second region of pixels; extracting a set of boundary points along the identified boundary between the first region of pixels and the second region of pixels; and determining consecutive pairs of anchor points, the consecutive pairs of anchor points aligning the identified boundary between the first region of pixels and the second region of pixels. a first boundary point of the identified pair of boundary points, the first anchor point being a first boundary point of the set of boundary points, and for a subsequent boundary point of the identified pair of boundary points, the first anchor point being an ending anchor point of a boundary point of a most recently identified pair; and an ending anchor point, the curvature between the starting anchor point and the ending anchor point meeting a predetermined criterion.

[0249] Embodiment A48: A non-transitory computer-readable medium as described in embodiment A47, wherein the instructions further include instructions for: causing a display of one or more other computing devices to display a second image based on the consecutive pairs of anchor points; and adjusting an identified boundary between the first region of pixels and the second region of pixels in response to determining one or more user inputs corresponding to manipulation of at least one anchor point of the consecutive pairs of anchor points.

[0250] Embodiment A49: A non-transitory computer-readable medium as described in embodiment A47, wherein adjusting an identified boundary between a first region of pixels and a second region of pixels is adjusting a contour of the identified boundary.

[0251] Embodiment A50: A non-transitory computer-readable medium as described in embodiment A47, wherein adjusting an identified boundary between a first region of pixels and a second region of pixels comprises correcting a class label corresponding to the first region of pixels or the second region of pixels.

[0252] Embodiment A51: A non-transitory computer-readable medium as described in embodiment A50, wherein correcting the class label corresponding to the first region of pixels or the second region of pixels is performed in response to receiving one or more user inputs from a human pathologist.

[0253] Embodiment A52: The non-transitory computer-readable medium of embodiment A47, wherein the curvature between the start anchor point and the end anchor point comprises a perpendicular distance between the start anchor point and the end anchor point.

[0254] Embodiment A53: The non-transitory computer-readable medium of embodiment A47, wherein the curvature between the start anchor point and the end anchor point meets a predetermined criterion when the curvature exceeds a perpendicular distance threshold.

[0255] Embodiment A54: The non-transitory computer-readable medium of embodiment A53, wherein the vertical distance threshold comprises a vertical distance of approximately 1 pixel.

[0256] Embodiment A55: A non-transitory computer-readable medium as described in embodiment A53, wherein the vertical distance threshold comprises a vertical distance of approximately 2 pixels.

[0257] Embodiment A56: The non-transitory computer-readable medium of embodiment A53, wherein the vertical distance threshold comprises a user-configurable threshold.

[0258] Embodiment A57: A non-transitory computer-readable medium as described in embodiment A56, wherein the user-settable threshold is configured to be adjusted to adjust the identified boundary between the first region of pixels and the second region of pixels according to a desired accuracy.

[0259] Embodiment A58: The non-transitory computer-readable medium of embodiment A57, wherein the desired accuracy varies based on the total number of consecutive pairs of anchor points.

[0260] Embodiment A59: The non-transitory computer-readable medium of embodiment A47, wherein for the last one of the identified pairs of boundary points, the ending anchor point is the last boundary point of the set of boundary points.

[0261] Embodiment A60: A non-transitory computer-readable medium as described in embodiment A47, wherein the instructions for determining one of the consecutive pairs of anchor points further include instructions for ceasing to identify the end anchor point if the curvature between the start anchor point and the end anchor point does not meet a predetermined criterion.

[0262] Embodiment A61: A non-transitory computer-readable medium as described in embodiment A47, wherein the instructions for identifying regions of pixels in an image include instructions for segmenting regions of pixels indicative of normal regions and regions of pixels indicative of disease regions.

[0263] Embodiment A62: A non-transitory computer-readable medium as described in embodiment A47, wherein the instructions for identifying regions of multiple pixels in an image further include instructions for inputting the image to a machine learning model trained to segment regions of multiple pixels in an image, and utilizing the machine learning model to segment at least a first region of pixels and a second region of pixels, and outputting a predicted class label for each of the first region of pixels and the second region of pixels.

[0264] Embodiment A63: A non-transitory computer-readable medium as described in embodiment A62, further comprising instructions for: displaying a second image based on a predicted class label for each of the first region of pixels and the second region of pixels; receiving one or more user inputs corresponding to a request to update the predicted class label for at least one of the first region of pixels or the second region of pixels; and displaying a third image based on the updated predicted class label.

[0265] Embodiment A64: The non-transitory computer-readable medium of embodiment A63, wherein the instructions further include instructions for inputting a third image into the machine learning model to retrain the machine learning model.

[0266] Embodiment A65: The non-transitory computer-readable medium of embodiment A47, further comprising instructions for determining another one of the consecutive pairs of anchor points, wherein the instructions for determining the other one of the consecutive pairs of anchor points further comprise instructions for identifying another starting anchor point, wherein the other starting anchor point is an ending anchor point of a first boundary point of the identified pair of boundary points, and identifying another ending anchor point when the curvature between the other starting anchor point and the other ending anchor point meets a predetermined criterion.

[0267] Embodiment A66: A non-transitory computer-readable medium as described in embodiment A65, wherein the instructions for determining the other one of the consecutive pairs of anchor points further include instructions for ceasing to identify the other end anchor point if the curvature between the other start anchor point and the other end anchor point does not satisfy a predetermined criterion.

[0268] Embodiment A67: A non-transitory computer-readable medium as described in embodiment A47, wherein the image includes images of one or more lesions, and wherein consecutive pairs of anchor points are configured to be utilized to adjust identified boundaries to accurately label the one or more lesions.

[0269] Embodiment A68: A non-transitory computer-readable medium described in embodiment A47, wherein adjusting the identified boundary utilizing consecutive pairs of anchor points makes the medical image suitable for use in clinical trials of one or more pharmaceuticals.

[0270] Embodiment A69: The non-transitory computer-readable medium of embodiment A47, wherein the image comprises a magnetic resonance imaging (MRI) image, a computed tomography (CT) image, an X-ray image, an ultrasound image, a positron emission tomography (PET) image, a single photon emission computed tomography (SPECT) image, or an optical coherence tomography (OCT) image.

[0271] VII. Definitions and Contextual Examples The present disclosure is not limited to these exemplary embodiments and applications or the manner in which the exemplary embodiments and applications operate or are described herein. Further, the figures may show simplified or partial views, and the dimensions of the elements in the figures may be exaggerated or out of proportion.

[0272] When reference is made to a list of elements (e.g., elements a, b, c), such reference is intended to include any one of the listed elements alone, any combination of fewer than all of the listed elements, and / or all combinations of the listed elements. The section divisions herein are for ease of reference only and do not limit any combination of the elements described.

[0273] Unless otherwise defined, scientific and technical terms used in connection with the present teachings described herein shall have the meanings commonly understood by those of ordinary skill in the art. Furthermore, unless the context requires otherwise, singular terms shall include the plural and plural terms shall include the singular. Generally, the nomenclature utilized in connection with, and techniques of, chemistry, biochemistry, molecular biology, pharmacology, and toxicology described herein are those well known and commonly used in the art.

[0274] Furthermore, when the terms "on," "mounted," "connected," "coupled," or similar expressions are used herein, one element (e.g., component, material, layer, substrate, etc.) may be "on," "mounted," "connected," or "coupled" to another element, regardless of whether the element is directly on, attached to, connected to, or coupled to the other element, or whether one or more intervening elements are between the one element and the other. Furthermore, when a reference is made to a list of elements (e.g., elements a, b, c), such reference is intended to include any one of the listed elements by itself, any combination of fewer than all of the listed elements, and / or all combinations of the listed elements. Section divisions herein are for ease of reference only and do not limit any combination of elements described.

[0275] The term "subject" can refer to a subject of a clinical trial, a person or animal undergoing treatment, a person or animal receiving anti-cancer therapy, a person or animal being monitored for remission or recovery, a person or animal undergoing preventative health analysis (e.g., due to their medical history), or any other person or patient or animal of interest. In various instances, "subject" and "patient" may be used interchangeably herein.

[0276] The term "OCT image" may refer to an image of a tissue, organ, etc., such as the retina, scanned or acquired using optical coherence tomography (OCT) imaging technology. The term may refer to one or both of a 2D "slice" image and a 3D "volume" image. Unless explicitly stated, the term may be understood to include an OCT volume image.

[0277] Unless otherwise defined, scientific and technical terms used in connection with the present teachings described herein shall have the meanings commonly understood by those of ordinary skill in the art. Furthermore, unless the context requires otherwise, singular terms shall include the plural and plural terms shall include the singular. Generally, the nomenclature utilized in connection with, and techniques of, chemistry, biochemistry, molecular biology, pharmacology, and toxicology described herein are those well known and commonly used in the art.

[0278] As used herein, "substantially" means sufficient to function for its intended purpose. Thus, the term "substantially" allows for slight, insignificant variations from an absolute or perfect state, dimension, measurement, result, etc., as would be expected by one of ordinary skill in the art, but does not noticeably affect overall performance. With respect to a parameter or characteristic that is a number or can be expressed as a number, "substantially" means within 10%.

[0279] As used herein, the term "about" when used in reference to a numerical value or a parameter or characteristic that can be expressed as a numerical value means within 10% of the numerical value. For example, "about 50" means a value in the range of 45 to 55.

[0280] The term "ones" means two or more.

[0281] As used herein, the term "plurality" can be 2, 3, 4, 5, 6, 7, 8, 9, 10 or more.

[0282] As used herein, the term "set" means one or more. For example, a set of items includes one or more items. As used herein, the term "subset" includes one or more of the items included in a referenced set. For example, a subset may include one item of the referenced set, or may include all items of the referenced set.

[0283] As used herein, the phrase "at least one of," when used in conjunction with a list of items, means that different combinations of one or more of the listed items may be used, or that only one of the items in the list may be required. An item may be a specific object, thing, step, action, process, or category. In other words, "at least one of" means that any combination or number of items from the list may be used, but not all of the items in the list may be required. For example, without limitation, "at least one of item A, item B, or item C" means item A; item A and item B; item B; item A, item B, and item C; item B and item C; or items A and C. In some cases, "at least one of item A, item B, or item C" means, but is not limited to, two of item A, one of item B, and ten of item C; four of item B and seven of item C; or some other suitable combination.

[0284] As used herein, a "model" may include one or more algorithms, one or more mathematical techniques, one or more machine learning (ML) algorithms, or a combination thereof.

[0285] As used herein, "machine learning" can include the practice of using algorithms to analyze data, learn from it, and then make decisions or predictions about something in the world. Machine learning uses algorithms that can learn from data without relying on rule-based programming.

[0286] As used herein, "artificial neural network" or "neural network" may refer to a mathematical algorithm or computational model that mimics an interconnected group of artificial neurons that process information based on a connectionist approach to computation. A neural network, sometimes referred to as a neural net, may use one or more layers of nonlinear units to predict an output for a received input. Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as the input for the next layer in the network, i.e., the next hidden layer or the output layer. Each layer of the network generates an output from the received input according to the current value of each parameter set. In various embodiments, a reference to a "neural network" may refer to one or more neural networks.

[0287] Neural networks, for example, may process information in two ways: they are in training mode when they are being trained (e.g., using a training data set), and they are in inference (or prediction) mode when they are practicing what they have learned (e.g., using a test data set). Neural networks may learn through a feedback process (e.g., backpropagation) that allows the network to adjust the weight coefficients of individual nodes in intermediate hidden layers (modify the behavior of individual nodes) so that their outputs match those of the training data. In other words, neural networks may learn by being fed training data (training examples), and eventually learn how to arrive at the correct output, even when presented with a new range or set of inputs.

[0288] Neural networks may process information in two ways: when they are being trained, they are in training mode; and when they put what they have learned into practice, they are in inference (or prediction) mode. Neural networks learn through a feedback process (e.g., backpropagation) that allows the network to adjust the weight coefficients of individual nodes in intermediate hidden layers (change their behavior) so that their outputs match those of the training data. In other words, neural networks learn by being fed training data (training examples) and eventually learn how to arrive at the correct output, even when presented with a new range or set of inputs. The neural network may include, for example, but is not limited to, at least one of a feedforward neural network (FNN), a recurrent neural network (RNN), a modular neural network (MNN), a convolutional neural network (CNN), a residual neural network (ResNet), an ordinary differential equation neural network (neural-ODE), or another type of neural network.

[0289] As used herein, "deep learning" may refer to the use of multi-layer artificial neural networks to automatically learn representations from input data such as images, videos, and text without human-provided knowledge to make highly accurate predictions in tasks such as object detection / identification, speech recognition, and language translation.

[0290] VIII. Further Considerations Headings and subheadings between sections and subsections herein are included merely to improve readability and do not imply that features may not be combined across sections and subsections, and therefore, the sections and subsections do not describe separate embodiments.

[0291] Some embodiments of the present disclosure include a system including one or more data processors. In some embodiments, the system includes a non-transitory computer-readable storage medium including instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein. Some embodiments of the present disclosure include a computer program product tangibly embodied in a non-transitory machine-readable storage medium, the computer program product including instructions configured to cause one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein.

[0292] The terms and expressions which have been employed are used as terms of description rather than of limitation, and there is no intention in the use of such terms and expressions to exclude equivalents of the features shown and described, or portions thereof, but it is recognized that various modifications are possible within the scope of the invention as claimed. Thus, although the claimed invention has been specifically disclosed by embodiments and optional features, it will be understood that modifications and variations of the concepts disclosed herein may be employed by those skilled in the art, and that such modifications and variations are deemed to be within the scope of the invention as defined by the appended claims.

[0293] While the present teachings will be described in conjunction with various embodiments, it is not intended that the present teachings be limited to such embodiments. On the contrary, the present teachings encompass various alternatives, modifications, and equivalents, as will be appreciated by those skilled in the art.

[0294] In describing various embodiments, the specification may present a method and / or process as a particular sequence of steps. However, to the extent that the method or process does not rely on the particular order of steps set forth herein, the method or process should not be limited to the particular order of steps set forth, and as one skilled in the art will readily appreciate, the order may be changed and still remain within the spirit and scope of the various embodiments.

[0295] Furthermore, the subject matter described herein may be embodied in systems, devices, methods, and / or articles, depending on the desired configuration. The implementations set forth in the description herein do not necessarily represent all implementations of the described subject matter. Rather, they are merely some examples consistent with aspects related to the described subject matter. While several variations have been detailed above, other modifications and additions are possible. In particular, additional features and / or variations may be provided in addition to those described herein. For example, the implementations described above may be directed to various combinations and subcombinations of the disclosed features and / or combinations and subcombinations of several additional features disclosed above. Additionally, the logic flow depicted in the accompanying figures and / or described herein does not necessarily require the particular order shown or sequential order to achieve desirable results. Other implementations may be within the scope of the following claims.

Claims

1. receiving an image relating to a portion of a subject's anatomy; extracting a plurality of boundary points for an initial boundary associated with a corresponding region of pixels of the image, the plurality of boundary points being associated with a sequential order for a selected two-dimensional plane corresponding to the image; evaluating the plurality of boundary points in the sequential order to select a plurality of anchor points from the plurality of boundary points, wherein evaluating the plurality of boundary points includes: determining a first boundary point of the plurality of boundary points to be a first anchor point; determining that a current boundary point of the plurality of boundary points being evaluated is a next anchor point of the plurality of anchor points when at least one perpendicular distance to a line extending between the previous anchor point and the current boundary point, calculated for a portion of the initial boundary located between the previous anchor point and the current boundary point, is greater than a selected threshold; evaluating the plurality of boundary points, generating an anchor point image for display in a graphical user interface of a display device, the anchor point image including a plurality of anchor indicators representing the plurality of anchor points; A method comprising:

2. The evaluating 2. The method of claim 1, further comprising determining that the current boundary point being evaluated is not a next anchor point of the plurality of anchor points when a calculated perpendicular distance to the portion of the initial boundary located between the previous anchor point and the current boundary point relative to the line extending between the previous anchor point and the current boundary point is not greater than the selected threshold.

3. The evaluating 3. The method of claim 1, further comprising determining that the current boundary point being evaluated is not a next anchor point in the plurality of anchor points when there is no boundary point in the sequential order between the previous anchor point and the current boundary point being evaluated.

4. The method of claim 1 , further comprising receiving a user input in the graphical user interface to move at least one anchor indicator of the plurality of anchor indicators.

5. 5. The method of claim 4, further comprising adjusting an original position of at least one of the plurality of anchor points corresponding to the at least one anchor indicator based on the user input so that the plurality of anchor points has a final position.

6. 6. The method of claim 4 or 5, further comprising generating adjustment data based on the user input, the adjustment data including an adjusted segmented image, at least some pixels of the segmented image being reclassified to form the adjusted segmented image based on the final positions of the plurality of anchor points.

7. The method of claim 6 , further comprising retraining a segmentation model, including a machine learning model, using the adjusted segmented image.

8. The evaluating The method of claim 1 , further comprising determining that a last boundary point of the plurality of boundary points is a last anchor point.

9. Determining that the current boundary point of the plurality of boundary points being evaluated is the next anchor point of the plurality of anchor points includes: calculating a perpendicular distance between each intermediate boundary point of the plurality of boundary points located between the previous anchor point and the current boundary point and a line extending between the previous anchor point and the current boundary point to form a set of perpendicular distances; determining that at least one vertical distance of the set of vertical distances is greater than the selected threshold; determining that the current boundary point will be the next anchor point in response to determining that the at least one perpendicular distance of the set of perpendicular distances is greater than the selected threshold; 9. The method of claim 1, comprising:

10. The method of any one of claims 1 to 9, wherein the selected threshold is selected to be between 1 and 10 pixel units.

11. 11. The method of claim 1, wherein the anchor point image displayed on the graphical user interface further includes a new boundary formed by line segments connecting the anchor indicators representing the anchor points.

12. receiving a user input at the graphical user interface to move at least one anchor indicator of the plurality of anchor indicators; adjusting the initial boundary to form a new boundary based on the user input; generating an adjusted image with the new boundary overlaid on the image; using the adjusted image to form an input for an algorithm; 12. The method of claim 1, further comprising:

13. 13. The method of any one of claims 1 to 12, wherein the image comprises a magnetic resonance imaging (MRI) image, a computed tomography (CT) image, an X-ray image, an ultrasound image, a positron emission tomography (PET) image, a single photon emission computed tomography (SPECT) image, or an optical coherence tomography (OCT) image.

14. 14. The method of claim 1, wherein the image is a two-dimensional image or a three-dimensional image.

15. receiving a plurality of images relating to a portion of an anatomy of a subject; performing segmentation using the plurality of images and the segmentation model to generate a plurality of segmented images; extracting, for each segmented image of the plurality of segmented images, a plurality of boundary points for each initial boundary of a set of initial boundaries associated with each segmented image, the plurality of boundary points being associated with a sequential order for a selected two-dimensional plane corresponding to the plurality of images; evaluating, for each segmented image among the plurality of segmented images, the plurality of boundary points for each initial boundary of the set of initial boundaries associated with each segmented image in accordance with the sequential order to select, for each initial boundary of the set of initial boundaries identified in each image, a plurality of anchor points from the plurality of boundary points; The evaluating step includes: determining a first boundary point of the plurality of boundary points to be a first anchor point; determining that a last boundary point of the plurality of boundary points is a last anchor point; determining that the current boundary point of the plurality of boundary points being evaluated is a next anchor point of the plurality of anchor points when at least one perpendicular distance of a set of perpendicular distances to a line extending between a previous anchor point and a current boundary point, calculated for a corresponding set of intermediate boundary points, is greater than a selected threshold; evaluating the plurality of boundary points in the sequential order, including: generating an anchor point image for display in a graphical user interface of a display device, the anchor point image including a plurality of anchor indicators representing a plurality of line segments connecting the plurality of anchor points and the plurality of anchor indicators; A method comprising:

16. the segmentation model comprises a machine learning model; receiving a user input to adjust a position of at least one of the plurality of anchor indicators; adjusting an original position of at least one anchor point of the plurality of anchor points corresponding to the at least one of the plurality of anchor indicators to form a final position of the plurality of anchor points; modifying corresponding segmented images of the plurality of segmented images using the final positions of the plurality of anchor points to form adjusted segmented images for use in retraining the segmentation model; 16. The method of claim 15, further comprising:

17. 1. A system including one or more computing devices, one or more non-transitory computer-readable storage media containing instructions; one or more processors coupled to the one or more storage media; wherein the one or more processors: receiving an image relating to a portion of a subject's anatomy; extracting a plurality of boundary points for an initial boundary associated with a corresponding region of pixels of the image, the plurality of boundary points being associated with a sequential order for a selected two-dimensional plane corresponding to the image; evaluating the plurality of boundary points in the sequential order to select a plurality of anchor points from the plurality of boundary points, wherein the evaluating includes: determining a first boundary point of the plurality of boundary points to be a first anchor point; determining that a current boundary point of the plurality of boundary points being evaluated is a next anchor point of the plurality of anchor points when at least one perpendicular distance to a line extending between the previous anchor point and the current boundary point, calculated for a portion of the initial boundary located between the previous anchor point and the current boundary point, is greater than a selected threshold; evaluating the plurality of boundary points, generating an anchor point image for display in a graphical user interface of a display device, the anchor point image including a plurality of anchor indicators representing the plurality of anchor points; 20. A system configured to execute instructions on a

18. 20. The system of claim 17, wherein the one or more processors are configured to execute the instructions to determine that the current boundary point being evaluated is not a next anchor point of the plurality of anchor points when there is no boundary point along the initial boundary between the previous anchor point and the current boundary point being evaluated.

19. the one or more processors:

19. The system of claim 17 or 18, further configured to execute the instructions for receiving a user input in the graphical user interface to move at least one anchor indicator of the plurality of anchor indicators.

20. the one or more processors:

20. The system of claim 19, further configured to execute the instructions for adjusting an original position of at least one anchor point of the plurality of anchor points corresponding to the at least one anchor indicator based on the user input to form a final position for the plurality of anchor points.

21. the one or more processors:

21. The system of claim 19 or 20, further configured to execute the instructions for generating adjustment data based on the user input, the adjustment data including an adjusted segmented image, at least some pixels of the segmented image being reclassified to form the adjusted segmented image based on the final positions of the plurality of anchor points.

22. the one or more processors:

22. The system of claim 21, further configured to execute the instructions for retraining a segmentation model, including a machine learning model, using the adjusted segmented image.

23. the one or more processors:

23. The system of any one of claims 17 to 22, further configured to execute the instructions for determining that a last boundary point of the plurality of boundary points is a last anchor point.

24. the one or more processors: calculating a perpendicular distance between each intermediate boundary point of the plurality of boundary points located between the previous anchor point and the current boundary point and a line extending between the previous anchor point and the current boundary point to form a set of perpendicular distances; determining that at least one vertical distance of the set of vertical distances is greater than the selected threshold; determining that the current boundary point will be the next anchor point in response to determining that the at least one perpendicular distance of the set of perpendicular distances is greater than the selected threshold; 24. The system of claim 17, further configured to execute the instructions on:

25. A system according to any one of claims 17 to 24, wherein the selected threshold is selected to be between 1 and 10 pixel units.

26. 26. The system of claim 17, wherein the anchor point image displayed on the graphical user interface further includes a new boundary formed by line segments connecting the anchor indicators representing the anchor points.

27. 27. The system of any one of claims 17 to 26, wherein the image comprises a magnetic resonance imaging (MRI) image, a computed tomography (CT) image, an X-ray image, an ultrasound image, a positron emission tomography (PET) image, a single photon emission computed tomography (SPECT) image, or an optical coherence tomography (OCT) image.

28. 28. The system of any one of claims 17 to 27, wherein the image is a two-dimensional image or a three-dimensional image.

Images