Generating regions of interest in volumetric structures within biomedical images using orthogonal planes

Abstract

Presented herein are systems and methods of generating regions of interest (ROIs) in biomedical images of volume structures within a subject. One or more processors may identify a plurality of biomedical images of a volume stucture within a subject. The one or more processors may receive a plurality of tracings. The one or more processors may determine a plurality of curves to identify the ROI within the volume structure in accordance with a curve morphing function based at least on the plurality of tracings. The one or more processors may generate the ROI using the plurality of curves determined within the volume structure. The one or more processors may store, using one or more data structures, an association between the subject and the ROI.

Description

GENERATING REGIONS OF INTEREST IN VOLUMETRICSTRUCTURES WITHIN BIOMEDICAL IMAGESUSING ORTHOGONAL PLANESCROSS REFERENCE TO RELATED APPLICATIONS

[0001] The present application claims the benefit of and priority to U.S. Provisional Patent Application No. 63 / 544,046 and filed on October 13, 2023, which is incorporate by reference in its entirety.BACKGROUND

[0002] A computing device may use computer vision techniques to process a digital image to generate an output about the input digital image.SUMMARY

[0003] Three-dimensional (3D) imaging modalities, such as magnetic resonance imaging (MRI) computed tomography (CT) scans, and ultrasounds, may provide a non-invasive method to visually capture anatomical structures within a human body. The anatomical structures may include; for example, organs, vessels, bones, tumors, and other objects within the body. From these 3D images, these particular structures can be detected, as distinguished from the background (e.g., tissue, fat, or organs outside the target structure) within the body. This identification of such structures can be used in a wide variety of applications, such as diagnosis, prognosis, treatment selection, and labeling datasets for training Al models to perform automated image segmentation tasks among others.

[0004] One approach to detect these structures may be to rely on fully-automated techniques, leveraging machine learning (ML) models to segment the 3D structure from image scans. A 3D image scan may be fed into an ML model, and then the ML model may produce a segmented image identifying the target structure from the scan. ML models, however, may behighly inaccurate resulting in misidentifications of target structures within the image and may be very imprecise leading to a wider volume being identified as the target structures. These ML models may be computationally complex and may rely on an enormous training dataset with correct annotations to be adequately trained, entailing large consumption of processing and memory resources. Another approach may be to use manual techniques, with a human (e.g., a clinician) examining each individual image to manually mark the structure within the 3D images using a graphical user interface (GUI). Although the manual technique may yield more accurate and precise results, this technique may be tedious and time-consuming and may rely on specialized expertise and knowledge, thereby limiting its accessibility. Not to mention, this approach may be impractical when dealing with 3D images with multiple planes and complex structures.

[9005] To address these and other technical challenges, a semi -automated technique may be used to identify a voluminous structure within a 3D biomedical image as a region of interest (ROI), a labeling process that can be used for development of automated ML models. A computing system can receive tracings that are manually input by a user examining the 3D biomedical image. The tracing plane can be a projection of additional slices within the biomedical image volume to capture additional features within the projected plane. Each tracing may define coordinates (e.g., pixel coordinates) corresponding to a boundary of the structure within the 3D biomedical image along one of the planes (e.g., anatomical planes). For example, the structure may be an elongated volume within the body. One tracing may be defined along one edge of the volume in a coronal plane of the 3D biomedical image, and another tracing may be defined along another edge of the volume in a sagittal plane. Tracings can be within two orthogonal planes rotated by any axes within the 3D biomedical image.

[0006] Using a curve morphing function and the tracings, the computing system may generate curves along another plane. The curve morphing function may be used to generate a parametric curve within the plane based on the points of the input tracings. In the example above, the computing system may generate a curve along a transverse plane of the 3D biomedical image using points formed by the tracings defined in the coronal and sagittal planes.In addition, the computing system may form a center line within a respective structure defined by the curves. By repeating this process across the planes of the 3D biomedical image, the computing system may generate a ROI as defined by the curves to correspond to the structure within the body. The 3D biomedical image along with the ROI may be used in a wide variety of applications, such as use as training data for a ML model used to localize, segment, or classify biomedical images.

[0007] In this manner, the curve morphing function used by the computing system can achieve high accuracy and precision in determining ROIs corresponding to target volumes within 3D biomedical images relative to fully-automated techniques. The curve morphing function can be spline based or leverage the equation of an ellipse. In addition, the use of the curve morphing function can reduce computing resources such as processor and memory as well as overhead from avoiding the use of computationally complex ML models. Relative to manual approaches, the curve morphing function can reduce the amount of effort and time in detecting the ROIs within images.

[0008] Aspects of the present disclosure are directed to systems and methods of generating regions of interest (ROIs) in biomedical images of a volume structure within a subject. One or more processors may identify a plurality of biomedical images of a volume structure within a subject. The plurality of biomedical images may include: (i) a first biomedical image along a first plane in the volume structure and (ii) a second biomedical image along a second plane oblique to the first plane in the volume structure. The one or more processors may receive a plurality of tracings. The plurality of tracings may include: (i) a first tracing defining a first boundary in the first biomedical image in which a ROI resides along the first plane and (ii) a second tracing defining a second boundary in the second biomedical image in which the ROI resides along the second plane. The one or more processors may determine a plurality of curves to identify the ROI within the volume structure in accordance with a curve morphing function based at least on the plurality of tracings. The one or more processors may generate the ROI using the plurality of curves determined within the volume structure. The one or moreprocessors may store, using one or more data structures, an association between the subject and the ROI.

[0009] In some embodiments, the one or more processors may generate a segmentation mask defining the ROI within the volume structure of the subject using the plurality of curves. In some embodiments, the one or more processors may acquire the plurality of biomedical images corresponding to a plurality of angiograms in the volume structure of the subject using computed tomography (CT). In some embodiments, the one or more processors may determine the plurality of curves by determining a plurality of points. The plurality of points may include: (i) a first point along the first plane to define the ROI within the volume structure and (ii) a second point along the second plane to the ROI within the volume structure, using the curve morphing function comprising at least one of (i) a shape interpolation and (ii) a Bezier curve.

[0010] In some embodiments, the one or more processors may generate a third tracing through a central axis of the ROI within the volume structure using the plurality of tracings. In some embodiments, the one or more processors may receive a selection of a slice along at least one of the first plane or the second plane in which the ROI resides within the volume structure within the subject. In some embodiments, the one or more processors may identify a plurality of slices proximal to the selected slice. In some embodiments, the one or more processors may generate at least one of the first biomedical image in the first plane or of the second biomedical image in the second plane using the plurality of slices. In some embodiments, the one or more processors may identify a collection of frames acquired over a time period. In some embodiments, the volume structure is a tubular structure within the subject. The tubular structure can correspond to at least one of a spine, an aneurysm, a blood vessel, or an esophagus. In some embodiments, the one or more processors may generate a training dataset including the plurality of biomedical images and the ROI within the plurality of biomedical images. In some embodiments, the one or more processors may provide the training dataset to train a machine learning model for performing at least one of localization, segmentation, or classification of biomedical images.BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The foregoing and other objects, aspects, features, and advantages of the present invention will become more apparent and better understood by referring to the following description taken in conjunction with the accompanying drawings, in which:(0012] FIG. 1 : Example anterior-posterior and lateral views of a patient AAA.(0013] FIG. 2: Example anterior-posterior and lateral view of a patient AAA, manual tracings (blue), and superior / inferior bounds (red).

[0014] FIG. 3: Examples of A) anterior-posterior view of a patient AAA and B) lateral (side) view of a patient AAA. The red is a ‘left’ tracing, and blue is the ‘right’ tracing. The yellow is a centerline that is automatically calculated.

[0015] FIG. 4A and 4B: An example is shown where four elliptical curves are generated for each quadrant based on the major and minor axis.

[0016] FIG. 5 A: Example 3D surface reconstruction of a patient AAA using the algorithmic approach.

[0017] FIG. 5B: Example anterior posterior view with centerline of the 3D point cloud.

[0018] FIG. 5C: Example lateral view with centerline of the 3D point cloud.

[0019] FIG. 6: Example anterior-posterior and lateral view of an abdominal aortic aneurysm. The left and right curves are manually traced. The center curve provides the centerline that can be converted into a 3D centerline using spatial positioning within a cube.

[0020] FIG. 7 is a block diagram of a system for generating regions of interest (ROI), in accordance with an illustrative embodiment.[00211 FIG. 8 is a block diagram of a process of identifying a volume within biomedical images and receiving tracings for a boundary corresponding to a region of interest (ROI), in accordance with an illustrative embodiment.

[0022] FIG. 9 is a block diagram of a process of determining curves to identify the ROI within the volume, in accordance with an illustrative embodiment.

[0023] FIG. 10 is a block diagram of a process of training a machine learning model (ML), in accordance with an illustrative embodiment.

[0024] FIG. 11 is a flow diagram of a method of generating regions of interest, in accordance with an illustrative embodiment.

[0025] FIG. 12 is a block diagram of a computing environment according to an example implementation of the present disclosure.DETAILED DESCRIPTION

[0026] Following below are more detailed descriptions of various concepts related to, and embodiments of, systems and methods for generating regions of interest (ROIs) in biomedical images. It should be appreciated that various concepts introduced above and discussed in greater detail below may be implemented in any of numerous ways, as the disclosed concepts are not limited to any particular manner of implementation. Examples of specific implementations and applications are provided primarily for illustrative purposes.

[0027] Section A describes a methodology for generating regions of interest.

[0028] Section B describes systems and method for generating regions of interest in volumetric structures within biomedical images using tracings of curves.(0029] Section C describes a network and computing environment.A. Methodology for Generating Regions of Interest[0030| Abdominal aortic aneurysm (AAA) is a ballooning of a major artery, if left untreated can rupture. There are over 2.5 million patients in the US with abdominal aortic aneurysms (AAA) and 200,000 new diagnoses each year. AAA rupture is associated with a 90% mortality rate, and the surgical risk increases from 1% using minimally invasive surgery to 40% for emergency surgery which can be costly (including emergency services via life flight). For over 60 years, clinicians have relied on societal guidelines to make AAA patient management decisions, but many patients experience rupture or unnecessary intervention. Following current guidelines, clinicians rely on the maximum diameter of the aneurysm (5.0 cm for women, 5.5 cm for men) to inform their decision to perform surgical intervention or continue monitoring the growth of the aneurysm. However, aneurysms that are smaller than the diameter criterion for surgical intervention have been reported to rupture at rates between 7-23.4%. Additionally, surveillance is costly for patients and insurers (-$40,000 USD over a 4-year period) and can cause unnecessary patient anxiety.

[0031] Segmentation of tubular structures from medical images can be time-consuming. The technology that is presented here allows for an extremely fast semi -automated method to identify an ROI from two 2D images that are orthogonal to one another. The orthogonal images can be taken directly from a 3D volume of tubular vessels (medical images). Tubular vessels can include aortas, carotid arteries, veins, aneurysms, and other structures within the body. The planes can be taken from any location as long as they are orthogonal. For the example that is presented, a snapshot of an abdominal aortic aneurysm (AAA) in the anterior-posterior view and lateral views were taken. A left curve was traced for the aneurysm (red), and the right curve was traced (blue) for both AP and lateral views. The centerline of the 2D image is shown in yellow. The 3D reconstruction can occur using two methods: 1) ellipsoidal reconstruction or 2) curve morphing algorithms and placing them radially about the craniocaudal axis. The ellipsoidal reconstruction can recreate a cuboidal volume using the two orthogonal images. The centerline can provide the centroid of the ellipse, and a single point left / right AP and left / right lateral provide the major and minor axes. An ellipse can be generated for each height level (z-axis).The 3D surface reconstruction need not be a point cloud but could become a 3D masking volume for further refinements during segmentation.

[0032] The software suite can include performing medical image analysis using biomechanical and shape analyses. However, image-based analyses are time consuming and can for example take up to 4 hours for manual segmentation. A quality management system software-based tool may be used to assess and audit the 3D geometries that are extracted from medical images. To alleviate the time-constraints, Al-based methods can automatically identify the regions of interests from medical images. A quality management system may be in place to test for appropriate outputs of the software. Therefore, regions of interest may be identified by manually tracing the boundary of AAA from medical images. This technology may involve an end-user to trace the aneurysms from two views that are perpendicular to one another. Using the spatial relationships of the tracings a region of interest can be identified faster than the traditional manual segmentation method.

[0033] Using various algorithms and programming approaches, a 3D surface can be generated that represents a patient’s AAA. This procedure is currently developed using certain specific software but can be converted into standalone software that is deployable and useful for clinicians. In addition, rigorous validation of the 3D surface against manual tracings can be performed to ensure that the proposed system is a suitable check for proper segmentations that are performed by the Al algorithms. Another potential use-case for this method can be to firstly identify the ROI prior to Al-based image segmentation to improve reliability. The manual tracing method may take roughly 30 seconds to complete, compared to the traditional method of 4 hours. This exponential time reduction can collect more data faster to perform high-throughput analysis of AAA.

[0034] This is an example of a method to extract a 3D region of interest from two orthogonal planes (or views) from abdominal aortic aneurysms (AAA). However, this approach can be adapted to other tubular structures in the human body that require manual segmentation for computational approaches to perform biomechanical and morphological analyses. FIG. 1represents two views that are orthogonal to one another. The chosen views are the anterior- posterior and lateral-view of a medical imaging volume from CT angiogram. However, any oblique planes that are orthogonal can be extracted from the 3D volume about any axis (this example uses the craniocaudal axis).|0035| The user inputs (which may be required) can be four tracings (but not limited to) of the boundary of the region of interest. This can typically be performed using axial slices from the 3D stack, and can take up to 4 hours for high-quality imaging. The inferior and superior boundaries are enacted in FIG. 2.

[0036] The tracings can have a ‘left’ and ‘right’ side relative to their respective centerline in each view (FIG. 3). The centerline of the tubular vessel may be created by taking the average of the left and right curves and their coordinates (e.g., averaging their points in space).

[0037] Five points for instance can be placed in 3D space, a point for the left anterior- posterior view, a point for the right anterior-posterior view, a point for the left later view, a point for the right lateral view, and a point for the centerline (this is the unifying ‘centroid’ of the four traced points. FIGs. 4A and 4B represent an example of a single ‘slice’ along the craniocaudal axis or longitudinal axis of the AAA. Intermediate points are generated from the ‘control’ points using ellipse equations. Intermediate points are generated from the relationship of the angle about the centroid with the major and minor axis: x = alcos2(0);y = blsin(0): r = ^ / x2+2r = a2cos20 + b2sin20The intermediate points can also be generated using curve morphing algorithms. Some examples of curve morphing (but not limited to) are shape interpolation and Bezier curves. The curve morphing would generate intermediate curves between the left anterior-posterior curve and the right lateral curve, right lateral curve and right anterior-posterior curve, left lateral curve and leftanterior-posterior, and left lateral curve and right anterior-posterior curve. The placement of the curves can utilize the relationship between the angle, intermediate radii, major and minor axes of the ellipse:

[0038] This method does / may not segment using axial slices (e.g., slice by slice segmentation). The reconstruction can take place using 4 hand-drawn curves, but additional hand-drawn curves can be made. The reconstruction can also be a voxel volume to identify the region of interest.

[0039] Lastly, the spatial relationship of the tracings and centerline can be mapped by creating a virtual 3D cube of the tracings and centerline (FIGs. 5A-C). Curve placement or using the equations of ellipses can create the intermediate points that traverse the centerline of the tubular vessel. This method can be applied to create the 3D volume; however, this method can also be used to create intermediate masks to segment an ROI from the original medical imaging dataset which can be later segmented using deep learning or standard image-processing techniques.B. Systems and Method for Generating Regions of Interest in Volumetric Structures Within Biomedical Images Using Tracings of Curves]0040| Referring now to FIG. 7, depicted is a block diagram of a system 100 for generating regions of interest (ROI) within a volume using orthogonal planes. In a brief overview, the system 100 can include at least one image processing system 105, at least one imaging device 110, at least one computing device 115, and at least one database 125, among others, communicatively coupled via at least one network 120. The image processing system 105 can include at least one image retriever 130, at least one tracing receiver 135, at least one slice handler 140, at least one curve handler 145, at least one volume generator 150, and at least one dataset generator 155, and at least one model trainer 160, among others. Each of thecomponents of the system 100 can be implemented using the computing system as described in Section C.

[0041] In further detail, the image processing system 105 can be any computing device comprising one or more processors coupled with memory and software capable of performing the various processes and tasks described herein. The image processing system 105 can be housed within a computing system (e.g., laptop, PC, smart device) or within a server group (e.g., a data center, a branch office, or a server site), and include instructions to manage the identifying of biomedical images, receiving of the tracings, and generation of the ROIs. The image processing system 105 can be in communication with the imaging device 110, computing device 115, the database 125, among others.100421 The image retriever 130 within the image processing system 105 can retrieve, receive, or otherwise obtain images captured by the imaging device 110. The tracing receiver 135 can receive, retrieve, or otherwise obtain a plurality of tracings from the computing device 115. The slice handler 140 can receive, obtain, or otherwise retrieve a selection of a slice from the computing device 115. The slice handler 140 can identify, generate, or otherwise calculate a plurality of slices based on the selected slice from the computing device 115. The curve handler 145 can determine, generate, or otherwise identify a plurality of curves based on the plurality of tracings and the selected slice. The volume generator 150 can generate, determine, or otherwise identify a ROI within the volume. The dataset generator 155 can generate, create, or otherwise establish a dataset to train the ML model. The model trainer 160 can train, establish or otherwise update the ML model using the dataset.[00431 The imaging device 110 can be any device capable of acquiring images of structure within subjects. The image can be a tomogram image acquired in accordance with a tomographic imaging technique, such as a magnetic resonance imaging (MRI) scanner, a nuclear magnetic resonance (NMR) scanner, high-energy electromagnetic radiation (X-ray) computed tomography (CT) scanner, an ultrasound imaging scanner, and a positron emission tomography (PET) scanner, and a photoacoustic spectroscopy scanner, among others. Although primarilydiscussed herein in terms of a CT scan, other imaging modalities besides those listed above may be supported by the image processing system 105. The imaging device 110 can be in communication with the image processing system 105 and the computing device 115 to provide acquired images. In some embodiments, the imaging device 110 may capture a set or collection of images represented as a video. Each image may correspond to an image frame within the video.

[0044] The computing device 115 can be any device comprising one or more processors coupled with memory and software and capable of providing an output projection image. The computing device 115 can be associated with an entity (e.g., clinician, physician, doctor) examining the subject or biomedical images from the subject. The computing device 115 can be in communication with the image processing system 105 and the imaging device 110 to exchange data. The computing device 115 can display images acquired from the imaging device 110 on a display. The display can be used to create tracings of the volume and select slices for the volume within the images.

[0045] Referring now to FIG. 8, depicted is a block diagram of a process 200 of identifying a volume within biomedical images and receiving tracings for a boundary corresponding to an ROI. The process 200 can include or correspond to operations performed in the system 100. Under the process 200, the imaging device 110 can capture one or more biomedical images 215A-N (sometimes referred to as biomedical images 215) of a volume structure 210 within a subject 205. The subject 205 can be a human or animal subject, among others. In some instances, the subject can be at risk of or afflicted by at least one condition or symptoms of a condition that involve aneurysms. The conditions can include atherosclerosis, hypertension, mycotic aneurysm, genetic disorders (e.g., Marfan syndrome, polycystic kidney disease), abdominal aortic aneurysm, thoracic aortic aneurysm, among other conditions.

[0046] The subject 205 can include at least one volume structure 210. The volume structure 210 can be any tubular structure within the subject 205, such as blood vessels (e.g., arteries, veins, capillaries), a trachea, a bronchi (e.g., bronchioles), an esophagus, an aneurysm, astomach, a spine, among other tubular structures within the subject 205. The biomedical image 215 can be an image, a scan, a visual, a graphic, among other forms of images based on the volume structure 210 of the subject 205. For instance, the imaging device 110 can capture an image of the coronary artery within the neck of the subject 205. In another example, the imaging device 110 can capture a CT scan of the small intestine of the subject 205.100471 In some embodiments, the imaging device 110 can capture a collection of frames over a period of time. The collection of frames can be a plurality of biomedical images captured within the period of time, to define a sequence, a frame-by frame, slideshow, video, a clip, an image series, among other collections of frames. For example, the imaging device 110 can capture 100-1000 frames of biomedical images 215 over a period of time to define the esophagus with the occurrence of peristalsis. Despite capturing the collection of frames, computing systems struggle to properly render the volume structures 210 within the subject 205 due to the large quantity of scans and labor intensive marking of data points to isolate the volume structure 210. Using aspects of the technical solution described herein, the system 100 can quickly and efficiently render the volume structure 210 while saving computer utilization and reducing memory.

[0048] Each biomedical image 215 of the plurality of biomedical images 215 can be captured on a plurality of planes 212A-N (generally referred to as planes 212). The planes 212 can correspond to, for example, a 2D plane (e.g., Cartesian plane), or a 3D plane (e.g., XY-plane, YZ-plane XZ-plane). In cylindrical or spherical coordinates, the plane 212can refer to a rotation about a central axis within the volume structure 210. The planes 212 can correspond to different views of the biomedical images 215, such as an anterior view, a posterior view, a lateral view, a superior view, an inferior view, an oblique view, a ventral view, a plantar view among other views along the plane 212A-C, as shown in FIG. 6. The planes 212 may be oblique with one another (e.g., orthogonal or substantially orthogonal, within 90%). For example, a first biomedical image 215A of a spine (e.g., volume structure 210) can be along a first plane 212A (e.g., anterior view) and a second biomedical image 215B of the spine can be along a second plane 212B (e.g., lateral view) oblique to the first plane. There may be a third plane 212C that isoblique to the first plane 212A and the second plane 212B. In some embodiments, the planes 212 may correspond to anatomical planes of the body of the subject 205, such as a sagittal (or lateral) plane, a coronal (or frontal) plane, or a transverse (or horizontal) plane, among others.

[0049] Each biomedical image 215 can have at least one ROI 217 (sometimes herein referred as a volume of interest (VOI), a structure of interest (SOI), or feature of interest (FOI)) corresponding to the volume structure 210. The ROI 217 can correspond to an area, portion, or location within the volume structure 210. For example, if the volume structure 210 is the spine, the ROI 217 can be the cervical spine, lumbar spine, sacral spine, or the coccygeal spine. In another example, if the volume structure 210 is the stomach, the ROI 217 can be the cardia (e.g., upper portion of the stomach), fundus, body, antrum, or the pylorus. In yet another example, if the volume structure 210 is the large intestine, the ROI 217 can be the cecum, ascending colon, sigmoid colon, or the rectum. In some instances, the ROI 217 corresponds to a portion of the volume structure that includes the aneurysm.

[0050] The image retriever 130 can identify, retrieve, or otherwise receive the plurality of biomedical images 215 of the volume structure 210 within the subject 205 from the imaging device 110, in response to the imaging device 110 transmitting the plurality of biomedical images 215. For example, the image retriever 130 can retrieve a plurality of images of an aorta, from the imaging device 110 to identify an aneurysm on the aorta as the ROI 217. In another example, the image retriever 130 can acquire the plurality of biomedical images 215 corresponding to a plurality of angiograms in the volume structure 210 of the subject 205 using computed tomography (CT).[00511 Concurrently, the image retriever 130 can store the plurality of biomedical images 215 or the collection of frames within the database 125. In this manner, the image retriever 130 can process the plurality of biomedical images 215 prior to transmitting the plurality of biomedical images 215 to the computing device 115. While processing the plurality of images 215, the image retriever 130 can remove noise and distortions to clean the images, upon reception from the imaging device 110. The image retriever 130 can store the plurality ofbiomedical images 215 according to a profile within the database of the subject 205. The profile can maintain information associated with the subject 205, such as subject name (e.g., anonymized identifier), date of acquisition, and medical condition, , among other information of the subject 205. Once the biomedical images 215 are stored and processed, the image retriever 130 can transmit the biomedical images 215 to the computing device 115.|0052| The image retriever 130 can send, transmit, or otherwise provide the plurality of biomedical images 215 to the computing device 115. Prior to transmission, the image retriever 130 can generate one or more instructions to cause the plurality of biomedical images 215 to display on a user interface of the computing device. The instructions can cause the computing device to render each biomedical image 215 in the respective planes 212. For example, the instructions can cause the computing device 115 to render a first image 215 of the aorta in an anterior view and a second image of the aorta in a lateral view. In some instances, the image retriever 130 can include the instructions within the image 215, therefore the image retriever 130 can save on bandwidth and storage by embedding the instructions within the images 215.

[0053] The computing device 115 can identify, retrieve, or otherwise receive the plurality of biomedical images 215 from the image retriever 130 or the imaging device 110. In some instances, the imaging device 110 can be electrically coupled to the computing device 115, therefore, the imaging device 110 can directly transmit biomedical images 215 to the computing device. However, without the use of the image retriever 130, the biomedical images 215 may include noise, excess pixels, or distortions. Upon receipt, the computing device 115 can render, display, or otherwise present at least one of the plurality of biomedical images 215 for display on the user interface. The one or more instructions embedded within the biomedical images 215 can include graphical user interface elements for use by a physician, lab technician, or an administrator. The graphical user interface elements can include buttons, text boxes, sliders, canvas, toolbar, grid, selection tools, among other graphical user interface elements. For example, the instructions can cause the biomedical images 215 to display on a canvas to allow for the generation of a plurality of tracings. In another example, the instructions can cause time biomedical images 215 to be displayed according to a slider to select slices of each image 215.To display the graphical user interface elements, the instructions can direct the computing device 115 to a drawing software that includes each of the graphical user interface elements.

[0054] The computing device 115 can create, determine, or otherwise generate a plurality of tracings 225A-N (hereinafter generally referred to as the tracings 225). To generate the tracings 225, the one or more instructions from the image retriever 130 can cause or trigger the computing device 115 to direct to the drawing application that includes the graphical user interface elements. The drawing application executing on the computing device 115 can create a transparent layer, placed on top of the images 215. The graphical user interface elements of the transparent layer can receive interactions from the computing device 115. The interactions can include a drawing, a mapping, a tracing, a highlight, a button click, a slide drag, among other interactions with the graphical user. The computing device 115 can generate the plurality of tracings 225 according to interactions with the graphical user interface elements. For example, the computing device 115 can generate the plurality of tracings 225 in response to a physician drawing on a canvas (e.g., graphical user interface element) corresponding to the images 215. In another example, computing device 115 can generate the tracings 225 according to interactions received at the transparent layer of the images 215.

[0055] The plurality of tracings 225 can include a first tracing 225A that defines a first boundary in the first biomedical image 215A along the first plane 212A. The first boundary can be an edge or outer limit of the volume structure 210 that defines the shape and the form in the respective plane 212 (e.g., first plane 212A). Thereby, the first tracing 225A can define the first boundary of the volume structure 210 within the image 215 as shown in FIG. 6. The first tracing 225A can include a plurality of colors according to the outer limits of the volume structure. For example, the first tracing 225A can include a red color for a first outer limit and blue color for a second outer limit opposite or distal the first outer limit. Each tracing 225 can include a plurality of pixels to define the plurality of colors. In some instances, the image retriever 130 can detect the outer limits of the volume structure 210. Once detected, the image retriever 130 can label the outer limits of the volume structure to define the respective colors for the tracings 225. The label can be transmitted along with the instructions, thereby causing the computing device 115 toautomatically assign the respective color to the outer limit of the volume structure 210 as the physician interacts with the transparent layer of the image 215.

[0056] The plurality of tracings 225 can include a second tracing 225B that defines a second boundary in the second biomedical image 215B along the second plane 212B. The second boundary can be an edge or outer limit of the volume structure 210 that defines the shape and the form in the respective plane 212 (e.g., second plane 212B). The second tracing 225B can define the second boundary of the volume structure 210 within the image 215 as shown in FIG.6. The first tracing 225A can include a plurality of colors according to the outer limits of the volume structure. For example, the second tracing 225B can include a red color for a first outer limit and blue color for a second outer limit opposite or distal to the first outer limit. The first plane 212A for the first tracing 225 A can be orthogonal or substantially orthogonal (e g., within 90%) of the second plane 212B for the second tracing 225B. The first tracing 225A and the second tracing 225B can differ due to the respective planes 212 of each tracing 225. For example, the first tracing 225A can occur in a lateral view (e.g., first plane 212A) of the coronary artery of a subject 205, whereas a second tracing 225B can occur in an anterior view (e g., second plane 212B) of the coronary artery of the subject. Upon successfully generating the tracings 225 for the volume structure 210, the computing device 115 can transmit the tracings to the tracing receiver 135.

[0057] The tracing receiver 135 can receive, retrieve, or otherwise identify the tracings 225 from the computing device 115. Upon reception of the tracings 225, the tracing receiver 135 can extract, analyze, and otherwise identify each pixel of the tracing 225 and the images 215. In operation, the tracing receiver 135 can use a pixel grid to identify the tracings 225 and the biomedical images 215 based on the pixels. The tracing receiver 135 can extract the transparent layer on top of the biomedical images 215 and apply the pixel grid on the transparent layer of the images 215. From here, the tracing receiver 135 can use the pixel grid to identify each tracing 225 in the plurality of tracings 225 based on the pixels. The tracing receiver 135 can smooth the pixels of the tracings 225 to provide a clearer tracing of the boundaries of the volume structure210. The tracing receiver 135 can analyze each transparent layer of each image 215 in the plurality of biomedical images 215 to generate a third tracing 225.

[0058] In some embodiments, the tracing receiver 135 can determine, identify, or otherwise generate a central tracing 225C through a centroid axis of the ROI 217 corresponding to the volume structure 210 using the plurality of tracings 225, as shown in FIGs. 5A-5C. Upon reception of the first tracing 225 A and the second tracing 225B, the tracing receiver 135 can automatically generate the central tracing 225C. The central tracing 225C can be generated on a layer of each image 215. Since each image 215 can be on a different plane 212, the central tracing 225C of the first image 215 can differ from the third image 215, as shown in FIG. 6. To generate the central tracing 225C, the tracing receiver 135 can determine median coordinates based on the coordinates of the first tracings 225 and the second tracing 225B. As the tracing receiver 135 establishes the mean between each pixel of the first tracing 225 A and the second tracing 225B, the tracing receiver 135 can use each median to generate the central tracing 225C. The tracing receiver 135 can perform this process iteratively to generate the central tracing 225C.

[0059] In conjunction, the computing device 115 can identify, obtain, or otherwise receive a selection of at least one slice 220 along at least one of the first plane 212A or the second plane 212B. The slice can correspond to a layer or a cross section of the volume structure 210 in which the ROI 217 resides. In some instances, the slice can correspond to an angle of the volume structure. In some instances, the slice 220 may not contain the ROI 217 corresponding to the volume structure 210. Prior to the computing device 115 receiving the selection, the image retriever 130 can present the plurality of slices 220 to the computing device 115. The plurality of slices 220 can be captured by the imaging device 110 or automatically determined by the imaging device 110 based on the biomedical images 215 of the volume structure 210. In this manner, the imaging device 110 can embed the plurality of slices 220 within the instructions. As the instructions are transmitted to the computing device 115, the instruction can further cause the computing device 115 to display the plurality of slices 220 on the user interface. From here, a physician can select at least one slice 220 of the volume structure 210 displayed on the graphical user interface of the computing device 115. Once thecomputing device 115 receives a selection of a slice, the computing device 115 can transmit the selected slice 220 to the slice handler 140.

[0060] The slice handler 140 can identify, retrieve, or otherwise receive the selection of the slice 220 along at least one of the first plane 212A or the second plane 212B from the computing device 115. The slice handler 140 can identify the selection of the slice 220 in response to the computing device 115 receiving the selection of the slice 220 by using an action listener configured to detect selections of the slice 220 (e.g., along the first plane 212A and the second plane 212B). The selected slice 220 may correspond to a section among the biomedical images 215 that is indicated by the user of the computing device 115 as containing the ROI 217 corresponding to the volume structure 210. In some instances, the slice handler 140 can identify the slice 220 along a lateral view (e.g., first plane 212A) of the volume structure 210. In some instances, the slice handler 140 can receive the selection of the slice 220 within an anterior view (e.g., second plane 212B) of the volume structure 210. In some instances, the slice handler 140 can identify a plurality of slices, each in at least one of the first plane 212A or the second plane 212B. For example, the slice handler 140 can receive a plurality of slices in the posterior view (e g., first plane 212A) of the volume structure 210.[00611 With receipt, the slice handler 140 can identify, determine, or otherwise generate a plurality of slices proximal to the selected slice 220. The slice handler 140 can execute a protocol to identify the plurality of slices 220 proximal to the selected slice 220. Using the protocol, the slice handler 140 can iterate through a plurality of slices 220 and calculate an average for the slices 220 based on proximity, distortions, whether the ROI is within the slice 220, among other factors. The slice hander 140 can determine the distance or proximity to the selected slice 220. For instance, the slice handler 140 can determine that each slice 220 in the plurality of slices 220 are within a 1-50 nm proximity to the selected slice 220. To make the determination, the image retriever 130 can transmit an accuracy for the volume structure 210 within the image 215 to the slice hander 140. The slice handler 140 can compare the accuracy of the volume structure 210 to a threshold within the database 125. If the accuracy is greater than the threshold, the slice handler 140 can identify a plurality of slices 220 which are more distal tothe selected slice 220 according to a level of deviation from the threshold. However, if the accuracy is lower than the threshold, the slice handler 140 can identify the plurality of slices 220 which are more proximal to the selected slice according to the level of deviation from the threshold.

[0062] In some embodiments, the slice handler 140 can generate, identify, or otherwise create the first image 215 in the first plane 212A using the plurality of slices 220. In addition, the slice handler 140 can generate, identify, or otherwise create the second image 215 in the second plane 212B using the plurality of slices 220. To generate the first image 215 in the first plane 212A, the image retriever 130 can transmit the first image 215 to the computing device 115. When transmitting the image 215, the image may not be in at least one plane 212 in the plurality of planes 212. Upon reception of the image, the computing device 115 can receive a selection of the slice 220 from the graphical user interface elements. From here, the slice handler 140 can generate the plurality of slices 220 proximal to the selected slice 220. Once generated, the slice handler 140 can generate the images in the at least one plane 212 by using the plurality of slices 220 to construct the biomedical images 215 in the respective planes 212.

[0063] In this manner, the slice handler 140 can continuously generate more accurate biomedical images 215 in the respective planes 212 for analysis by a physician. Once the biomedical images 215 are generated in the first plane 212A or the second plane 212B, the slice handler 140 can transmit the biomedical images 215 to the computing device 115. Using the technical solution described herein, the image processing system 105 can generate images in a plurality of planes with high accuracy without the need to use excess computing resources and storage to capture multiple images without any regard to which slice. Rather, the image processing system 105 can generate slices of the images using the protocol to reduce the use of computing resources, memory, and manual intervention by a user (e.g., physician, doctor, lab technician).

[0064] Referring now to FIG. 9, depicted is a block diagram of a process 300 of determining curves to identify the ROI within the volume. Under the process 300, the curvehandler 145 can receive the plurality of tracings 225 from the tracing receiver 135. The plurality of tracings 225 can be on the transparent layers of the biomedical images 215 on the respective planes 212. In some instances, the curve handler 145 can extract or obtain the plurality of tracings 225 from the transparent layer of the images 215. To extract the plurality of tracings 225 from the tracing receiver 135, the curve handler 145 can parse the transparent layer of the image 215 to identify the one or more pixels of the respective tracings 225. Once the pixels are identified, the curve handler 145 can generate the plurality of tracings 225 from the tracing receiver 135. In this manner, the curve handler 145 can receive the plurality of tracings 225 from the tracing receiver 135 or extract the plurality of tracings 225 from the transparent layer of the image 215.

[0065] The curve handler 145 can determine, calculate, or otherwise generate a plurality of curves 305. The plurality of curves 305A-N (herein referred to as curves 305) can combine to form a circular shape (e.g., circle, ellipse, or disk, among others) by using the plurality of tracings 225. To generate the plurality of curves 305, the curve handler 145 can use the first tracing 225 A in the first image 215 and the second tracing 225B in the second image 215 in accordance with at least one curve morphing function 306 (sometimes herein referred to as a spline function) based on the plurality of tracings 225. The curve morphing function 306 can include, for example, a Bezier curve, a B-spline, a non-uniform rational B-spline, a Hermite curve spline, a Catmull-Roll spline, a Cardinal spline, or a Kochanek-B artels spline function, among others.

[0066] The curves 305 can be defined along a third plane 212C that is different from the first plane 212A or the second plane 212B. The third plane 212C defining the curves 305 can be orthogonal or substantially orthogonal (e.g., within 90%) of both the first plane 212A and the second plane 212B. For example, the first plane 212A may be the front plane, the second plane 212B may be the sagittal plane, and the third plane 212C may be the transverse plane. The curve morphing function 306 can recreate, alter or otherwise transform a first curve 305 into a second curve 305, a second curve 305 into a third curve 305, and so on. In operation, the curve morphing function 306 can automatically adjust, the shapes, positions or characteristics of eachcurve 305 to create the circular shape. The curve handler 145 can feed at least two curves 305 to the curve morphing function 306 to generate each subsequent curve 305 in the plurality of curves 305. In some instances, the curve morphing function 306 can receive an initial curve 305 based on the tracings 225 and a target curve 305 to achieve despite a difference in shape, position, or size of each curve 305.100671 In using the curve morphing function, the curve handler 145 can determine, identify, or otherwise generate a plurality of points 307A-N (hereinafter generally referred to as the points 307) for each curve 305 along the third plane 212C. The points 307 can interconnect the boundaries defined by the tracings 225 and form the curves 305. Using shape interpolation, the curve morphing function 306 can tune, adjust, or otherwise modify the plurality of points 307 while interconnecting the plurality of curves 305 along the third plane 212C. For example, to connect the first curve 305 and the second curve 305, the curve handler 145 can generate the plurality of points 307 to connect the first curve 305 and the second curve 305. The curve handler 145 can use shape interpolation to adjust each point 307 in the plurality of points 307 to interconnect the respective curves 305 over a predetermined number of steps or frames. In further detail, shape interpolation can involve an interpolation parameter identifying the initial curve 305 and the target curve 305 and the number of steps or frames to achieve the target curve 305.

[0068] Continuing on, the curve handler 145 can determine, identify, or otherwise generate a plurality of points 307A-N for each curve 305 using the curve morphing function 306. The curve handler 145 can generate at least two points 307 (e.g., control points) and by applying the curve morphing function 306, the control morphing function 306 can identify a plurality of control points 307. Using the control points 307, the curve morphing function can direct the direct and shape of the curves 305. The plurality of points 307A-N can include a first point 307 along the first plane and a second point 307 along the second plane to define the ROI 217’ within the volume structure 210 as shown in FIG. 4B. The first point 307 in the first plane 212A can correspond to the outer limits (e.g., boundaries) of the first tracing 225A, whereas the secondpoint 307 in the second plane 212B can correspond to the outer limits (e.g., boundaries) of the second tracing 225B.

[0069] In some embodiments, in generating the curves 305, the curve handler 145 can generate a coordinate grid, a graph, or a plane to establish the curves 305. Using the first boundary of the first tracing 225A, the curve handler 145 can graph each outer limit equidistant from the third tracing 225 along the central axis. From here, the curve handler 145 can graph the outer limits defined by the second boundary of the second tracing 225B perpendicular to a line segment generated between the outer limits of the first tracing 225A, through the central axis. For example, the curve handler 145 can graph a first curve 305 corresponding to a first outer limit (e.g., first boundary of the first tracing 225 A in the first plane 212A) of the aorta a radius away from the central axis of the third tracing 225 in a first direction (e g., 30° from the central axis, 45° from the central axis, 90° from the central axis, etc.). Subsequently, the curve handler 145 can graph a second curve 305 corresponding to the second outer limit of the aorta, equidistant from the first outer limit, in a second direction opposite the first direction.Continuing on, the curve handler 145 can graph a third curve corresponding to a first outer limit (e g., second boundary of the second tracing 225B in the second plane 212B) of the aorta a radius away from the central axis, perpendicular to a line segment created by the first curve 305 and the second curve 305. Similarly, the curve handler 145 can graph the fourth curve 305, however the fourth curve 305 can be in an opposite direction from the third curve 305.|0070| The volume generator 150 can generate, determine, or otherwise identify the ROI 217’ using the plurality of curves 305 corresponding to the volume structure 210. The ROI 217’ can correspond to a specific area within the volume structure 210 that is for analysis based on the clinical purpose. The clinical purpose can be, for example, localizing, segmenting, or classifying abnormalities, lesions, aneurysms, pathology, among others, from the images 215. In some instances, the ROI 217’ can be an organ, a mass, a tumor, an injured region, a region part of study, among others. To generate the ROI 217’, the volume generator 150 use the tracings 225 to generate a plurality of levels for the volume structure 210. Once the levels are established, the volume generator 150 can iteratively use the curves 305 to generate the ROI 217’ by followingthe third tracing 335 through the central axis. Once completed, the volume structure 210 can create the ROI 217’.

[0071] In some embodiments, the volume generator 150 can generate, determine, or otherwise identify a segmentation mask (e.g., semantic segmentation mask, instance segmentation mask, panoptic segmentation mask, among others) defining the ROI 217’ within the volume structure 210 of the subject 205 using the plurality of curves 305. The volume generator 150 can assign a label to each pixel in the tracings 225 to generate the segmentation mask. The segmentation mask can be a separate image to distinguish the ROI 217’ or the volume structure 219 by using different colors, hues, intensities, among other characteristics. Once the segmentation mask is identified, the volume generator 150 can apply the segmentation mask to the volume structure 210 to define the ROI 217’.

[0072] Referring now to FIG. 10, depicted is a block diagram of a process 400 of training machine learning models. Under the process 400, the dataset generator 155 can store, maintain, or otherwise house an association between the subject 205 and the ROI 217’. The association can be a link, a map, or a connection between the subject 205 and the ROI 217’ . The one or more data structures can include an array, a linked list, a stack, a tree, a hash table, among others. For example, the data structure can be a hash table where the subject 205 is the key to the hash table and the ROI 217’ is the value of the hash table. The hash table can include a plurality of keys (i.e., for each subject 205) mapped to a plurality of values. In another example, the data structure can be a plurality of linked lists. A first linked list can correspond to a first subject 205. Each node in the first linked list corresponding to each ROI 217’ of the first subject 205. Concurrently, a second linked list can correspond to a second subject 205. Each node in the second linked list corresponding to each ROI 217’ of the second subject 205. In some implementations, the dataset generator 155 can store an association between the image 215 and the ROI 217’

[0073] The dataset generator 155 can generate, create, or otherwise determine a training dataset 405. The training dataset 405 can include a set of examples based on sample images215 A-N’ (herein referred to as images 215’). The dataset generator 155 can extract, retrieve, or otherwise receive the sample images 215’ from the database 125. The sample images 215’ can be based on images received by the imaging device 110. Within the images 215’, the dataset generator 155 can embed a sample ROI 217’. The sample ROI 217’ can generated by the volume generator 150 and extracted from the database 125 based on the association between the subject 205 and the ROI 217’. The ROI 217’ can represent a desired outcome or prediction that the machine learning model 415 is trained to classify, identify, or predict to implement supervised learning. In some implementations, to execute unsupervised learning, the training dataset 405 may not include the sample image 215’ to allow the machine learning model 415 to learn patterns or structures of the ROI 217’ within the image 215. The training dataset can use the sample image 215’ and the label 410 as input features for the machine learning model 415. The label 410 may include any information to facilitate supervised learning. For example, the label 410 may include information about the subject 205 or the images 215, such as the medical condition, the coordinate definitions of the ROI 217’, among others. In some instances, the sample image 215’ can be the image 215.

[0074] The model trainer 160 can initialize, train, or otherwise establish one or more machine learning models 415 using the training dataset 405In some embodiments, the model trainer 160 can train the machine learning model 415 for image localization. Image localization can include an input image, feature extraction, object identification, bounding box prediction, and the output. In this manner, the machine learning model 415 can identify, determine, or otherwise capture a position of the ROI 217’ within the volume structure 210 to output a section of the image corresponding to the ROI 217’. During training, the model trainer 160 can apply a section (e.g., a bounding box) to the ROI 217’ within the volume structure 210 to train the machine learning model 415 to receive indications of ROIs 310 within the volume structure. The machine learning model 415 may be trained to identify a section corresponding to an ROI within the image. In some embodiments, the model trainer 160 can train the machine learning model 415 for image segmentation (e.g., semantic segmentation, instance segmentation, panoptic segmentation, among others). The model trainer 160 can train the machine learning model 415to determine which pixels of the image correspond to ROIs within the image. For instance, the model trainer 160 can train the machine learning model 415 to identify coordinates of an aneurysm within an image scan of an aorta in a subject.

[0075] In some embodiments, the model trainer 160 can train the machine learning model 415 for image classification. The model trainer 160 can train the machine learning model 415 using the training dataset 405 and predefined classes for the volume structures 210 within the subject. The predefined classes can include, for example, types of organs or structures within the body of a subject 205. For example, the model trainer 160 can train the machine learning model 415 to classify a given image as containing an aneurysm.

[0076] In this manner, the image processing system 105 can generate the ROIs within a volume structure, while avoiding the shortcomings of fully automated techniques and manual techniques. Relative to automated techniques, the image processing system 105 can generate ROIs with higher accuracy and precision, with the use of the input tracings. This can significantly reduce the amount of time and effort to generate the ROI’s. The image processing system 105 can reduce memory, increase performance in real-time (or near real time) by using the selection of a slice, the tracings of the volume structure, and the identified plurality of curves. Furthermore, since the image processing system 105 can achieve higher performance in near real-time, the subject can receive treatment subsequently to the capture of the biomedical images. Therefore, the image processing system 105 may permit a clinician examining the subject to make more accurate assessment about the subject and any conditions (e.g., aneurysms) on ROI within the volume structure (e.g., tubular structure) in the subject. The images along with the definition of ROIs can be used to generate high-quality training data for training machine learning models to perform various tasks, such as localization, segmentation, and classification.

[0077] Referring now to FIG. 11, depicted is a flow diagram of a method 500 of generating a region of interest (ROI) within a biomedical image. The method can be implemented by any components detailed herein, such as system 100 or system 600. Under themethod 500, a computing system can identify a plurality of biomedical images (505). The computing system can receive a plurality of tracings (510). The computing system can determine a plurality of curves (515). The computing system can generate the ROI (520). The computing system can provide the ROI to a machine learning model for training (525).C. Network and Computing Environment

[0078] Various operations described herein can be implemented on computer systems. FIG. 12 shows a simplified block diagram of a representative server system 600, computing system 614, and network 626 usable to implement certain embodiments of the present disclosure. In various embodiments, server system 600 or similar systems can implement services or servers described herein or portions thereof. Computing system 614 or similar systems can implement clients described herein. The system 100 described herein can be similar to the server system 600. Server system 600 can have a modular design that incorporates a number of modules 602 (e g., blades in a blade server embodiment); while two modules 602 are shown, any number can be provided. Each module 602 can include processing unit(s) 604 and local storage 606.

[0079] Processing unit(s) 604 can include a single processor, which can have one or more cores, or multiple processors. In some embodiments, processing unit(s) 604 can include a general-purpose primary processor as well as one or more special-purpose co-processors such as graphics processors, digital signal processors, or the like. In some embodiments, some, or all processing units 604 can be implemented using customized circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions that are stored on the circuit itself. In other embodiments, processing unit(s) 604 can execute instructions stored in local storage 606. Any type of processors in any combination can be included in processing unit(s) 604.

[0080] Local storage 606 can include volatile storage media (e.g., DRAM, SRAM, SDRAM, or the like) and / or non-volatile storage media (e.g., magnetic, or optical disk, flash memory, or the like). Storage media incorporated in local storage 606 can be fixed, removable, or upgradeable as desired. Local storage 606 can be physically or logically divided into varioussubunits such as a system memory, a read-only memory (ROM), and a permanent storage device. The system memory can be a read-and-write memory device or a volatile read-and-write memory, such as dynamic random-access memory. The system memory can store some or all of the instructions and data that processing unit(s) 604 need at runtime. The ROM can store static data and instructions that are needed by processing unit(s) 604. The permanent storage device can be a non-volatile read-and-write memory device that can store instructions and data even when module 602 is powered down. The term “storage medium” as used herein includes any medium in which data can be stored indefinitely (subject to overwriting, electrical disturbance, power loss, or the like) and does not include carrier waves and transitory electronic signals propagating wirelessly or over wired connections.[0081 | In some embodiments, local storage 606 can store one or more software programs to be executed by processing unit(s) 604, such as an operating system and / or programs implementing various server functions such as functions of the system 100 or any other system described herein, or any other server(s) associated with system 100 or any other system described herein.

[0082] Software” refers generally to sequences of instructions that, when executed by processing unit(s) 604, cause server system 600 (or portions thereof) to perform various operations, thus defining one or more specific machine embodiments that execute and perform the operations of the software programs. The instructions can be stored as firmware residing in read-only memory and / or program code stored in non-volatile storage media that can be read into volatile working memory for execution by processing unit(s) 604. Software can be implemented as a single program or a collection of separate programs or program modules that interact as desired. From local storage 606 (or non-local storage described below), processing unit(s) 604 can retrieve program instructions to execute and data to process in order to execute various operations described above.[0083| In some server systems 600, multiple modules 602 can be interconnected via a bus or other interconnect 608, forming a local area network that supports communicationbetween modules 602 and other components of server system 600. Interconnect 608 can be implemented using various technologies, including server racks, hubs, routers, etc.

[0084] A wide area network (WAN) interface 610 can provide data communication capability between the local area network (e.g., through the interconnect 608) and the network 626, such as the Internet. Other technologies can be used to communicatively couple the server system 600 with the network 626, including wired (e.g., Ethernet, IEEE 602.3 standards) and / or wireless technologies (e.g., Wi-Fi, IEEE 602.11 standards).[0085| In some embodiments, local storage 606 is intended to provide working memory for processing unit(s) 604, providing fast access to programs and / or data to be processed while reducing traffic on interconnect 608. Storage for larger quantities of data can be provided on the local area network by one or more mass storage 612 that can be connected to interconnect 608. Mass storage 612 can be based on magnetic, optical, semiconductor, or other data storage media. Direct attached storage, storage area networks, network-attached storage, and the like can be used. Any data stores or other collections of data described herein as being produced, consumed, or maintained by a service or server can be stored in mass storage 612. In some embodiments, additional data storage resources may be accessible via WAN interface 610 (potentially with increased latency).

[0086] Server system 600 can operate in response to requests received via WAN interface 610. For example, one of modules 602 can implement a supervisory function and assign discrete tasks to other modules 602 in response to received requests. Work allocation techniques can be used. As requests are processed, results can be returned to the requester via WAN interface 610. Such operation can generally be automated. Further, in some embodiments, WAN interface 610 can connect multiple server systems 600 to each other, providing scalable systems capable of managing high volumes of activity. Other techniques for managing server systems and server farms (collections of server systems that cooperate) can be used, including dynamic resource allocation and reallocation.[0087| Server system 600 can interact with various user-owned or user-operated devices via a wide-area network such as the Internet. An example of a user-operated device is shown in FIG. 6 as computing system 614. Computing system 614 can be implemented, for example, as a consumer device such as a smartphone, other mobile phone, tablet computer, wearable computing device (e.g., smart watch, eyeglasses), desktop computer, laptop computer, and so on.

[0088] For example, computing system 614 can communicate via WAN interface 610. Computing system 614 can include computer components such as processing unit(s) 616, storage device 618, network interface 620, user input622, and user output624. Computing system 614 can be a computing device implemented in a variety of form factors, such as a desktop computer, laptop computer, tablet computer, smartphone, other mobile computing device, wearable computing device, or the like.

[0089] Processing unit 616 and storage device 618 can be similar to processing unit(s) 604 and local storage 606 described above. Suitable devices can be selected based on the demands to be placed on computing system 614. For example, computing system 614 can be implemented as a “thin” client with limited processing capability or as a high-powered computing device. Computing system 614 can be provisioned with program code executable by processing unit(s) 616 to enable various interactions with server system 600.

[0090] Network interface 620 can provide a connection to the network 626, such as a wide area network (e.g., the Internet) to which WAN interface 610 of server system 600 is also connected. In various embodiments, network interface 620 can include a wired interface (e.g., Ethernet) and / or a wireless interface implementing various RF data communication standards such as Wi-Fi, Bluetooth, or cellular data network standards (e.g., 3G, 4G, LTE, etc.).

[0091] User input device 622 can include any device (or devices) via which a user can provide signals to computing system 614; computing system 614 can interpret the signals as indicative of particular user requests or information. In various embodiments, user input device 622 can include any or all of a keyboard, touch pad, touch screen, mouse, or other pointing device, scroll wheel, click wheel, dial, button, switch, keypad, microphone, and so on.[0092| User output device 624 can include any device via which computing system 614 can provide information to a user. For example, user output device 624 can include display-to- display images generated by or delivered to computing system 614. The display can incorporate various image generation technologies, e.g., a liquid crystal display (LCD), light-emitting diode (LED) display including organic light-emitting diodes (OLED), projection system, cathode ray tube (CRT), or the like, together with supporting electronics (e.g., digital -to-analog or analog-to- digital converters, signal processors, or the like). Some embodiments can include a device such as a touchscreen that function as both input and output device. In some embodiments, other user output devices 624 can be provided in addition to or instead of a display. Examples include indicator lights, speakers, tactile “display” devices, printers, and so on.[00931 Some embodiments include electronic components, such as microprocessors, storage, and memory that store computer program instructions in a computer readable storage medium. Many of the features described in this specification can be implemented as processes that are specified as a set of program instructions encoded on a computer readable storage medium. When one or more processing units execute these program instructions, they cause the processing unit(s) to perform various operations indicated in the program instructions. Examples of program instructions or computer code include machine code, such as is produced by a compiler, and files including higher-level code that are executed by a computer, an electronic component, or a microprocessor using an interpreter. Through suitable programming, processing unit(s) 604 and 616 can provide various functionality for server system 600 and computing system 614, including any of the functionality described herein as being performed by a server or client, or other functionality.

[0094] It will be appreciated that server system 600 and client computing system 614 are illustrative and that variations and modifications are possible. Computer systems used in connection with embodiments of the present disclosure can have other capabilities not specifically described here. Further, while server system 600 and client computing system 614 are described with reference to particular blocks, it is to be understood that these blocks are defined for convenience of description and are not intended to imply a particular physicalarrangement of component parts. For instance, different blocks can be but need not be located in the same facility, in the same server rack, or on the same motherboard. Further, the blocks need not correspond to physically distinct components. Blocks can be configured to perform various operations, e.g., by programming a processor or providing appropriate control circuitry, and various blocks might or might not be reconfigurable depending on how the initial configuration is obtained. Embodiments of the present disclosure can be realized in a variety of apparatus including electronic devices implemented using any combination of circuitry and software.

[0095] While the disclosure has been described with respect to specific embodiments, one skilled in the art will recognize that numerous modifications are possible. Embodiments of the disclosure can be realized using a variety of computer systems and communication technologies, including but not limited to specific examples described herein. Embodiments of the present disclosure can be realized using any combination of dedicated components and / or programmable processors and / or other programmable devices. The various processes described herein can be implemented on the same processor or different processors in any combination. Where components are described as being configured to perform certain operations, such configuration can be accomplished, e.g., by designing electronic circuits to perform the operation, by programming programmable electronic circuits (such as microprocessors) to perform the operation, or any combination thereof. Further, while the embodiments described above may make reference to specific hardware and software components, those skilled in the art will appreciate that different combinations of hardware and / or software components may also be used and that particular operations described as being implemented in hardware might also be implemented in software or vice versa.

[0096] Computer programs incorporating various features of the present disclosure may be encoded and stored on various computer readable storage media; suitable media include magnetic disk or tape, optical storage media such as compact disk (CD) or digital versatile disk (DVD), flash memory, and other non-transitory media. Computer readable media encoded with the program code may be packaged with a compatible electronic device, or the program codemay be provided separately from electronic devices (e.g., via Internet download or as a separately packaged computer-readable storage medium).

[0097] Thus, although the disclosure has been described with respect to specific embodiments, it will be appreciated that the disclosure is intended to cover all modifications and equivalents within the scope of the following claims.

Claims

WHAT IS CLAIMED IS:

1. A method of generating regions of interest (ROIs) in biomedical images, comprising: identifying, by one or more processors, a plurality of biomedical images of a volume structure within a subject, the plurality of biomedical images comprising: (i) a first biomedical image along a first plane in the volume structure and (ii) a second biomedical image along a second plane oblique to the first plane in the volume structure; receiving, by the one or more processors, a plurality of tracings comprising (i) a first tracing defining a first boundary in the first biomedical image in which a ROI resides along the first plane and (ii) a second tracing defining a second boundary in the second biomedical image in which the ROI resides along the second plane; determining, by the one or more processors, a plurality of curves corresponding to the volume structure in accordance with a curve morphing function based at least on the plurality of tracings; generating, by the one or more processors, the ROI using the plurality of curves corresponding to the volume structure; and storing, by the one or more processors, using one or more data structure s, an association between the subject and the ROI.

2. The method of claim 1, further comprising generating, by the one or more processors, a segmentation mask defining the ROI within the volume structure of the subject using the plurality of curves.

3. The method of claim 1, wherein identifying the plurality of biomedical images further comprises acquiring the plurality of biomedical images corresponding to a plurality of angiograms in the volume structure of the subject using computed tomography (CT).

4. The method of claim 1, wherein determining the plurality of curves further comprises determining a plurality of points comprising (i) a first point along the first plane to define theROI within the volume structure and (ii) a second point along the second plane to the ROI within the volume structure, using the curve morphing function comprising at least one of (i) a shape interpolation and (ii) a Bezier curve.

5. The method of claim 1, further comprising generating, by the one or more processors, a third tracing through a central axis of the ROI within the volume structure using the plurality of tracings.

6. The method of claim 1, wherein receiving the plurality of tracings further comprises: receiving, by the one or more processors, a selection of a slice along at least one of the first plane or the second plane in which the ROI resides of the volume structure within the subject; and identifying, by the one or more processors, a plurality of slices proximal to the selected slice.

7. The method of claim 6, further comprising generating, by the one or more processors, at least one of the first biomedical image in the first plane or the second biomedical image in the second plane using the plurality of slices.

8. The method of claim 1, wherein identifying the plurality of biomedical images, further comprises identifying, a collection of frames acquired over a time period.

9. The method of claim 1, wherein the volume structure is a tubular structure within the subject, wherein the tubular structure corresponds to at least one of a spine, an aneurysm, a blood vessel, or an esophagus.

10. The method of claim 1, further comprising: generating, by the one or more processors, a training dataset including the plurality of biomedical images and the ROI within the plurality of biomedical images; andproviding, by the one or more processors, the training dataset to train a machine learning model for performing at least one of localization, segmentation, or classification of the plurality of biomedical images.

11. A system for generating regions of interest (ROIs) in biomedical images, comprising: one or more processors coupled with memory to: identify a plurality of biomedical images of a volume structure within a subject, the plurality of biomedical images comprising: (i) a first biomedical image along a first plane in the volume structure and (ii) a second biomedical image along a second plane oblique to the first plane in the volume structure; receive a plurality of tracings comprising (i) a first tracing defining a first boundary in the first biomedical image in which a ROI resides along the first plane and (ii) a second tracing defining a second boundary in the second biomedical image in which the ROI resides along the second plane; determine a plurality of curves to identify the ROI within the volume structure in accordance with a curve morphing function based at least on the plurality of tracings; generate the ROI using the plurality of curves determined within the volume structure; and store, using one or more data structure s, an association between the subject and the ROI.

12. The system of claim 11, the one or more processors are further configured to generate a segmentation mask defining the ROI within the volume structure of the subject using the plurality of curves.

13. The system of claim 11, wherein the one or more processors are further configured to acquire the plurality of biomedical images corresponding to a plurality of angiograms in the volume structure of the subject using computed tomography (CT).

14. The system of claim 11, wherein the one or more processors are further configured to determine a plurality of points comprising (i) a first point along the first plane to define the ROI within the volume structure and (ii) a second point along the second plane to the ROI within the volume structure, using the curve morphing function comprising at least one of (i) a shape interpolation and (ii) a Bezier curve.

15. The system of claim 11, the one or more processors are further configured to generate a third tracing through a central axis of the ROI within the volume structure using the plurality of tracings.

16. The system of claim 11, wherein the one or more processors are further configured to: receive a selection of a slice along at least one of the first plane or the second plane in which the ROI resides of the volume structure within the subject; and identify a plurality of slices proximal to the selected slice.

17. The system of claim 16, the one or more processors are further configured to generate at least one of the first biomedical image in the first plane or the second biomedical image in the second plane using the plurality of slices.

18. The system of claim 11, wherein the one or more processors are further configured to identify a collection of frames acquired over a time period.

19. The system of claim 11, wherein the volume structure is a tubular structure within the subject, wherein the tubular structure corresponds to at least one of a spine, an aneurysm, a blood vessel, or an esophagus.

20. The system of claim 11, wherein the one or more processors are further configured to: generate a training dataset including the plurality of biomedical images and the ROI within the plurality of biomedical images; andprovide the training dataset to train a machine learning model for performing at least one of localization, segmentation, or classification of the plurality of biomedical images.

Images