Real-time planning of brachytherapy

Abstract

Presented herein are systems and methods for determining quality of brachytherapy treatment plans from catheter placement in organs in subjects. A computing system can identify a biomedical image of a section associated with an organ of a subject to be administered with brachytherapy in accordance with a treatment plan. The computing system can detect, from the biomedical image, (i) a plurality of positions corresponding to a plurality of catheters placed within the organ and (ii) a contour defining a structure of interest (SOI) in the biomedical image corresponding to a feature within the organ. The computing system can provide the plurality of positions and the contour as input to a machine learning (ML) model. The computing system can generate, based on providing the input to the ML model, a metric indicating a quality of the treatment plan for the plurality of catheters placed relative to the feature within the organ.

Description

REAL-TIME PLANNING OF BRACHYTHERAPYSTATEMENT OF GOVERNMENT SUPPORT[00011 This invention was made with government support under CA008748 awarded by the National Institutes of Health. The government has certain rights in the invention.CROSS REFERENCE TO RELATED APPLICATIONS

[0002] The present application claims priority to US Provisional Patent Application No. 63 / 665,634, titled “Real-Time Planning of High-Dose-Rate Brachytherapy,” filed June 28, 2024, which is incorporated herein by reference in its entirety.BACKGROUND

[0003] A computing device may use computer vision techniques to process an input image to generate an output.SUMMARY|0004| Aspects of the present disclosure are directed to systems, methods, devices, non- transitory computer readable media for determining quality of brachytherapy treatment plans from catheter placement in organs in subjects. One or more processors coupled with memory can identify a biomedical image of a section associated with an organ of a subject to be administered with brachytherapy in accordance with a treatment plan. The one or more processors can detect, from the biomedical image, (i) a plurality of positions corresponding to a plurality of catheters placed within the organ and (ii) a contour defining a structure of interest (SOI) in the biomedical image corresponding to a feature within the organ. The one or more processors can provide the plurality of positions and the contour as input to a machine learning (ML) model. The ML model may be established using a plurality of examples. Each example of the plurality of examples can include (i) a respective plurality of positions corresponding to a respective plurality of catheters placed within a respective organ of a respective subject, (ii) arespective contour defining a respective SOI in a respective biomedical image corresponding to a feature in the respective organ, and (iii) a respective metric indicating a quality of a respective treatment plan for the respective plurality of catheters placed relative to the feature within the organ. The one or more processors can generate, based on providing the plurality of positions and the contour to the ML model, a metric indicating a quality of the treatment plan for the plurality of catheters placed relative to the feature within the organ. The one or more processors can store, using one or more data structures, an association between the treatment plan and the metric.

[0005] In some embodiments, the one or more processors can provide, for presentation via a user interface, an output about the plurality of catheters placed within the organ, based on the metric for the treatment plan for the subject. In some embodiments, the one or more processors can receive, via the user interface, an indication of acceptance of the treatment plan defining placement of the plurality of catheters at the corresponding plurality of positions in the organ. In some embodiments, the brachytherapy can be administered on the feature in the organ of the subject, subsequent to presentation of the output.

[0006] In some embodiments, the one or more processors can receive, via the user interface, an indication of rejection of the treatment plan defining placement of the plurality of catheters at the corresponding plurality of positions in the organ. At least one of (i) a dosage to be provided via the plurality of catheters, (ii) a number of the plurality of catheters, or (iii) a placement of the plurality of catheters of the treatment plan can be modified, subsequent to presentation of the output.

[0007] In some embodiments, the one or more processors can receive, via an imaging device, a plurality of biomedical images of a section associated with the organ of the subject. Each of the plurality of biomedical image can correspond to a respective frame of a plurality of frames. In some embodiments, the one or more processors can to detect, based on providing the biomedical image to an image segmentation model, the plurality of positions and the contour. The image segmentation model can be established using the plurality of examples. Each of theplurality of examples can include (i) the respective biomedical image of a respective section associated with the respective organ of the respective subject, (ii) the respective plurality of positions, (ii) the respective contour.

[0008] In some embodiments, the metric indicating the quality of the treatment plan can be based on a plurality of dose-volumetric parameters in brachytherapy. The plurality of dose- volumetric parameters can include at least one of a D2cc metric, a D20cc metric, a D15cc metric, a DIOcc metric, a D5cc metric, a Dice metric, a DO. Icc metric, or a D20% metric. In some embodiments, the biomedical image can be in accordance with ultrasound imaging. The ultrasound imaging can include at least one of an abdominal ultrasound, a pelvic ultrasound, an obstetric ultrasound, a breast ultrasound, a transvaginal ultrasound, a transrectal ultrasound, a doppler ultrasound, or a musculoskeletal ultrasound.[0009[ In some embodiments, the brachytherapy can include at least one of a high-dose rate (HDR) brachytherapy, medium-dose rate (MDR) brachytherapy, low-dose rate (LDR) brachytherapy, or pulsed-dose rate (PDR) brachytherapy. A radioactive isotope for the brachytherapy can include at least one of iridium-192, iodine-125, palladium -103, caesium-131, caesium-137, cobalt-60, ruthenium- 106, or radium 226. In some embodiments, the subject is at risk of or diagnosed with cancer on the organ, and wherein the cancer comprises at least one of lung cancer, brain cancer, head and neck cancer, colon cancer, rectal cancer, uterine cancer, endometrial cancer, stomach cancer, prostate cancer, ovarian cancer, cervical cancer, bladder cancer, skin cancer, or breast cancer.BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The foregoing and other objects, aspects, features, and advantages of the disclosure will become more apparent and better understood by referring to the following description taken in conjunction with the accompanying drawings, in which:(00111 FIG. 1 is a diagram of a prostate HDR workflow under one approach.

[0012] FIG. 2 is a diagram of a proposed prostate HDR workflow.

[0013] FIG. 3 is a diagram of catheter tracking and organ segmentation on ultrasound.(Solid line: ground truth; dotted line: detection results.)

[0014] FIGs. 4A and 4B depict the ground truth and the tracked catheters in 3D rendering.

[0015] FIGs. 5A and 5B depict catheter tracking and organ segmentation. (Solid line: ground truth; dotted line: detection results.)10016] FIG. 6A depicts scatter plots of testing patients for ground truth percentile (x- axis) and predicted percentile (y-axis) for rectum 2cc.

[0017] FIG. 6B depicts scatter plots of testing patients for ground truth percentile (x-axis) and predicted percentile (y-axis) for urethra D20%.

[0018] FIGs. 7A-7H depict results of needle detection on two exemplary patients (upper and bottom rows) from apex to base of prostate (from left to right). (Solid line: ground truth; dotted line: detection results.)

[0019] FIGs. 8A-8H depict results of organ segmentation on two exemplary patients (upper and bottom rows) from apex to base of prostate (from left to right). (Solid line: ground truth; dotted line: detection results.)

[0020] FIG. 9 depicts a block diagram of a system for determining quality of brachytherapy treatment plans from catheter placement in organs in subjects, in accordance with an illustrative embodiment.

[0021] FIG. 10 depicts a block diagram of a process to train machine learning (ML) models in the system for determining quality of brachytherapy treatment plans, in accordance with an illustrative embodiment.

[0022] FIG. 11 depicts a block diagram of a process to execute machine learning (ML) models in the system for determining quality of brachytherapy treatment plans, in accordance with an illustrative embodiment.

[0023] FIG. 12 depicts a block diagram of a process to package outputs in the system for determining quality of brachytherapy treatment plans, in accordance with an illustrative embodiment.

[0024] FIG. 13 depicts a flow diagram of a method of determining quality of brachytherapy treatment plan from catheter placement in organs in subjects, in accordance with an illustrative embodiment.

[0025] FIG. 14 is a block diagram of a computing environment according to an example embodiment of the present disclosure.DETAILED DESCRIPTION

[0026] Following below are more detailed descriptions of various concepts related to, and embodiments of, systems and methods for determining quality of brachytherapy treatment plan from catheter placement in organs in subjects. It should be appreciated that various concepts introduced above and discussed in greater detail below may be implemented in any of numerous ways, as the disclosed concepts are not limited to any particular manner of implementation. Examples of specific implementations and applications are provided primarily for illustrative purposes.

[0027] Section A describes real-time planning of high-dose-rate brachytherapy.

[0028] Section B describes instant plan quality prediction on transrectal ultrasound for high-dose-rate prostate brachytherapy.

[0029] Section C describes systems and methods for determining quality of brachytherapy treatment plan from catheter placement in organs in subjects.[0030| Section D describes a network environment and computing environment which may be useful for practicing various computing related embodiments described herein.A. Real-Time Planning of High-Dose-Rate Brachytherapy(0031 ] In the current procedure of high-dose-rate prostate brachytherapy, physicians insert catheters guided by ultrasound in the operating room. Subsequently, computer tomography (CT), magnetic resonance (MR), or ultrasound images are acquired, and manual delineation of target / organs-at-risk is performed for treatment plan optimization. Catheter placement relies on physician experience, and lacking feedback on plan quality during the implantation. Sub-optimal catheter implantation may lead to suboptimal plans or additional catheter adjustments requiring additional anesthesia time. A novel automatic, real-time catheter tracking and target or organ segmentation method can be used with the current plan optimization program to potentially provide an instant plan quality feedback permitting physicians to optimize needle placement, and expediting the subsequent planning process.

[0032] A deep learning neural network was developed to take the last 5 frames of the real-time videos from ultrasound and provide the coordinates of all the catheters it detected, as well as the contours of prostate, rectum and urethra, on the last frame. After the ultrasound probe scanned the entire prostate region, the catheter coordinates on each frame were then fitted to corresponding 3D lines in order to produce the line functions of each catheter in 3D space, as well as the segmented contours of each frame were stacked together. A total of 518 patients who underwent prostate HDR brachytherapy as boost treatment were retrospectively investigated, each of which had ultrasound images acquired, contoured and digitized for treatment planning after catheter placement. Among them, 482 patients were used for the training cohort and 36 patients were used for the testing cohort. The median number of catheters per patient was 14.

[0033] Among the 477 catheters in the testing patients, the proposed method successfully detected 472 catheters, with an accuracy of 99.0%. The average displacement between the detected catheters and the ground truth catheters on 2D ultrasound images is 0.63±0.55 mm. Themean Dice score for prostate segmentation is 0.90±0.08. The maximum distance of rectum between ground truth and segmentation is 2.80±1.71 mm on average among all patients. The mean center distance of urethra between ground truth and segmentation is 0.76±0.56 mm. The mean time of processing each frame is 15.54±1.31 ms.|0034| The accuracy and efficiency of the proposed method in tracking catheters and segmenting target and organs have been demonstrated with retrospective ultrasound data. It is seen that the proposed artificial intelligence-based method can facilitate a real-time, US-based automatic treatment planning program for prostate HDR brachytherapy.

[0035] In the prostate High-Dose-Rate (HDR) workflow, catheter placement relies heavily on physician experience, with no feedback provided to the physician on the quality of the plan during the catheter placement process. If the physician is dissatisfied with the plan after the patient wakes up, it becomes challenging and often infeasible to add or adjust catheters, especially with CT- or MRI-based planning. While adding or adjusting catheters is possible with ultrasound (US)-based planning, the process is tedious and requires starting over, or it can be inaccurate due to the rough estimation of new catheter positions on the existing plan. (See FIG. 1.) Hence, there is a clear need for real-time feedback to improve the planning process.

[0036] Longer anesthesia times during surgeries or procedures can be associated with increased rates of non-urological complications (NUCs) and perioperative mortality. For example, according to a Routh, Ashley C., et al., 2002, the incidence of NUCs rose significantly with longer anesthesia durations: 3.1% for anesthesia lasting less than 4 hours, 5.8% for 4-6 hours, and 13.5% for anesthesia exceeding 6 hours. The odds of experiencing NUCs were 1.91 times higher for the 4-6 hour group and 4.84 times higher for the group with anesthesia lasting more than 6 hours. Moreover, perioperative mortality was highest in the group with the longest anesthesia duration. Minimizing anesthesia time can reduce the risk of complications. For brachytherapy procedures, speeding up the procedure through real-time catheter tracking and organ segmentation can further reduce anesthesia risks, enhancing patient safety and outcomes.

[0037] Improving procedure efficiency can benefit medical personnel and overall hospital operations (Lavallee et al., 2024). The reduction in overall procedure time can benefit patient safety by reducing anesthesia risk and can optimize the use of operating room resources. Shorter procedure times can mean that medical personnel can manage more cases within the same time frame, improving overall hospital operations and productivity, and reducing patient wait times.

[0038] A dose volume histogram (DVH) is a histogram relating radiation dose to tissue volume in radiation therapy planning. DVHs are most commonly used as a plan evaluation tool and to compare doses from different plans or to structures. DVHs can be visualized in either of two ways: differential DVHs or cumulative DVHs. D20cc, D15cc, DIOcc, D5cc, D2cc, Dice, and DO. Icc are defined as the minimal doses received by the highest irradiated volumes of 20, 15, 10, 5, 2, 1, and 0.1 cc of normal tissue, respectively. D20% means a dose received by 20% of volume of a tissue with HDR-brachytherapy.

[0039] The proposed workflow introduces real-time feedback on plan quality directly in the operating room (OR) to effectuate shorter anesthesia times as well as reducing overall procedure time. The proposed workflow allows the physician to adjust catheter placement before starting the planning process, thereby minimizing unnecessary restarts and enhancing patient safety and treatment quality. By providing immediate feedback, the aim is to streamline the entire HDR brachytherapy workflow and improve clinical outcomes. (See FIG. 2.)

[0040] To estimate plan quality in real-time, the following components can be needed: real-time catheter tracking, real-time organ segmentation, and real-time quality prediction based on the results from the first two steps. This comprehensive system ensures that the physician receives timely and accurate feedback during the catheter placement process. FIG. 3 is a diagram of catheter tracking (red boxes) and organ segmentation on ultrasound (solid line: ground truth; dotted line: organ segmentation results).

[0041] The system can employ advanced Al models for segmentation and catheter tracking. Segmentation and catheter tracking can be performed using the YOLOv8n model, which is an open-source software with a size of less than 10 MB and can run on a laptop. The model processes at a speed of 5 milliseconds per frame using a GPU (RTX 3070) and 50 milliseconds per frame using a CPU (i7-l 1800H). The models (e.g., YOLOv8) were implemented for catheter digitization and organ segmentation. YOLOv8 is an open source cutting-edge model for real-time object detection and image segmentation. In the context of ultrasound imaging, two models were trained separately to track catheters and segment organs, leveraging its real-time processing capabilities and robust detection algorithms to handle the complex and dynamic nature of medical images. This model is particularly suitable for applications requiring both high precision and speed, making it an ideal choice for medical imaging tasks.

[0042] For the training of catheter tracking model, the detection version of model was used with the inputs of the 2D slices of ultrasound images, and the training target is the corresponding coordinates of catheters if there are any on that slice of image. For organ segmentation, the segmentation version of model was used with the same inputs of the 2D slices of ultrasound images, and the training target is the coordinates of contours of prostate, rectum, and urethra if there are any on that slice of image. Note that the slices without any catheters or organs were also included as training dataset.[0043| Plan quality prediction can be performed using a regressor model, which is approximately 150 kB in size and returns results in 10 milliseconds. The input for this model can include the results from the catheter tracking and organ segmentation models. The output is the percentile ranking among all patients in the database for each dose constraint, with the plan normalized to 95% coverage of Rx dose. The output of the model may be the percentile among all patient in database for each dose constraint. Note that all plans are normalized to 95% coverage of Rx dose.

[0044] To predict the plan quality based on the information of organ contours and catheter positions, a machine learning model (e.g., a ResnetlOl or Residual Network with 101 layers) may be implemented. It is a deep convolutional neural network renowned for its exceptional performance in image recognition tasks. Since the input of model is supposed to be image, the contours and catheter coordinates were converted into binary maps by assigning different values to these objects: prostate with 60, rectum with 120, urethra with 180, catheter with 240, and background with 0. The 2D slices were stacked of these binary maps for each patient as a 3D binary volume. The input for this model is thus the 3D binary volumes of each patient.

[0045] In this present disclosure, the plan quality was quantified based on the commonly used dosimetric metrics in prostate HDR brachytherapy: rectum D2cc and urethra D20%. Since these two metrics are largely dependent on how the plan dose is normalized, all the plans were normalized in the dataset to have 100% of the prescription dose to cover 95% of the prostrate. The training target of this model is thus the number of rectum D2cc and urethra D20% for each patient plan after normalization.

[0046] The training data consists of HDR prostate boost patients treated , with a robust training, validation, and test set. A cohort of 504 patients were retrospectively collected who underwent prostate high-dose-rate (HDR) boost brachytherapy treatment with 15 Gy in the institution. Each patient has planning TRUS acquired during procedure in the OR, with contours of prostate, rectum, and urethra delineated by physicians, catheters digitized by physicists, and plan optimized by physicists and approved by physicians. The TRUS images were acquired on axial slices and saved as DICOM with size of 1024x768, pixel spacing of 0.1x0.1 mm2and slice thickness of 1 mm. The related plan file and structure file were also read to extract the location of catheters and organ contours on the transrectal ultrasound (TRUS) images. Among the 504 patients, 456 of them were used for training, 24 for validation, and 24 for testing.

[0047] The above deep learning networks were implemented using Python 3.6 and Pytorch on a Nvidia A40 graphic card with 48GB of memory. For catheter detection and organsegmentation, the default setting of model training parameters was used except epochs was set to 100. For plan quality prediction model, to reduce the computational cost, the input 3D binary volume is first zero-padded to 1024x1024x128, and then down sampled to 128x 128x 128. Random horizontal flipping and random 3D translation were applied as data augmentation methods to reduce overfitting. A total of 200 epochs were used. Optimization was performed using the Adam gradient optimizer. The learning rate was 2x 10-3. The catheter detection and organ segmentation model were less than 10 MB, and the plan quality prediction model was less than 350 MB. The inference time for catheter detection and organ segmentation model were around 1 ms for each slice and for plan quality prediction model was 17ms for each patient.

[0048] To quantitatively evaluate the performance of the proposed workflow, the generated results from Al models with ground truth by corresponding metrics was compared. For catheter tracking, the distance between the detected catheter and the ground truth were evaluated. For the segmentation of prostate, Dice Similarity Coefficient (DSC) was used to evaluate the overlapping of the segmented results and physician’s contours. For rectum, the Hausdorff Distance (HD) was measured between the segmented contours and ground truth contours on the side close to prostate where the rectum D2cc is sensitive. For urethra, the center of distance was used (CD) between segmentation results and ground truth considering the tiny size of urethra. For plan quality prediction, first, all the patient plans were ranked in the dataset by their rectum D2cc and urethra D20% metrics respectively. For each testing patient, the predicted dosimetric value was used to find its percentile in the dataset, and compared with its true percentile using the ground truth dosimetric value.(0049] During clinical implementation, the catheter tracking and organ segmentation models may be implemented and results may be saved for each axial TRUS image in real-time when physician is moving probe forward or backward in the rectum during the procedure. Combined with the information of the location of each TRUS image read from the stepper, the latest catheter tracking and organ segmentation results on slice at each depth may then be fed into the quality prediction model.

[0050] The preliminary results are promising. For real-time tracking catheters, a 99% detection rate for catheters among 35 patients was achieved (successfully detected 473 / 477 (99%) catheters among 35 patients), with an average displacement of only 0.63±0.55 mm between detected and ground truth catheters. For organ segmentation, the model attained a Dice score of 0.90±0.08 for the prostate, and acceptable maximum distances for rectum and urethra segmentations. The maximum distance between the segmented rectum and the ground truth is 2.80±1.71 mm. The error in the center of the urethra is 0.76±0.56 mm. These results demonstrate the accuracy and reliability of the real-time tracking and segmentation models. FIG. 4A depicts the ground truth and FIG. 4B depicts the tracked catheters in 3D rendering. FIG. 5A depicts catheter tracking and FIG. 5B depicts organ segmentation (solid line: ground truth; dotted line: results).(0051] In terms of real-time quality prediction, the system successfully identified ‘Below Average’ plans with a high true positive rate: 83% for rectum D2cc and 87% for urethra D20%. FIG. 6A depicts scatter plots of testing patients for ground truth percentile (x-axis) and predicted percentile (y-axis) for rectum 2cc. FIG. 6B depicts scatter plots of testing patients for ground truth percentile (x-axis) and predicted percentile (y-axis) for urethra D20%. It is seen that most dots are accumulated along the diagonal line, which indicates high relevance between prediction and ground truth.[0052 It is noted that the proposed method can provide these information to the physician without the need for digitizing, optimization, and manual dose shaping, thus significantly streamlining the treatment planning process. In sum, the proposed method shows high detection rates and accuracy in both catheter tracking and organ segmentation, along with effective plan quality prediction. From a clinical perspective, the integration of real-time feedback can streamline the HDR brachytherapy workflow, reducing overall treatment time and enhancing precision. The practical deployment of this system in clinical settings may entail thorough validation and training for medical personnel to ensure seamless integration into existing workflows.

[0053] In conclusion, the proposed method for real-time feedback in prostate HDR brachytherapy shows great potential in enhancing clinical workflow and improving treatment outcomes. The high detection rates and accuracy in both catheter tracking and organ segmentation, along with effective plan quality prediction, highlight the effectiveness of the approach. The aim is to refine Al models further and validate the system in clinical settings. Specifically, the dataset may be expanded to include broader patient demographics.B. Instant Plan Quality Prediction on Transrectal Ultrasound for High-Dose-Rate Prostate Brachytherapy

[0054] Purpose: The feasibility of Al to provide an instant feedback of the potential plan quality based on live needle placement, and before planning is initiated, was investigated.|0055[ Materials and Methods: YOLOv8 was utilized to perform automatic organ segmentation and needle detection on 2D transrectal ultrasound images. The segmentation and detection results for each patient were then fed into a plan quality prediction model based on ResNetlOl . Its outputs are values of selected dose volume metrics. Imaging and plan data from 504 prostate HDR boost patients (456 for training, 24 for validation, and 24 for testing) treated in the clinic were included in this study. The segmentation, needle detection, and prediction results were compared to the clinical results (ground truth).

[0056] Results: For prediction model, the p-values of t-test between the predicted values and ground truth for either rectum D2cc or urethra D20% were larger than 0.8. The sensitivity of prediction model in finding implant geometries resulting in below-median rectum D2cc and urethra D20% were 83% and 87%.

[0057] Conclusion: The proposed method has great potential to facilitate the current prostate HDR brachytherapy workflows by providing valuable feedback during needle insertion, and facilitating decision making of where and if additional needles are required.Introduction

[0058] High-dose-rate (HDR) brachytherapy has been widely practiced for prostate cancer treatment. During this procedure, typically eight to eighteen needles are interstitially implanted in the prostate under the guidance of transrectal ultrasound (TRUS) in the operating / procedure room (OR) after patient is anesthetized. After implantation, planning images, of which the modality depends on the choice of the institution, are acquired for target / organs-at-risk (OARs) delineation as well as catheter digitization in the treatment planning system, followed by plan optimization.

[0059] Despite the ability to optimize dwell locations and dwell times via inverse optimization, needle placement is an essential component that heavily impacts the achievable plan quality. Currently, needle placement relies on physician experience and lacks instant feedback on expected plan quality. A suboptimal needle implantation is difficult to identify until the plan optimization stage is initiated. If needle implantation is found unsatisfying, adding extra needles is an option for patients who are still under sedation but will require additional OR time to re-acquire planning image and repeat the planning process. If anesthesia has already been reversed, changes in needle placement under TRUS guidance may be practically impossible. A way sometimes used to minimize this risk is to implant a number of needles exceeding the minimum needed to achieve a good plan. This also has a cost in terms of OR time, longer and harder digitization process, and possible additional risk to the patient from the extra punctures. Thus, it is desirable to have an instant feedback of the potential plan quality based on the current needle placement such that physicians can adjust it before planning image acquisition, thereby minimizing unnecessary restarts, and enhancing patient safety and treatment quality.

[0060] In this disclosure, a workflow was proposed to provide instant plan quality prediction in the OR for HDR prostate brachytherapy by utilizing artificial intelligence (Al). The proposed method is expected to predict dose volumetric histogram of organs-at-risk on the axial TRUS images across the prostate right after the implants are done, allowing physicians to amend needle insertion in time without going through the entire planning process includingcontouring, digitization and plan optimization. Providing useful feedback requires accurately and rapidly localizing the position of all needles and the shape of prostate and OARs as the first step.

[0061] Recently, deep learning has been introduced in medical imaging field as a powerful tool in detection and segmentation tasks. A few studies have demonstrated the feasibility in using deep learning to detect needles on TRUS for prostate HDR brachytherapy. Moreover, auto-segmentation on TRUS for prostate and relevant organs-at-risk (OARs) has also been shown practicable. However, currently none of these studies investigated the possibility of using these Al-generated results to further predict plan quality using Al. On the other hand, predicting dose or dosimetric metrics have been actively studied in external beam radiation therapy, while similar studies for brachytherapy are sparse.[0062| In this study, Al models were used to track needles and segment prostate and OARs, and investigated the feasibility of further using Al to predict plan quality based on these Al-generated results. By providing immediate plan quality feedback with the proposed method, the aim is to streamline the entire prostate HDR brachytherapy workflow and improve clinical outcomes.Methods and materialsData

[0063] In this study, a cohort of 504 patients treated at the institution with HDR prostate boost brachytherapy to a total dose of 15 Gy were retrospectively collected. Each patient has planning TRUS images acquired during procedure in the OR. Contours of prostate, rectum and urethra were delineated by physicians, catheters were digitized by physicists, and plans were optimized by physicists and approved by physicians. The planning process was performed on Vitesse treatment planning system from Varian. The TRUS images were acquired on axial slices and saved as DICOM with size of 1024 x 768, pixel spacing of 0.1 x 0.1 mm2and slice thickness of 1 mm. The related RTplan file and RTstructure file were also read to extract the location ofcatheters and organ contours on the TRUS images. Among the 504 patients, 456 of them were used for training, 24 for validation, and 24 for testing.Needle detection and organ segmentation

[0064] YOLOv8 models were implemented for needle detection and organ segmentation. YOLOv8 is an open source model for real-time object detection and image segmentation. It builds upon previous versions of YOLO by incorporating advanced features such as improved architecture, more efficient use of computational resources, and higher detection accuracy. In the context of ultrasound imaging, two YOLOv8 models were trained separately to track needles and segment organs, leveraging its real-time processing capabilities and robust detection algorithms to handle the complex and dynamic nature of medical images. This model is particularly suitable for applications requiring both high precision and speed, making it an ideal choice for medical imaging tasks.

[0065] For the training of needle tracking model, the detection version of YOLOv8 was used with the inputs of the 2D slices of ultrasound images, and the training target is the corresponding coordinates of the manually digitized needles, if there are any on that slice of image. For organ segmentation, the segmentation version of YOLOv8 was used with the same inputs of the 2D slices of ultrasound images, and the training target is the coordinates of manually generated contours of prostate, rectum and urethra if there are any on that slice of image. Note that the slices without any catheters or organs were also included as training dataset.

[0066] For both detection and segmentation version of YOLOv8 models, different sizes of models were provided. All the available sizes of modes were evaluated: YOLOv8-n(ano), YOLOv8-s(mall), YOLOv8-m(edian), YOLOv8-l(arge) and YOLOv8-x(L). These models have increasing number of parameters, and are expected to have a greater capacity to learn complex patterns and representations from the data.Plan quality prediction

[0067] To predict the plan quality based on the information of organ contours and needle positions, a ResnetlOl, e.g.. Residual Network with 101 layers, was implemented. It is a deep convolutional neural network renowned for its exceptional performance in image recognition tasks. Since the input of ResnetlOl is supposed to be images, the contours and needle coordinates were converted into binary maps by assigning different values to these objects: prostate with 60, rectum with 120, urethra with 180, catheter with 240, and background with 0. The 2D slices of these binary maps were stacked for each patient as a 3D binary volume. The input for this model was thus the 3D binary volumes of each patient.

[0068] The plan quality was quantified based on the commonly used dosimetric metrics in prostate HDR brachytherapy: rectum D2cc and urethra D20%. Since these two metrics are largely dependent on how the plan dose is normalized, all the plan in the dataset was normalized to have 100% of prescription dose to cover 95% of prostate. The training target of this model was rectum D2cc and urethra D20% for each patient plan after normalization. By doing this, the prediction model was expected to tell the physician how the current needle pattern can spare dose in rectum and urethra when the plan is normalized to 95% of CTV covered by prescription dose.Implementation and evaluation

[0069] The above deep learning networks were implemented using Python 3.6 and Pytorch. Training was implemented on a Nvidia A40 graphic card with 48GB memory, while testing was implemented on a Nvidia 2080Ti with 11GB memory. For catheter detection and organ segmentation, the default setting of YOLOv8 training parameters were used except epochs was set to 100. For plan quality prediction model, to reduce the computational cost, the input 3D binary volume is first zero-padded to 1024 x 1024 x 128, and then downsampled to 128 x 128 x 128. Random horizontal flipping and random 3D translation were applied as data augmentation methods to reduce overfitting. A total of 200 epochs were used. Optimization was performedusing the Adam gradient optimizer. The learning rate was 2 * 10'3. The needle detection and organ segmentation models were less than 10 MB and the plan quality prediction model was less than 350 MB. The inference time for needle detection and organ segmentation model were around 15 ms for each slice and for plan quality prediction model was 17 ms for each patient.|0070| To quantitatively evaluate the performance of the proposed workflow, the generated results from Al models were compared with the corresponding metrics in the ground truth clinical plans. For needle tracking, the distance between the detected needles and the ground truth was evaluated. For the segmentation of prostate, Dice Similarity Coefficient (DSC) was used to evaluate the overlapping of the segmented results and physician's contours. For rectum, the Hausdorff Distance was measured at 95% (HD95) between the segmented contours and ground truth contours on the side close to prostate where the rectum D2cc is sensitive. For urethra, the center of mass distance (CMD) between segmentation results and ground truth was used, considering the tiny size of urethra. For plan quality prediction, all the patient plans were first ranked in the dataset by their rectum D2cc and urethra D20% metrics respectively. For each testing patient, the predicted dosimetric value was used to find its percentile in the dataset, and compared with its true percentile using the ground truth dosimetric value.ResultsNeedle detection

[0071] FIGs. 7A-H shows the catheter detection results at different slices on two exemplary patients by Yolov8-n. Qualitatively, almost all the needles were successfully detected, the localized positions were close to manual digitization. Table 1 summarized the quantitative performance of catheter detection by different sizes of Yolo models. For example, for Yolov8-n, among the total of 25327 needles on the 2D slices of all testing patients, the needle detection model successfully detected 23739 of them, which resulted into a sensitivity of 93.7%. The model also returned 2212 needles that do not correspond to any ground truth needles, which resulted into a false discovery rate of 8.5%. Among the 23739 needles that were successfullydetected, the distance between the detected location and the ground truth location is 0.65 mm on average. Overall, all the five models presented very close performance. They were able to detect more than 90% of the needles with submillimeter accuracy and less than 10% false discovery rate on each axial TRUS slice.|0072| Table 1

[0073] The performance of needle detection and organ segmentation and the combined inference time by different sizes of models. The ground truth total number of needles is 25327.Organ segmentation

[0074] FIGs. 8A-H demonstrate the organ segmentation results from prostate apex to base on two exemplary patients by Yolov8-n. The segmentation model was able to provide the prostate contours accurately close to physician's contours around mid-gland, and with less accuracy at base and apex. Urethra and the upper surface of rectum were also segmented, withaccuracy apparently dependent on contrast shown on the images. When the evaluation metrics for rectum were calculated, only the region in the fan-shaped ultrasound scan plane was considered. The quantitative results of organ segmentation are summarized in Table 1. All the five models demonstrated similar performance. The average inference time for performing both needle detection and organ segmentation on each slice are listed in Table 1 for the different sizes of models. For a TRUS volume with 100 slices, the total processing time would be around 1 second to 3 seconds.

[0075] Table 2

[0076] Mean ± SD of dose volume histogram (DVH) metrics of ground truth and prediction and their difference on the testing patients. P-values are listed for the t-test between the values of ground truth and prediction.Rectum D2cc (%) Urethra D20% (%)Ground Truth 60.0 ± 7.2 113.6 ± 8.1Prediction 59.6 ± 9.2 113.8 ± 8.0Difference -0.4 ± 7.0 0.2 ± 5.7P-value 0.86 0.94Plan quality prediction

[0077] Considering the very close performance of all the five models, the smallest model, Yolov8-n, was selected to perform needle detection and organ segmentation for the investigation of plan quality prediction. As seen from Table 2, for both metrics of Rectum D2cc and Urethra D20%, the predicted values are very close to the ground truth on average (p > 0.05). The sensitivity and specificity of using predicted values in finding needle placement with below- median quality for each metric was further investigated. As an example, for rectum D2 cc, among the 24 testing patients, there were 10 patients having predicted values below the median value among the entire patient dataset in comparison with 12 patients actually below the median from ground truth, which resulted into a sensitivity of 83%. The remaining results are summarized in Table 3. The average inferring time of the prediction model was 17.5 ms.

[0078] Table 3

[0079] Sensitivity and specificity of prediction model in finding below-median-quality catheter placement regarding each metrics.1 Rectum D2cc (%) Urethra D20% (%)Sensitivity 83% 87%Specificity 75% 89%Discussion|0080| This study is the first demonstration that Al can be utilized to provide valuable immediate feedback on projected brachytherapy implant quality. While further prospective analysis of the impact of this approach is needed to confirm its applicability to a variety of clinical situations, the potential of reducing the experience gap in brachytherapy can be of great importance for many clinics. A novel workflow was presented to provide instant plan quality prediction for high-dose-rate prostate brachytherapy by utilizing Al. The proposed workflow aims to streamline the entire HDR prostate brachytherapy workflow by providing immediate feedback on the catheter placement during needle implantation. The feedback is essential for physicians to timely optimize needle placement without the need to start treatment planning to have that feedback. The use of Al has the potential to reduce variability in implant quality from physician procedural experience. This approach can be of particular utility for clinics with low brachytherapy volume, permitting to overcome a skill and procedure experience gap that could otherwise discourage the use of brachytherapy or result in suboptimal implant quality. Busy brachytherapy centers can also benefit from the implementation of this technology as it would offer a quality assurance step to their clinical operation.|0081[ The method uses three open-source deep learning models to perform needle detection, organ segmentation and plan quality prediction. In-house Al models for needle detection and organ segmentation have been investigated in previous publications. Compared with those results, the models present comparable performance. The DSC for prostate segmentation was found to be comparable to the results in other studies focusing on segmentingprostate on TRUS. For example, one approach reported average DSC of 0.92 and 0.93 on prostate in their studies. Moreover, the segmentation performance on rectum and urethra was found superior to studies where rectum HD95 was reported at 1.90 mm and urethra CMD at 1.82 mm. For needle detection, since previous studies focused on detection in 3D space while the study performed on 2D slices, a direct comparison is not feasible. However, in general, these methods are able to detect around 95% catheters in 3D space, which is close to the 94% achieved in 2D slices in this study. Although the performance of open-source models is comparable to the previous in-house Al studies, using open-source models are more advantageous since they are more accessible and easier to implement. A physicist with basic knowledge in python and deep learning is expected to be able to reproduce the method in this study, without advanced knowledge in modifying neural network architectures. These advantages would facilitate a widespread integration of Al in brachytherapy workflow.

[0082] On the other hand, using Al to predict plan quality for prostate HDR brachytherapy has not been investigated before. The most relevant study is using deep learning to predict dose map for cervical cancer HDR brachytherapy. In this study, the output of the model is the predicted 3D dose volume. Compared with the output of several scalar numbers, generating a 3D volume may inevitably require much more computational resources, and thus may not be suitable for instant usage. As mentioned in their study, it took about 12 seconds to predict the dose map, comparing with 17.5 ms of the method described herein in providing selected dose volume metrics with a similar level of GPU. While 12 seconds is not in itself a long wait time, it would not be conducive to provide continuous implant feedback, as the method described herein would.

[0083] Similar performance of Yolov8 models with different sizes were shown in this study. A potential reason can be the limited number of training dataset, which may lead to overfitting in larger models. In this study, a total of 34258 slices of TRUS were used for training, which are much less than the commonly used dataset in natural images such as COCO dataset with 118K image. For brachytherapy, the size of available dataset for a single institutionis inevitably small due to the limited patient throughput. In future, a multi-institution dataset can be collected to enrich not only the size but also the diversity of dataset.

[0084] This study demonstrated the methodology feasibility of using Al in plan quality prediction in prostate HDR brachytherapy. Although this study used dataset from the institution where the brachy plan is performed on TRUS images, the proposed method may also be applicable to CT-based or MRI-based planning as long as the catheter placement step is under TRUS guidance. Moreover, although the focus of this disclosure was in HDR prostate, the workflow was believed to be generalizable to low-dose-rate (LDR) prostate brachytherapy procedures as well.

[0085] Presented herein is an approach to predict plan quality based on the current needle placement. It provides useful information for physicians in making decision on whether to accept the current placement or not. A further direction of this study can be investigating the optimal catheter placement pattern given the contours of prostate and OARs. Considering the operating uncertainty of physicians, the actual placement may deviate from the optimal position. Thus, a live update on the optimal pattern after each needle implanted is more desirable.Conclusion

[0086] A novel workflow was developed to provide instant plan quality prediction based on TRUS in OR for high-dose-rate prostate brachytherapy by utilizing Al. The method automatically segmented prostate and OARs, and detected placed needles. The segmentation and detection results were then used predict relevant dose volume metrics. It provides an effective solution in giving feedback on catheter placement to physicians, which is essential to minimize unnecessary restarts and compromise in plan quality, thus enhancing patient safety and treatment outcomes.C. Systems and Methods for Determining Quality of Brachytherapy Treatment Plans from Catheter Placement in Organs of Subjects

[0087] Referring now to FIG. 9 depicts a block diagram of a system 100 for determining quality of brachytherapy treatment plan from catheter placement in organs in subjects. In brief overview, the system 100 can include at least one data processing system 105, at least one imaging device 110, at least one administrative device 115, and at least one database 120, communicatively coupled via at least one network 125. The data processing system 105 can include at least one model trainer 130, at least one feature detector 135, at least one location evaluator 140, at least one data indexer 145, at least one output packager 150, at least one feedback handler 155, at least one image segmentation model 160, and at least one prediction model 165, among others. Each of the components of the system 100 may be implemented using hardware or a combination of software and hardware as described in Section D.

[0088] In further detail, the data processing system 105 can be any computing device comprising one or more processors coupled with memory and software capable of performing the various processes and tasks described herein. The data processing system 105 can be housed within a computing system (e.g., laptop, PC, smart device) or within a server group (e.g., a data center, a branch office, or a server site), and include instructions to receive biomedical images associated with brachytherapy plans and determine quality metrics for the brachytherapy plans. The data processing system 105 can be in communication with the imaging device 110, the administrative device 115, and the database 120, among others. The data processing system 105 can perform any of the functionalities described herein in Sections A and B.[0089| The data processing system 105 may have one or more components, modules, processes, and threads to perform the various processes and tasks described herein. On the data processing system 105, the model trainer 130 can initialize, train, and establish the image segmentation model 160 and the prediction model 160. The feature detector 135 can execute the image segmentation model 160 using a biomedical image associated with a brachytherapy plan to detect structures of interest (SOIs) and positions of catheters. The location evaluator 140 can execute the prediction model 165 using the SOIs and the positions of catheters to determine a quality metric for the brachytherapy plan. The data indexer 145 can obtain the biomedical image of a section with an organ to be administered with brachytherapy. The output packager 150 cangenerate an output to provide based on the quality metric for the brachytherapy plan. The feedback handler 155 can receive responses to the output on the brachytherapy plan.

[0090] The image segmentation model 160 can be any type of artificial intelligence (Al) algorithm or machine learning (ML) model to detect structures of interest (SOIs) and positions of catheters from biomedical images. The image segmentation model 160 can include, for example, a deep learning artificial neural network (ANN) (e.g., a residual network (ResNet), convolution neural network (CNN), a recurrent neural network (RNN), a transformer model, a diffusion model, an autoencoder model, etc ), a clustering algorithm (e.g., k-nearest neighbors), a support vector machine (SVM), a decision tree, a Bayesian model, or a regression model (e.g., linear or logistic regression), among others. In general, the image segmentation model 160 can include inputs and outputs related to one another via a set of weights. The set of weights may be in accordance with the ML model (e.g., ResNet) used to implement the image segmentation model 160. The input can include a biomedical image having at least one SOI corresponding to an organ feature and one or more objects corresponding to catheters placed about the organ feature in accordance with a brachytherapy plan. The output can include a contour defining the SOI within the biomedical image and one or more positions corresponding to the catheters.

[0091] The prediction model 165 can be any type of Al algorithm or ML model to determine a quality metric for the brachytherapy plan based on the contour and the one or more positions detected in the biomedical image. The prediction model 165 can include, for example, a deep learning artificial neural network (ANN) (e.g., a residual network (ResNet), convolution neural network (CNN), a recurrent neural network (RNN), a transformer model, a diffusion model, an autoencoder model, etc.), a clustering algorithm (e.g., k-nearest neighbors), a support vector machine (SVM), a decision tree, a Bayesian model, or a regression model (e.g., linear or logistic regression), among others. In general, the prediction model 165 can include inputs and outputs related to one another via a set of weights. The set of weights may be in accordance with the ML model (e.g., CNN) used to implement the prediction model 165. The input can include the contour corresponding to the SOI and the one or more positions detected in the biomedical image. The output can include the quality metric for the brachytherapy plan.

[0092] The imaging device 110 (sometimes herein generally referred to as an imaging device or an image acquirer) can be any device that can acquire biomedical images of subjects. The biomedical image can be in accordance with ultrasound imaging. The ultrasound imaging can include, for instance, at least one of abdominal ultrasound, a pelvic ultrasound, an obstetric ultrasound, a breast ultrasound, a transvaginal ultrasound, a transrectal ultrasound, a doppler ultrasound, or a musculoskeletal ultrasound, among others. The ultrasound imaging can be in accordance with any mode to capture visualization of the organs within the subject. The modes can include, for example, at least one of a brightness mode (B-mode) to capture intensity of reflected ultrasound waves, a motion-mode (M-mode) to capture motion, or Doppler mode, among others. The imaging device 110 can have a transducer to convert electrical energy to sound waves to apply to a target area, to receive the reflected sound waves back from the target area, and to convert back the received sound wave to electrical energy to perform imaging. The imaging device 110 can be in communication with the data processing system 105, the administrative device 115, and the database 120, among others. While described primarily herein in terms of ultrasound imaging, other imaging modalities can be used, such as X-rays, computed tomography (CT), magnetic resonance imaging (MRI), or positron emission tomography (PET), among others.]0093| The administrative device 115 (sometimes herein referred to as a client device, a client, or an end user computing device) can be any computing device including one or more processors coupled with memory and software and capable of performing the various processes and tasks described herein. The administrative device 115 can be in communication with the data processing system 105 and the imaging device 110, via the network 125. The administrative device 115 can have at least one display. The administrative device 115 can be associated with an entity (e.g., a clinician) managing the scanning of biomedical images of a subject or the administration of brachytherapy on the subject. The display can present information provided by the data processing system 105.

[0094] The database 120 (sometimes herein referred to as a data repository) can store and maintain various resources and data associated with the data processing system 105, the imagingdevice 110, and the administrative device 115, among others. For example, the database 120 can store training data used by the model trainer 130 to train the image segmentation model 160 and the prediction model 165. The database 120 can include a database management system (DBMS) to arrange and organize the data maintained thereon. The database 120 can be in communication with the data processing system 105, the imaging device 110, and the administrative device 115, via the network 125. While running various operations, the data processing system 105, the imaging device 110, and the administrative device 115 can access the database 120 to retrieve identified data therefrom. The data processing system 105, the imaging device 110, and the administrative device 115 can also write data onto the database 120 from running such operations.

[0095] Referring now to FIG. 10, depicted is a block diagram of a process 200 to train machine learning (ML) models in the system 100 for determining quality of brachytherapy treatment plan. Under the process 200, the model trainer 130 can initialize, train, and establish the image segmentation model 160 and the prediction model 165. To initialize, the model trainer 130 can instantiate or create the image segmentation model 160 and the prediction model 165, each with the weights assigned to random values. To train the image segmentation model 160 and the prediction model 165, the model trainer 130 can retrieve, obtain, or identify training data 205 from the database 120. The training data 205 can identify or include a set of examples. The set of examples can be used to train the image segmentation model 160 and the prediction model 165 in accordance with learning technique (e.g., supervised or weakly supervised learning).

[0096] Each example of the training data 205 can be associated with a given subject 210. The subject 210 can be a human or an animal. The subject 210 can be at risk of or diagnosed with cancer. The cancer can include, for example, at least one of lung cancer, brain cancer, head and neck cancer, colon cancer, rectal cancer, uterine cancer, endometrial cancer, stomach cancer, prostate cancer, ovarian cancer, cervical cancer, bladder cancer, skin cancer, or breast cancer, among others. The cancer can affect or be associated with at least one organ 215 (sometimes herein referred to as organ-as-risk (OAR)) in the subject 210. The organ 215 can correspond to any anatomical site associated with the cancer in the subject 210 administered withbrachytherapy. The anatomical site for the organ 215 can include, for example, lung, brain, head and neck, colon, rectum, uterus, stomach, prostate, ovaries, cervix, bladder, skin, or breast, among others. In some embodiments, the organ 215 can correspond to a primary site where the cancer originated in the subject 210. In some embodiments, the organ 215 can correspond to a secondary site to which the cancer metastasized to in the subject 210 from the primary site. The organ 215 can have at least one feature 220 (sometimes herein referred to as a gross tumor volume (GTV), planning target volume (PTV), or a target) associated with the cancer. The feature 220 can include, for example, a tumor, such as soft-tissue sarcoma, bone sarcoma, or malignant fibrous histiocytoma, among others.

[0097] The example can be generated from a prior or simulated administration of brachytherapy to the subject 210, in accordance with a treatment plan. Brachytherapy may be a form of radiation therapy in which a radioactive source (e.g., radioactive isotope) is placed inside or adjacent to the area to be treated. The brachytherapy can include, for example, a high-dose rate (HDR) brachytherapy, medium-dose rate (MDR) brachytherapy, low-dose rate (LDR) brachytherapy, or pulsed-dose rate (PDR) brachytherapy, among others. HDR brachytherapy can include dosage rates more than 12 Grays (Gy) per hour, such as 12 Gy per hour. Each administration session of HDR brachytherapy can have a time duration ranging between 1-30 minutes. MDR brachytherapy can include dosage rates between 2-12 Gy per hour. Each administration session of MDR brachytherapy can have a time duration ranging between 1-6 hours. LDR can include dosage rates between 0.4-2.0 Gy per hour. Each administration session of LDR brachytherapy can have a time duration ranging between 1 day to 6 months. PDR can include dosage rates between 0.4-2.0 Gy per hour, with discrete pulses ranging between 5 minutes to 2 hours. A radioactive isotope used for the brachytherapy can include, for example, at least one of iridium- 192, iodine-125, palladium- 103, caesium-131, caesium-137, cobalt-60, ruthenium- 106, or radium 226, among others.

[0098] For the administration of brachytherapy, one or more catheters 230A-N (hereinafter generally referred to as catheters 230) can be arranged, situated, or otherwise positioned about the organ 215 or about the feature 220 in the organ 215. Any number ofcatheters 230 can be placed within the organ 215 of the subject 210 manually by a clinician managing the administration of brachytherapy to the subject 210. Each catheter 230 can include a device used to deliver the radioactive source to the feature 220 of the organ 215. The catheter 230 can include, for example, a conduit or a tube for the delivery of radioactive isotope from outside the body of the subject 210 toward the feature 220 of the organ 215. The catheter 230 can be inserted into body cavities or through the tissue or the organ 215. The number of catheters 230 positioned about the feature 220, the positioning of the catheters 230 within the organ 215, and the dosage to be administered through the catheters 230 can define the treatment plan for the delivery of brachytherapy.

[0099] In each example, the training data 205 can identify or include a set of biomedical images 235A-N (hereinafter generally referred to as biomedical images 235). In some embodiments, the set of biomedical image 235 can be acquired in accordance with ultrasound imaging. The ultrasound imaging can include, for example, at least one of an abdominal ultrasound, a pelvic ultrasound, an obstetric ultrasound, a breast ultrasound, a transvaginal ultrasound, a transrectal ultrasound, a doppler ultrasound, or a musculoskeletal ultrasound, among others. In some embodiments, the set of biomedical images 235 can form a video of the section 238, and each biomedical image 235 can correspond to a respective frame in the video. The number of frames in the video and by extension the number of biomedical images 235 can range between 2 to 100 frames. The sampling rate can range between 1 to 15 MHz.[01001 Each biomedical image 235 can be of at least one section 238 in the subject 210. The section 238 can correspond to a cross-sectional region or volume of the subject 210 from which the biomedical image 235 is acquired or derived. The section 238 can contain, include, or otherwise be associated with the organ 215 (or the feature 220 of the organ 215) in the subject 210. The biomedical image 235 can include at least one structure of interest (SOI) 240. The SOI 240 can correspond to the feature 220 within the organ 215. For instance, the SOI 240 can be a segment of the biomedical image 235 that corresponds to a depiction of the feature 220. In addition, the biomedical image 235 can have include one or more objects 245 A-N (hereinafter generally referred to objects 245) corresponding the one or more catheters 230. Each object 245can correspond to a respective catheter 230 placed within the organ 215 of the subject 210. For instance, the object 245 can be a respective portion of the biomedical image 235 that corresponds to a depiction of the respective catheter 230.

[0101] The example of the training data 205 can identify or include one or more positions 250A-N (hereinafter generally referred to as positions 250). The one or more positions 250 can correspond to the one or more catheters 230 placed within the organ 215 of the subject 210, and by extension, correspond to the one or more objects 245 within the biomedical image 235. Each position 250 can define, specify, or otherwise identify a corresponding position of a respective catheter 230 within the organ 215 within the biomedical image 235. For instance, each position 250 can define a boundary (e.g., in terms of pixel coordinates) of a respective object 245 corresponding to the respective catheter 230 within the biomedical image 235. In some embodiments, the example can include the one or more positions 250 corresponding to the one or more catheters 230, for each biomedical image 235 in the video. At least one of the positions 250 can be at least partially different from frame to frame across different biomedical images 235 (e.g., due to movement of the organ 215). The one or more positions 250 have been manually generated by a clinician examining the biomedical image 235.

[0102] The example of the training data 205 can also identify or include at least one contour 255. The contour 255 can specify, identify, or otherwise define the SOI 240 in the biomedical image 235 corresponding to the feature 220 in the organ 215. For instance, the contour 255 can define a boundary of the SOI 240 within the biomedical image 235. The boundary can be defined in terms of pixel coordinates within the biomedical image 235 or coordinates in a virtual anatomical model of the organ 215 or the subject 210. In some embodiments, the example can include the contour 255 corresponding to the SOI 240 for each biomedical image 235 in the video. The SOI 240 can be at least partially different from frame to frame across different biomedical images 235 (e.g., due to movement of the organ 215). The contour 255 may have been manually generated by a clinician examining the biomedical image 235 and outlining the SOI 240 therein.

[0103] In addition, the example of the training data 205 can identify or include at least one quality metric 260 for the treatment plan. The quality metric 260 can identify, measure, or otherwise indicate a quality of the treatment plan, as defined by the one or more catheters 230 placed relative to the feature 220 in the organ 215 and by extension, by the one or more positions 250 relative to the contour 255 in the biomedical image 235. In general, the quality metric 260 can be used to assess or evaluate the effectiveness of the brachytherapy on the feature 220 in the organ 215 targeted for treatment. In some embodiments, the quality metric 260 can be based on a set of dose-volumetric parameters in the brachytherapy. The dose-volumetric parameters can quantify a distribution of the radiation dose within the feature 220 targeted by the brachytherapy of the treatment plan. The dose-volumetric parameters can identify or include, for example, at least one of: D2cc (dose to 2 cubic centimeters of volume) metric, a D20cc (dose to 20 cubic centimeters of volume) metric, a D15cc (dose to 15 cubic centimeters of volume) metric, a DIOcc (dose to 10 cubic centimeters of volume) metric, a D5cc (dose to 5 cubic centimeters of volume) metric, a Dice (dose to 1 cubic centimeter of volume) metric, a DO. Icc (dose to 0. 1 cubic centimeters of volume) metric, or a D20% (dose to 20% of target volume) metric, among others. Which dose-volumetric parameters is used may depend on the cancer or the organ 215 associated with the cancer. The quality metric 260 may have been measured from the prior or simulated administration of the brachytherapy via the one or more catheters 230 to the feature 220 of the organ 215 in the subject 210.

[0104] The feature detector 135 can feed, apply, or otherwise provide each biomedical image 235 as an input to the image segmentation model 160. In some embodiments, the feature detector 135 can provide the set of biomedical images 235 of the video in serial (e.g., frame-by- frame) into the image segmentation model 160. The feature detector 135 can process the input biomedical image 235 in accordance with the set of weights of the image segmentation model 160. From providing the biomedical image 235 to the image segmentation model 160, the feature detector 135 can identify, determine, or otherwise detect one or more positions 250’A-N (hereinafter generally referred to as positions 250’) and at least one contour 255’. The one or more positions 250 can correspond to the one or more catheters 230 placed within the organ 215of the subject 210. The one or more positions 250’ can identify or define the corresponding one or more objects 245 within the biomedical image 235. The contour 255’ can define the SOI 240 within the biomedical image 235. The one or more positions 250’ generated by the image segmentation model 160 can be similar to the one or more positions 250 identified in the example of the training data 205. In addition, the contour 255’ generated by the image segmentation model 160 can be similar to the contour 255 identified in the example of the training data 205. In some embodiments, the feature detector 135 can produce, create, or otherwise generate at least one segmented biomedical image 235’A-N (hereinafter generally referred to as segmented biomedical image 235’) corresponding to the input biomedical image 235. Each segmented biomedical image 235’ can identify or include the one or more positions 250’ and the contour 255’.(0105] The model trainer 130 can calculate, generate, or otherwise determine at least one loss metric 270. The loss metric 270 can be based on a comparison between the one or more positions 250 as identified by the training data 205 versus the one or more positions 250’ detected using the image segmentation model 160. The loss metric 270 can be based on a comparison between the contour 255 identified by the training data 205 and the contour 255’ detected using the image segmentation model 160. In some embodiments, the comparison may be on per frame-by-frame basis for each of the biomedical images 235 of the video. The loss metric 270 can be in accordance with any number of loss functions (e.g., for the comparison between positions 250 and 250’), such as a norm loss (e.g., LI or L2), mean absolute error (MAE), mean squared error (MSE), a quadratic loss, a cross-entropy loss, a Hausdorff distance, or a Huber loss, among others. In some embodiments, the loss metric 270 can be in accordance with a similarity measure (e.g., for the comparison between the contour 255 and 255’), such as cosine similarity, Jaro distance, Jaccard index, or Dice coefficient, among others.

[0106] Using the loss metric 270, the model trainer 130 can modify or update one or more weights of the image segmentation model 160. The updating of the weights can be in accordance with a back propagation and optimization function (sometimes referred to herein as an objective function) with one or more parameters (e.g., learning rate, momentum, weightdecay, and number of iterations). The optimization function can define one or more parameters at which the weights of the image segmentation model 160 are to be updated. The optimization function can be in accordance with stochastic gradient descent, and can include, for example, an adaptive moment estimation (Adam), implicit update (ISGD), and adaptive gradient algorithm (AdaGrad), among others. The model trainer 130 can iteratively train the image segmentation model 160 until convergence. Upon convergence, the model trainer 130 can store and maintain the set of weights of the image segmentation model 160 for use in inference. In some embodiments, the model trainer 130 can train and establish the prediction model 165, upon completion of training and establishment of the image segmentation model 160.

[0107] The location evaluator 140 can feed, apply, or otherwise provide the one or more positions 250’ and the contour 255’ (or the one or more positions 250 and the contour 255 from the training data 205) as input to the prediction model 165. The location evaluator 140 can process the input one or more positions 250’ and the contour 255’ in accordance with the set of weights of the prediction model 165. Based on providing the input to the prediction model 165, the location evaluator 140 can calculate, generate, or otherwise determine at least one quality metric 260’ for the treatment plan, as defined by the one or more catheters 230 placed relative to the feature 220 in the organ 215. The quality metric 260’ determined by the prediction model 165 can be similar to the quality metric 260 identified in the example of the training data 205. In some embodiments, the quality metric 260 can be based on a set of dose-volumetric parameters in the brachytherapy. The dose-volumetric parameters can quantify a distribution of the radiation dose within the feature 220 targeted by the brachytherapy of the treatment plan. The dose-volumetric parameters can identify or include, for example, at least one of: a D2cc metric, a D20cc metric, a D15cc metric, a DlOcc metric, a D5cc metric, a Dice metric, a DO. lcc metric, or a D20% metric, among others.

[0108] The model trainer 130 can calculate, generate, or otherwise determine at least one loss metric 275. The loss metric 275 can be based on a comparison between the quality metric 260 as identified by the example of the training data 205 versus the quality metric 260’ determined using the prediction model 165. In some embodiments, the comparison may be on aper frame-by-frame basis for each of the biomedical images 235 of the video. The loss metric 275 can be in accordance with any number of loss functions, such as a norm loss (e.g., LI or L2), mean absolute error (MAE), mean squared error (MSE), a quadratic loss, a cross-entropy loss, or a Huber loss, among others.

[0109] Using the loss metric 275, the model trainer 130 can modify or update one or more weights of the prediction model 165. The updating of the weights can be in accordance with a back propagation and optimization function (sometimes referred to herein as an objective function) with one or more parameters (e g., learning rate, momentum, weight decay, and number of iterations). The optimization function can define one or more parameters at which the weights of the prediction model 165 are to be updated. The optimization function can be in accordance with stochastic gradient descent, and can include, for example, an adaptive moment estimation (Adam), implicit update (ISGD), and adaptive gradient algorithm (AdaGrad), among others. The model trainer 130 can iteratively train the prediction model 165 until convergence. Upon convergence, the model trainer 130 can store and maintain the set of weights of the prediction model 165 for use in inference.[OHO] Referring now to FIG. 11, depicted is a block diagram of a process 300 to execute machine learning (ML) models in the system 100 for determining quality of brachytherapy treatment plan. Under the process 300, at least one subject 310 can be evaluated for the administration of brachytherapy in accordance with a treatment plan. The subject 310 can be at risk of or diagnosed with cancer. The cancer can include, for example, at least one of lung cancer, brain cancer, head and neck cancer, colon cancer, rectal cancer, uterine cancer, endometrial cancer, stomach cancer, prostate cancer, ovarian cancer, cervical cancer, bladder cancer, skin cancer, or breast cancer, among others. The subject 310 may be examined by a clinician at a clinic or hospital, for the administration of brachytherapy to address the cancer. Through the interactions with the administrative device 115 to access the functionalities of the data processing system 105, the clinician can make changes to the brachytherapy treatment plan in near real-time.

[0111] The brachytherapy may be a form of radiation therapy in which a radioactive source (e.g., radioactive isotope) is placed inside or adjacent to the area to be treated for addressing the cancer in the subject 310. The brachytherapy can include, for example, a high- dose rate (HDR) brachytherapy, medium-dose rate (MDR) brachytherapy, low-dose rate (LDR) brachytherapy, or pulsed-dose rate (PDR) brachytherapy, among others. HDR brachytherapy can include dosage rates more than 12 Grays (Gy) per hour, such as 12 Gy per hour. Each administration session of HDR brachytherapy can have a time duration ranging between 1-30 minutes. MDR brachytherapy can include dosage rates between 3-12 Gy per hour. Each administration session of MDR brachytherapy can have a time duration ranging between 1-6 hours. LDR can include dosage rates between 0.4-2.0 Gy per hour. Each administration session of LDR brachytherapy can have a time duration ranging between 1 day to 6 months. PDR can include dosage rates between 0.4-2.0 Gy per hour, with discrete pulses ranging between 5 minutes to 2 hours. A radioactive isotope used for the brachytherapy can include, for example, at least one of iridium- 192, iodine-125, palladium- 103, caesium-131, caesium-137, cobalt-60, ruthenium- 106, or radium 326, among others.

[0112] The cancer can affect or be associated with at least one organ 315 (sometimes herein referred to as organ-at-risk (OAR)) in the subject 310. The organ 315 can correspond to any anatomical site associated with the cancer in the subject 310 administered with brachytherapy. The anatomical site for the organ 315 can include, for example, lung, brain, head and neck, colon, rectum, uterus, stomach, prostate, ovaries, cervix, bladder, skin, or breast, among others. In some embodiments, the organ 315 can correspond to a primary site where the cancer originated in the subject 310. In some embodiments, the organ 315 can correspond to a secondary site to which the cancer metastasized to in the subject 310 from the primary site. The organ 315 can have at least one feature 320 (sometimes herein referred to as a gross tumor volume (GTV), planning target volume (PTV), or a target) associated with the cancer. The feature 320 can include, for example, a tumor, such as soft-tissue sarcoma, bone sarcoma, or malignant fibrous histiocytoma, among others.

[0113] For the administration of brachytherapy, one or more catheters 330A-N (hereinafter generally referred to as catheters 330) can be arranged, situated, or otherwise positioned about the organ 315 or about the feature 320 in the organ 315. Any number of catheters 330 (e.g., ranging between 1 to 100) can be placed within the organ 315 of the subject 310 by a clinician managing the administration of brachytherapy to the subject 310. Each catheter 330 can include a device used to deliver the radioactive source to the feature 320 of the organ 315. The catheter 330 can include, for example, a conduit or a tube for the delivery of radioactive isotope from outside the body of the subject 310 toward the feature 320 of the organ 315. The catheter 330 can be inserted into body cavities or through the tissue or the organ 315. The number of catheters 330 positioned about the feature 320, the positioning of the catheters 330 within the organ 315, and the dosage to be administered through the catheters 330 can define the treatment plan for the delivery of brachytherapy for the subject 310.|0114| The imaging device 110 can create, produce, or otherwise generate a set of biomedical images 335A-N (hereinafter generally referred to as biomedical images 335). In some embodiments, the set of biomedical image 335 can be acquired in accordance with ultrasound imaging. The ultrasound imaging can include, for example, at least one of an abdominal ultrasound, a pelvic ultrasound, an obstetric ultrasound, a breast ultrasound, a transvaginal ultrasound, a transrectal ultrasound, a doppler ultrasound, or a musculoskeletal ultrasound, among others. To carry out the ultrasound imaging, the imaging device 110 can be arranged, positioned, or otherwise placed along an exterior skin of the subject 210 near the organ 315 or the feature 320 of the organ 315. With the placement, the imaging device 110 can execute the ultrasound imaging of the section 338 to generate the set of biomedical images 335. In some embodiments, the set of biomedical images 335 can form a video of the section 338, and each biomedical image 335 can correspond to a respective frame in the video. The number of frames in the video and by extension the number of biomedical images 335 can range between 2 to 100 frames. The sampling rate can range between 1 to 15 MHz.

[0115] Each biomedical image 335 can be of at least one section 338 in the subject 310.The section 338 can correspond to a cross-sectional region or volume of the subject 310 fromwhich the biomedical image 335 is acquired or derived. The section 338 can contain, include, or otherwise be associated with the organ 315 (or the feature 320 of the organ 315) in the subj ect 310. The biomedical image 335 can include at least one structure of interest (SOI) 340. The SOI 340 can correspond to the feature 320 within the organ 315. For instance, the SOI 340 can be a segment of the biomedical image 335 that corresponds to a depiction of the feature 320. In addition, the biomedical image 335 can have include one or more objects 345A-N (hereinafter generally referred to objects 345) corresponding the one or more catheters 330. Each object 345 can correspond to a respective catheter 330 placed within the organ 315 of the subject 310. For instance, the object 345 can be a respective portion of the biomedical image 335 that corresponds to a depiction of the respective catheter 330.

[0116] With the acquisition, the imaging device 110 can transmit, send, or otherwise provide the set of biomedical images 335 to the data processing system 105. In some embodiments, the administrative device 115 can provide the set of biomedical images 335 acquired via the imaging device 110 to the data processing system 105. The dataset indexer 145 can obtain, receive, or otherwise identify the set of biomedical images 335 from the imaging device 110 or the administrative device 115. In some embodiments, the dataset indexer 145 can parse or process the video containing the set of biomedical images 335 of the section 338 in of the subject 310. From parsing the video, the dataset indexer 145 can extract or identify the individual biomedical images 335 forming the video. Each biomedical image 335 can correspond to a frame of a set of frames for the video. In some embodiments, the dataset indexer 145 can pass a set number (e.g., 3 to 15 frames) of biomedical images 335 for further processing by the feature detector 135 and the location evaluator 140.|0117] The feature detector 135 feed, apply, or otherwise provide each biomedical image 335 as an input to the image segmentation model 160. In some embodiments, the feature detector 135 can provide the set of biomedical images 335 of the video in serial (e.g., frame-by- frame) into the image segmentation model 160. The feature detector 135 can process the input biomedical image 335 in accordance with the set of weights of the image segmentation model 160. From providing the biomedical image 335 to the image segmentation model 160, thefeature detector 135 can identify, determine, or otherwise detect one or more positions 350A-N (hereinafter generally referred to as positions 350’) and at least one contour 355. The one or more positions 350 can correspond to the one or more catheters 330 placed within the organ 315 of the subject 310. ‘

[0118] The one or more positions 350 can identify or define the corresponding one or more objects 345 within the biomedical image 335. Each position 350 can define, specify, or otherwise identify a corresponding position of a respective catheter 330 within the organ 315 within the biomedical image 335. For instance, each position 350 can define a boundary of a respective object 345 corresponding to the respective catheter 330 within the biomedical image 335. The position 350 can be defined in terms of pixel coordinates within the biomedical image 235 or coordinates in a virtual anatomical model of the organ 215 or the subject 210. In some embodiments, the one or more positions 350 can correspond to the one or more catheters 330, for each biomedical image 335 in the video. At least one of the positions 350 can be at least partially different from frame to frame across different biomedical images 335 (e.g., due to movement of the organ 315).

[0119] In addition, the contour 355 can define the SOI 340 within the biomedical image 335. The contour 355 can specify, identify, or otherwise define the SOI 340 in the biomedical image 335 corresponding to the feature 320 in the organ 315. For instance, the contour 355 can define a boundary of the SOI 340 within the biomedical image 335. The boundary can be defined in terms of pixel coordinates within the biomedical image 335 or coordinates in a virtual anatomical model of the organ 315 or the subject 310. In some embodiments, the contour 355 can correspond to the SOI 340 for each biomedical image 335 in the video. The SOI 340 can be at least partially different from frame to frame across different biomedical images 335 (e.g., due to movement of the organ 315). In some embodiments, the feature detector 135 can produce, create, or otherwise generate at least one segmented biomedical image 330’A-N (hereinafter generally referred to as segmented biomedical image 335’) from providing the corresponding biomedical image 335 to the image segmentation model 160. Each segmented biomedical image 335’ can identify or include the one or more positions 350 and the contour 355.

[0120] The location evaluator 140 can feed, apply, or otherwise provide the one or more positions 350 and the contour 355 as input to the prediction model 165. The location evaluator 140 can process the input one or more positions 350 and the contour 355 in accordance with the set of weights of the prediction model 165. Based on providing the input to the prediction model 165, the location evaluator 140 can calculate, generate, or otherwise determine at least one quality metric 360 for the treatment plan. The quality metric 360 can identify, measure, or otherwise indicate a quality of the treatment plan, as defined by the one or more catheters 330 placed relative to the feature 320 in the organ 315 and by extension, by the one or more positions 350 relative to the contour 355 in the biomedical image 335. In general, the quality metric 360 can be used to assess or evaluate the effectiveness of the brachytherapy on the feature 320 in the organ 315 targeted for treatment. In some embodiments, the quality metric 360 can be based on a set of dose-volumetric parameters in the brachytherapy. The dose-volumetric parameters can quantify a distribution of the radiation dose within the feature 320 targeted by the brachytherapy of the treatment plan. The dose-volumetric parameters can identify or include, for example, at least one of: a D2cc metric, a D20cc metric, a D15cc metric, a DIOcc metric, a D5cc metric, a Dice metric, a DO.lcc metric, or a D20% metric, among others.

[0121] Referring now to FIG. 12, depicted is a block diagram of a process 400 to package outputs in the system 100 for determining quality of brachytherapy treatment plan. Under the process 400, the output packager 150 can store and maintain an association between the treatment plan (e.g., for the one or more catheters 330) and the quality metric 360. The association may be maintained using one or more data structures or data files on the database 120. The data structures may include, for example, an array, matrix, table, linked list, binary tree, heap, stack, queue, class object, or data file, among others. The files may include, for example, extensible markup language (XML), comma-separated values (CSV), structured query language (SQL), or JavaScript Object Notation (JSON), among others. The association may be between the subject 310 or the treatment plan with at least any one or more of: the set of biomedical images 335 (or 335’), the one or more positions 350, the contour 355, or the quality metric 360, among others.

[0122] The output packager 150 can produce, create, or otherwise generate at least one output 405 based on the quality metric 360 indicating the quality of the treatment plan for the subject 310. The output 405 can include any information about the treatment plan, the quality metric 360 for the treatment plan, the set of biomedical images 335 (or 335’), or one or more catheters 330 placed relative to the feature 320 in the organ 315, among others. For example, the output package 150 can generate the output 405 to include the set of biomedical images 335’ along with the quality metric 360. The information of the output 405 can be used by a recipient (e.g., user of the administrative device 115 and clinician examining the subject 310) to accept or reject the treatment plan. With the generation, the output packager 150 can send, transmit, or otherwise provide the output 405 to the administrative device 115 for presentation.

[0123] The administrative device 115 can retrieve, obtain, or otherwise receive the output 405 from the data processing system 105. With receipt, the administrative device 115 can parse or process the output 405 to extract or identify the information about the treatment plan. The administrative device 115 can display, render, or otherwise present the information of the output 405 via at least one user interface 410. The user interface 410 can be a graphical user interface (GUI) of an application accessible via the administrative device 115. In some embodiments, the user interface 410 can include one or more elements to accept or reject the treatment plan in view of the information of the output 405. For instance, using the biomedical image 335’ and the quality metric 360 presented via the user interface 410, a user (e.g., a clinician examining the subject 310) may decide whether to proceed with the treatment plan and accept or to modify the treatment plan and reject.

[0124] Based on the interaction with the user interface 410, the administrative device 115 can produce, create, or otherwise generate at least one response 415. When the interaction is to accept the treatment plan, the administrative device 115 can generate the response 415 to indicate acceptance of the treatment plan with the catheters 330 at the current positions 350. In conjunction, the clinician examining the subject 310 can decide to provide, deliver, or administer a therapy 420 through the one or more catheters 330, as placed relative to the feature 320 of the organ 315. The therapy 420 can be administered on the feature 320 of the organ 315 in thesubject 310 to address the cancer. The therapy 420 can include injecting, implanting, or otherwise adding of the radioactive source (e.g., radioactive isotope of the brachytherapy) from outside the body of the subject 310 through the one or more catheters 330. With the addition, the radioactive source can deliver ionizing radiation toward the feature 320 (e.g., to treat and kill tumor associated with the cancer) in the organ 315.

[0125] Conversely, when the interaction is to reject the treatment plan, the administrative device 115 can generate the response 415 to indicate rejection of the treatment plan with the catheters 330 at the current positions 350. In conjunction, in view of the information of the output 405, the clinician examining the subject 310 for the administration of the brachytherapy can alter, change, or otherwise modify the treatment plan. The modification can include, for example, one or more of: a change (e.g., an increase or decrease) of the dosage provided through the one or more catheters 330; a change (e.g., an increase or decrease) in the number of catheters 330; or a change in the positioning of the catheters 330 relative to the feature 320 in the organ 315, among others. For instance, seeing that the positions 350 corresponding to the catheters 330 is at a distance from the contour 355 corresponding to the feature 320, the clinician can decide to shift or move the position of the catheters closer towards the feature 320 in the organ 315. In addition, when the information indicates that the quality metric 360 that is lower than expected by the clinician, the clinician can increase the dosage of the brachytherapy or increase the number of catheters 330. With the generation of the response 415, the administrative device 115 can return, transmit, or otherwise provide the response 415 to the data processing system 105.

[0126] The feedback handler 155 can retrieve, identify, or otherwise receive the response 415 via the user interface 410 from the administrative device 115. Upon receipt, the feedback handler 155 can process or parse the response 415 to extract or identify the indication of acceptance or rejection of the treatment plan. If the response 415 is to indicate acceptance of the treatment plan, the feedback handler 155 can store and maintain an association between the treatment plan and the indication of the acceptance of the treatment plan. The association can be stored and maintained using the data structures or files. The feedback handler 155 can alsorefrain from reacquisition of the set of biomedical image 335 and the regeneration of the quality metric 360.

[0127] On the other hand, if the response 415 is to indicate rejection of the treatment plan, the feedback handler 155 can store and maintain an association between the treatment plan and the indication of the rejection of the treatment plan. The association can be stored and maintained using the data structures or files. In addition, the feedback handler 155 can repeat the functionalities as detailed herein in conjunction with processes 300 and 400 to receive a set of biomedical images 335 and re-calculate the quality metric 360, with the modifications to the treatment plan made by the clinician in view of the information presented via the user interface 410. For example, the imaging device 110 can acquire and send a new set of biomedical images to the data processing system 105. The dataset indexer 145 can receive the new set of biomedical images. The feature detector 135 can detect a new set of positions and a new contour from the new biomedical images based on providing the new set of biomedical images to the image segmentation model 160. The location evaluator 140 can provide the new set of positions and the new contour to the prediction model 165 to determine a new quality metric. The output packager 150 can generate and provide a new output based on the new quality metric.

[0128] By using the image segmentation model 160 and the prediction model 165 in this way, the data processing system 105 can provide highly accurate, near real-time results regarding the effectiveness of the placements of the catheters 330 during the brachytherapy procedure. From a computing perspective, these more precise and accurate results can avoid wasteful consumption of computing resources (e.g., processor and memory) as well as network bandwidth due to multiple back-and-forth communications, in comparison to other less precise and accurate approaches. Furthermore, the user interface 410 can allow for repeated and iterative accessing of the functionalities of the data processing system 105 can also improve the quality of humancomputer interactions (HCI) between the user and the data processing system 105.

[0129] In addition, from a clinical perspective, since the image segmentation model 160 and the prediction model 165 are trained on previous, successful brachytherapy procedures, theoutputs regarding predicted effectiveness can be highly precise and accurate. This can reduce the reliance on subjective clinician experience and allow for objective, data-driven decisions regarding treatment plans for brachytherapy. The presentation of the information via the user interface 410 can allow clinicians to accept or modify the treatment plans based on near real-time feedback to increase and enhance the effectiveness of the brachytherapy procedure. By providing near real-time feedback on catheter placement and quality, the clinician can use the information to improve the administration of brachytherapy to better target the tumors and minimize damage to the surrounding tissues. The introduction of real-time feedback also can reduce the duration of anesthesia required during procedures. Shorter procedure times can minimize associated risks and improve patient outcomes.

[0130] Referring now to FIG. 13, depicted is a flow diagram of a method 500 of determining quality of brachytherapy treatment plan from catheter placement in organs in subjects. The method 500 can be implemented using or performed using any one or more of the components described herein, such as the system 100 or the system 600. Under the method 500, a computing system (e.g., the data processing system 105) can identify a set of biomedical images (e.g., the biomedical images 335) associated with a brachytherapy plan for a subject (e g., the subject 310) (505). The computing system can detect catheter positions (e.g., the positions 350) corresponding to catheters (e.g., the catheters 330) and a contour (e.g., the contour 355) corresponding to a structure of interest (SOI) (e.g., the SOI 340) from the biomedical image (510). The computing system can generate a quality metric (e.g., the metric 360) using the catheter position and the contour (515). The computing system can provide an output on the brachytherapy plan (520). The computing system can determine whether the brachytherapy plan is accepted or rejected (525). If accepted, brachytherapy can be administered to an organ feature (e.g., the feature 320 or the organ 315) in the subject (530). Otherwise, if rejected, the therapy plan can be modified (e.g., manually by the physician) (535) and the method 500 can be repeated from (505).D. Computing and Network Environment

[0131] Various operations described herein may be implemented on computer systems. FIG. 14 shows a simplified block diagram of a representative server system 600, client computing system 614, and network 626 usable to implement certain embodiments of the present disclosure. In various embodiments, server system 600 or similar systems may implement services or servers described herein or portions thereof. Client computing system 614 or similar systems may implement clients described herein. The system 100 described herein may be similar to the server system 600. Server system 600 may have a modular design that incorporates a number of modules 602 (e.g., blades in a blade server embodiment); while two modules 602 are shown, any number may be provided. Each module 602 may include processing unit(s) 604 and local storage 606.

[0132] Processing unit(s) 604 may include a single processor, which may have one or more cores, or multiple processors. In some embodiments, processing unit(s) 604 may include a general-purpose primary processor as well as one or more special-purpose co-processors such as graphics processors, digital signal processors, or the like. In some embodiments, some or all processing units 604 may be implemented using customized circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions that are stored on the circuit itself. In other embodiments, processing unit(s) 604 may execute instructions stored in local storage 606. Any type of processors in any combination may be included in processing unit(s) 604.[0133| Local storage 606 may include volatile storage media (e.g., DRAM, SRAM,SDRAM, or the like) and / or non-volatile storage media (e.g., magnetic or optical disk, flash memory, or the like). Storage media incorporated in local storage 606 may be fixed, removable or upgradeable as desired. Local storage 606 may be physically or logically divided into various subunits such as a system memory, a read-only memory (ROM), and a permanent storage device. The system memory may be a read-and-write memory device or a volatile read-and-write memory, such as dynamic random-access memory. The system memory may store some or all of the instructions and data that processing unit(s) 604 need at runtime. The ROM may store static data and instructions that are needed by processing unit(s) 604. The permanent storagedevice may be a non-volatile read-and-write memory device that may store instructions and data even when module 602 is powered down. The term “storage medium” as used herein includes any medium in which data may be stored indefinitely (subject to overwriting, electrical disturbance, power loss, or the like) and does not include carrier waves and transitory electronic signals propagating wirelessly or over wired connections.

[0134] In some embodiments, local storage 606 may store one or more software programs to be executed by processing unit(s) 604, such as an operating system and / or programs implementing various server functions such as functions of the system 100 of FIG. 9 or any other system described herein, or any other server(s) associated with system 100 or any other system described herein.

[0135] “Software” refers generally to sequences of instructions that, when executed by processing unit(s) 604 cause server system 600 (or portions thereof) to perform various operations, thus defining one or more specific machine embodiments that execute and perform the operations of the software programs. The instructions may be stored as firmware residing in read-only memory and / or program code stored in non-volatile storage media that may be read into volatile working memory for execution by processing unit(s) 604. Software may be implemented as a single program or a collection of separate programs or program modules that interact as desired. From local storage 606 (or non-local storage described below), processing unit(s) 604 may retrieve program instructions to execute and data to process in order to execute various operations described above.

[0136] In some server systems 600, multiple modules 602 may be interconnected via a bus or other interconnect 608, forming a local area network that supports communication between modules 602 and other components of server system 600. Interconnect 608 may be implemented using various technologies including server racks, hubs, routers, etc.

[0137] A wide area network (WAN) interface 610 may provide data communication capability between the local area network (interconnect 608) and the network 626, such as theIntemet. Technologies may be used, including wired (e.g., Ethernet, IEEE 602.3 standards) and / or wireless technologies (e.g., Wi-Fi, IEEE 602.11 standards).

[0138] In some embodiments, local storage 606 is intended to provide working memory for processing unit(s) 604, providing fast access to programs and / or data to be processed while reducing traffic on interconnect 608. Storage for larger quantities of data may be provided on the local area network by one or more mass storage subsystems 612 that may be connected to interconnect 608. Mass storage subsystem 612 may be based on magnetic, optical, semiconductor, or other data storage media. Direct attached storage, storage area networks, network-attached storage, and the like may be used. Any data stores or other collections of data described herein as being produced, consumed, or maintained by a service or server may be stored in mass storage subsystem 612. In some embodiments, additional data storage resources may be accessible via WAN interface 610 (potentially with increased latency).

[0139] Server system 600 may operate in response to requests received via WAN interface 610. For example, one of modules 602 may implement a supervisory function and assign discrete tasks to other modules 602 in response to received requests. Work allocation techniques may be used. As requests are processed, results may be returned to the requester via WAN interface 610. Such operation may generally be automated. Further, in some embodiments, WAN interface 610 may connect multiple server systems 600 to each other, providing scalable systems capable of managing high volumes of activity. Other techniques for managing server systems and server farms (collections of server systems that cooperate) may be used, including dynamic resource allocation and reallocation.

[0140] Server system 600 may interact with various user-owned or user-operated devices via a wide-area network such as the Internet. An example of a user-operated device is shown in FIG. 14 as client computing system 614. Client computing system 614 may be implemented, for example, as a consumer device such as a smartphone, other mobile phone, tablet computer, wearable computing device (e.g., smart watch, eyeglasses), desktop computer, laptop computer, and so on.

[0141] For example, client computing system 614 may communicate via WAN interface 610. Client computing system 614 may include computer components such as processing unit(s) 616, storage device 618, network interface 620, user input device 622, and user output device 624. Client computing system 614 may be a computing device implemented in a variety of form factors, such as a desktop computer, laptop computer, tablet computer, smartphone, other mobile computing device, wearable computing device, or the like.

[0142] Processing unit(s) 616 and storage device 618 may be similar to processing unit(s) 604 and local storage 606 described above. Suitable devices may be selected based on the demands to be placed on client computing system 614; for example, client computing system 614 may be implemented as a “thin” client with limited processing capability or as a high- powered computing device. Client computing system 614 may be provisioned with program code executable by processing unit(s) 616 to enable various interactions with server system 600.

[0143] Network interface 620 may provide a connection to the network 626, such as a wide area network (e.g., the Internet) to which WAN interface 610 of server system 600 is also connected. In various embodiments, network interface 620 may include a wired interface (e g., Ethernet) and / or a wireless interface implementing various RF data communication standards such as Wi-Fi, Bluetooth, or cellular data network standards (e.g., 3G, 4G, 5G, LTE, etc.).

[0144] User input device 622 may include any device (or devices) via which a user may provide signals to client computing system 614. The client computing system 614 may interpret the signals as indicative of particular user requests or information. In various embodiments, user input device 622 may include any or all of a keyboard, touch pad, touch screen, mouse or other pointing device, scroll wheel, click wheel, dial, button, switch, keypad, microphone, and so on.

[0145] User output device 624 may include any device via which client computing system 614 may provide information to a user. For example, user output device 624 may include a display to display images generated by or delivered to client computing system 614. The display may incorporate various image generation technologies, e.g., a liquid crystal display(LCD), light-emitting diode (LED) including organic light-emitting diodes (OLED), projection system, cathode ray tube (CRT), or the like, together with supporting electronics (e.g., di ital-to- analog or analog-to-digital converters, signal processors, or the like). Some embodiments may include a device such as a touchscreen that function as both input and output device. In some embodiments, other user output devices 624 may be provided in addition to or instead of a display. Examples include indicator lights, speakers, tactile “display” devices, printers, and so on.

[0146] Some embodiments include electronic components, such as microprocessors, storage and memory that store computer program instructions in a computer-readable storage medium. Many of the features described in this specification may be implemented as processes that are specified as a set of program instructions encoded on a computer-readable storage medium. When these program instructions are executed by one or more processing units, they cause the processing unit(s) to perform various operation indicated in the program instructions. Examples of program instructions or computer code include machine code, such as is produced by a compiler, and files including higher-level code that are executed by a computer, an electronic component, or a microprocessor using an interpreter. Through suitable programming, processing unit(s) 604 and 616 may provide various functionality for server system 600 and client computing system 614, including any of the functionality described herein as being performed by a server or client, or other functionality.

[0147] It will be appreciated that server system 600 and client computing system 614 are illustrative and that variations and modifications are possible. Computer systems used in connection with embodiments of the present disclosure may have other capabilities not specifically described here. Further, while server system 600 and client computing system 614 are described with reference to particular blocks, it is to be understood that these blocks are defined for convenience of description and are not intended to imply a particular physical arrangement of component parts. For instance, different blocks may be but need not be located in the same facility, in the same server rack, or on the same motherboard. Further, the blocks need not correspond to physically distinct components. Blocks may be configured to performvarious operations, e.g., by programming a processor or providing appropriate control circuitry, and various blocks might or might not be reconfigurable depending on how the initial configuration is obtained. Embodiments of the present disclosure may be realized in a variety of apparatus including electronic devices implemented using any combination of circuitry and software.

[0148] While the disclosure has been described with respect to specific embodiments, one skilled in the art will recognize that numerous modifications are possible. Embodiments of the disclosure may be realized using a variety of computer systems and communication technologies including but not limited to the specific examples described herein. Embodiments of the present disclosure may be realized using any combination of dedicated components and / or programmable processors and / or other programmable devices. The various processes described herein may be implemented on the same processor or different processors in any combination. Where components are described as being configured to perform certain operations, such configuration may be accomplished, e.g., by designing electronic circuits to perform the operation, by programming programmable electronic circuits (such as microprocessors) to perform the operation, or any combination thereof. Further, while the embodiments described above may make reference to specific hardware and software components, those skilled in the art will appreciate that different combinations of hardware and / or software components may also be used and that particular operations described as being implemented in hardware might also be implemented in software or vice versa.

[0149] Computer programs incorporating various features of the present disclosure may be encoded and stored on various computer-readable storage media; suitable media include magnetic disk or tape, optical storage media such as compact disk (CD) or DVD (digital versatile disk), flash memory, and other non-transitory media. Computer-readable media encoded with the program code may be packaged with a compatible electronic device, or the program code may be provided separately from electronic devices (e.g., via Internet download or as a separately packaged computer-readable storage medium).

[0150] Thus, although the disclosure has been described with respect to specific embodiments, it will be appreciated that the disclosure is intended to cover all modifications and equivalents within the scope of the following claims.

Claims

WHAT IS CLAIMED IS:

1. A method of determining quality of brachytherapy treatment plans from catheter placement in organs in subjects, comprising: identifying, by one or more processors, a biomedical image of a section associated with an organ of a subject to be administered with brachytherapy in accordance with a treatment plan; detecting, by the one or more processors, from the biomedical image, (i) a plurality of positions corresponding to a plurality of catheters placed within the organ and (ii) a contour defining a structure of interest (SOI) in the biomedical image corresponding to a feature within the organ; providing, by the one or more processors, the plurality of positions and the contour as input to a machine learning (ML) model, wherein the ML model is established using a plurality of examples, each example of the plurality of examples comprising (i) a respective plurality of positions corresponding to a respective plurality of catheters placed within a respective organ of a respective subject, (ii) a respective contour defining a respective SOI in a respective biomedical image corresponding to a feature in the respective organ, and (iii) a respective metric indicating a quality of a respective treatment plan for the respective plurality of catheters placed relative to the feature within the organ; generating, by the one or more processors, based on providing the plurality of positions and the contour to the ML model, a metric indicating a quality of the treatment plan for the plurality of catheters placed relative to the feature within the organ; and storing, by the one or more processors, using one or more data structures, an association between the treatment plan and the metric.

2. The method of claim 1, further comprising providing, by the one or more processors, for presentation via a user interface, an output about the plurality of catheters placed within the organ, based on the metric for the treatment plan for the subject.

3. The method of claim 2, further comprising receiving, by the one or more processors, via the user interface, an indication of acceptance of the treatment plan defining placement of the plurality of catheters at the corresponding plurality of positions in the organ.

4. The method of claim 2, further comprising administering, via the plurality of catheters, brachytherapy on the feature in the organ of the subject, subsequent to presentation of the output.

5. The method of claim 2, further comprising: receiving, by the one or more processors, via the user interface, an indication of rejection of the treatment plan defining placement of the plurality of catheters at the corresponding plurality of positions in the organ; and modifying at least one of (i) a dosage to be provided via the plurality of catheters, (ii) a number of the plurality of catheters, or (iii) a placement of the plurality of catheters of the treatment plan, subsequent to presentation of the output.

6. The method of claim 1, wherein identifying the biomedical image further comprises receiving, via an imaging device, a plurality of biomedical images of a section associated with the organ of the subject, each of the plurality of biomedical image corresponding to a respective frame of a plurality of frames.

7. The method of claim 1, wherein detecting the plurality of positions and the contour further comprises detecting, based on providing the biomedical image to an image segmentation model, the plurality of positions and the contour.

8. The method of claim 7, wherein the image segmentation model is established using the plurality of examples, each of the plurality of examples comprising (i) the respective biomedical image of a respective section associated with the respective organ of the respective subject, (ii) the respective plurality of positions, (ii) the respective contour.

9. The method of claim 1, wherein the metric indicating the quality of the treatment plan is based on a plurality of dose-volumetric parameters in brachytherapy.

10. The method of claim 9, wherein the plurality of dose-volumetric parameters includes at least one of a D2cc metric, a D20cc metric, a D15cc metric, a DIOcc metric, a D5cc metric, a Dice metric, a DO.lcc metric, or a D20% metric.

11. The method of claim 1, wherein the biomedical image is in accordance with ultrasound imaging, wherein the ultrasound imaging comprises at least one of an abdominal ultrasound, a pelvic ultrasound, an obstetric ultrasound, a breast ultrasound, a transvaginal ultrasound, a transrectal ultrasound, a doppler ultrasound, or a musculoskeletal ultrasound.

12. The method of claim 1, wherein the brachytherapy comprises at least one of a high-dose rate (HDR) brachytherapy, medium-dose rate (MDR) brachytherapy, low-dose rate (LDR) brachytherapy, or pulsed-dose rate (PDR) brachytherapy, and wherein a radioactive isotope for the brachytherapy comprises at least one of iridium- 192, iodine-125, palladium- 103, caesium-131, caesium-137, cobalt-60, ruthenium- 106, or radium 226.

13. The method of claim 1, wherein the subject is at risk of or diagnosed with cancer on the organ, and wherein the cancer comprises at least one of lung cancer, brain cancer, head and neck cancer, colon cancer, rectal cancer, uterine cancer, endometrial cancer, stomach cancer, prostate cancer, ovarian cancer, cervical cancer, bladder cancer, skin cancer, or breast cancer.

14. A system for determining quality of brachytherapy treatment plans from catheter placement in organs in subjects, comprising: one or more processors coupled with memory, configured to: identify a biomedical image of a section associated with an organ of a subject to be administered with brachytherapy in accordance with a treatment plan;detect, from the biomedical image, (i) a plurality of positions corresponding to a plurality of catheters placed within the organ and (ii) a contour defining a structure of interest (SOI) in the biomedical image corresponding to a feature within the organ; provide the plurality of positions and the contour as input to a machine learning (ML) model, wherein the ML model is established using a plurality of examples, each example of the plurality of examples comprising (i) a respective plurality of positions corresponding to a respective plurality of catheters placed within a respective organ of a respective subject, (ii) a respective contour defining a respective SOI in a respective biomedical image corresponding to a feature in the respective organ, and (iii) a respective metric indicating a quality of a respective treatment plan for the respective plurality of catheters placed relative to the feature within the organ; generate, based on providing the plurality of positions and the contour to the ML model, a metric indicating a quality of the treatment plan for the plurality of catheters placed relative to the feature within the organ; and store, using one or more data structures, an association between the treatment plan and the metric.

15. The system of claim 14, wherein the one or more processors are further configured to provide, for presentation via a user interface, an output about the plurality of catheters placed within the organ, based on the metric for the treatment plan for the subject.

16. The system of claim 15, wherein the one or more processors are further configured to receive, via the user interface, an indication of acceptance of the treatment plan defining placement of the plurality of catheters at the corresponding plurality of positions in the organ.

17. The system of claim 15, wherein the brachytherapy is administered on the feature in the organ of the subject, subsequent to presentation of the output.

18. The system of claim 15, wherein the one or more processors are further configured to receive, via the user interface, an indication of rejection of the treatment plan defining placement of the plurality of catheters at the corresponding plurality of positions in the organ; and wherein at least one of (i) a dosage to be provided via the plurality of catheters, (ii) a number of the plurality of catheters, or (iii) a placement of the plurality of catheters of the treatment plan is modified, subsequent to presentation of the output.

19. The system of claim 14, wherein the one or more processors are further configured to receive, via an imaging device, a plurality of biomedical images of a section associated with the organ of the subject, each of the plurality of biomedical image corresponding to a respective frame of a plurality of frames.

20. The system of claim 14, wherein the one or more processors are further configured to detect, based on providing the biomedical image to an image segmentation model, the plurality of positions and the contour.

21. The system of claim 20, wherein the image segmentation model is established using the plurality of examples, each of the plurality of examples comprising (i) the respective biomedical image of a respective section associated with the respective organ of the respective subject, (ii) the respective plurality of positions, (ii) the respective contour.

22. The system of claim 14, wherein the metric indicating the quality of the treatment plan is based on a plurality of dose-volumetric parameters in brachytherapy.

23. The system of claim 22, wherein the plurality of dose-volumetric parameters includes at least one of a D2cc metric, a D20cc metric, a D15cc metric, a DIOcc metric, a D5cc metric, a Dice metric, a DO.lcc metric, or a D20% metric.

24. The system of claim 14, wherein the biomedical image is in accordance with ultrasound imaging, wherein the ultrasound imaging comprises at least one of an abdominal ultrasound, a pelvic ultrasound, an obstetric ultrasound, a breast ultrasound, a transvaginal ultrasound, a transrectal ultrasound, a doppler ultrasound, or a musculoskeletal ultrasound.

25. The system of claim 14, wherein the brachytherapy comprises at least one of a high-dose rate (HDR) brachytherapy, medium-dose rate (MDR) brachytherapy, low-dose rate (LDR) brachytherapy, or pulsed-dose rate (PDR) brachytherapy, and wherein a radioactive isotope for the brachytherapy comprises at least one of iridium- 192, iodine-125, palladium-103, caesium-131, caesium-137, cobalt-60, ruthenium- 106, or radium 226.

26. The system of claim 14, wherein the subject is at risk of or diagnosed with cancer on the organ, and wherein the cancer comprises at least one of lung cancer, brain cancer, head and neck cancer, colon cancer, rectal cancer, uterine cancer, endometrial cancer, stomach cancer, prostate cancer, ovarian cancer, cervical cancer, bladder cancer, skin cancer, or breast cancer.

Images