System and method for segmentation of magnetic resonance images using deep learning with domain adaptation

Abstract

Systems and methods disclosed herein are directed to segmentation of magnetic resonance (MR) images of a subject and include receiving at least one MR image of the subject, providing the at least one MR image of the subject to a segmentation system comprising a neural network-based unsupervised domain adaptation (UDA) model, a foundation model, and a mask- guided semi-supervised (MGSS) network, generating a segmentation of a region of interest of the at least one MR image of the subject using the segmentation system and displaying the segmentation of the at least one MR image of the subject.

Description

790482.00574SYSTEM AND METHOD FOR SEGMENTATION OF MAGNETIC RESONANCE IMAGES USING DEEP LEARNING WITH DOMAIN ADAPTATIONCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on, claims priority to, and incorporates herein by reference in its entirety U. S. Serial No. 63 / 770,083 filed March 11, 2025 and entitled “System And Method For SAM-Driven MaskNet For Left Ventricle Segmentation On Cine DENSE With Unsupervised Domain Adaptation.”STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

[0002] N / ABACKGROUND

[0003] Myocardial strain has emerged as an essential biomarker for the early detection of ventricular dysfunction and plays a crucial role in diagnosing and managing various cardiovascular diseases. Strain imaging offers a more sensitive assessment of myocardial function than traditional ejection fraction measurements, which may overlook subclinical dysfunction. Consequently, accurate and reproducible quantification of myocardial strain is critical for clinical decision-making and risk stratification. Cardiovascular magnetic resonance imaging (MRI) using displacement encoding with stimulated echoes (DENSE) has proven to be a highly effective technique for strain measurement, providing superior spatial and temporal resolution to precisely track myocardial motion. Unlike other modalities, DENSE MRI directly encodes tissue displacement into the phase of the MRI signal, subsequently allowing accurate and reproducible strain computation.

[0004] Traditional myocardial strain quantification from cine DENSE MRI relies on semiautomatic segmentation, which still requires considerable human intervention. While manual contouring is the most commonly used segmentation approach and reference standard, it is labor-intensive and time-consuming, inevitably introducing inter- and intra-observer variability. As an established application, deep learning (DL)-based segmentation is particularly well suited for DENSE MRI. While previous studies have demonstrated that fully supervised DL-basedQB\ 100934698.1 1790482.00574methods using UNet are effective in automating segmentation in DENSE images, the limitations include the need for relatively large training datasets of DENSE images (typically over 50) and the manual segmentation of these images. Acquiring a large DENSE training dataset is generally challenging, as it is not yet part of routine clinical MR imaging. Additionally, creating a labeled dataset manually requires experienced annotators, yet conventional annotators often lack familiarity with DENSE images. These pose major challenges in developing robust and generalizable segmentation models. In contrast, cine steady-state free precession (SSFP) images are routinely acquired as part of standard clinical cardiac MRI protocols and are widely accessible across imaging centers. Most annotators can perform manual segmentation on cine SSFP images with sufficient familiarity. Commercial and prototypical research tools for semiautomatic and fully automatic segmentation exist. Notably, considerable multi-center, multivendor, multi-cardiac-disease and expert-labeled cine SSFP images are publicly available.

[0005] It would be desirable to provide a system and method for segmentation of magnetic resonance (MR) images that enables the use of widely available labeled MR images (e.g., cine SSFP images) for training to enhance segmentation performance for MR techniques with limited MR training images (e.g., DENSE MRI).SUMMARY

[0006] In accordance with an embodiment, a method for segmentation of magnetic resonance (MR) images of a subject includes receiving, using a processor device, at least one MR image of the subject, providing, using the processor device, the at least one MR image of the subject to a segmentation system comprising a neural network-based unsupervised domain adaptation (UDA) model, a foundation model, and a mask-guided semi-supervised (MGSS) network, generating, using the segmentation system, a segmentation of a region of interest of the at least one MR image of the subject and displaying, using a display, the segmentation of the at least one MR image of the subject.

[0007] In accordance with another embodiment, a system for generating a segmentation of magnetic resonance (MR) images of a subject includes a neural network-based unsupervised domain adaptation (UDA) model configured to transfer knowledge from a set of labeled source images to a set of unlabeled target images and generate a first segmentation of at least one MR image of the subject, a post-processing module coupled to the neural-network-based UDA modelQB\ 100934698.1 2790482.00574and configured to generate a set of prompts derived from the first segmentation, a foundation model configured to receive the set or prompts from the post-processing module and the at least one MR image of the subject and to generate a second segmentation and a mask based on the at least one MR image of the subject and the set of prompts, a partitioning module configured to receive the second segmentation and to partition the second segmentation into a pseudo-labeled set of data and an unlabeled set of data, and a mask-guided semi-supervised (MGSS) network configured to receive the mask, the pseudo-labeled set of data and the unlabeled set of data and to generate a final segmentation of the at least one MR image of the subject.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The present invention will hereafter be described with reference to the accompanying drawings, wherein like reference numerals denote like elements.

[0009] FIG. 1 is a block diagram of a system for segmentation of magnetic resonance (MR) images of a subject in accordance with an embodiment;

[0010] FIGs. 2A-2C are block diagrams of components of the system for segmentation of MR images of FIG. 1 in accordance with an embodiment;

[0011] FIG. 3 illustrates a method for generating a segmentation of an MR image in accordance with an embodiment;

[0012] FIG. 4 illustrates a method for generating strain measurements using DENSE MR images of a subject in accordance with an embodiment.

[0013] FIG. 5 is a block diagram of an example computer system in accordance with an embodiment; and

[0014] FIG. 6 is a block diagram of an example magnetic resonance imaging (MRI) system in accordance with an embodiment.DETAILED DESCRIPTION

[0015] The present disclosure describes an automated system for segmentation of MR images (e g., DENSE MR images) with a framework that includes a neural network-based unsupervised domain adaptation (UDA) model, a foundation model, and a mask-guided semi-supervised (MGSS) network. While the following description is discussed in terms of using the disclosed segmentation system and framework for DENSE MR images and cardiac imaging, it should be understood that in some embodiments the disclosed segmentation system and framework may beQB\ 100934698.1 3790482.00574used for other MR applications and techniques, for example, flow imaging, parametric mapping (e.g., Tl, T2, etc.) techniques, etc. and other anatomy, for example, the liver, brain, and vessels. Advantageously, the system for segmentation can be trained using both (1) widely available labeled MR images (e.g., cine steady-state free precession [SSFP]) and (2) unlabeled MR images (e.g., DENSE MR images) for a technique with limited training images available. While the following description is discussed in terms of using cine SSFP images, it should be understood that in some embodiments MR images that are widely available and acquired with different techniques can also be used, for example, angiography images, etc. In addition, while the following description is discussed in terms of segmentation of magnetic resonance images, it should be understood that the disclosed systems and methods for segmentation may be used with other modalities such as, for example, computed tomography (CT), ultrasound, etc. Moreover, while the following description is discussed in terms of segmentation, it should be understood that the disclosed systems and methods may be used for other image processing and analysis applications such as, for example, image enhancement (e.g., super-resolution) and image classification.

[0016] In some embodiments, the neural network (e.g., a generative adversarial network (GAN))-based UDA model and the foundation model can be used to generate a set of DENSE MR data that includes pseudo-labeled DENSE MR data and unlabeled DENSE MR data.Advantageously, the set of DENSE MR data can be generated by the neural network-based UDA model and the foundation model using both labeled cine SSFP images and DENSE MR images. UDA is a technique that enables knowledge transfer from labeled source domains (e.g., the labeled cine SSFP images) to unlabeled target domains (e.g., the unlabeled DENSE MR images), thereby mitigating the reliance on manual annotations in the target domain. By aligning feature distributions across domains, UDA improves model generalization and segmentation accuracy on unseen data. The foundation model (e.g., Segmenting Anything Model [SAM] is a machine learning or deep learning model that is trained on vast datasets so that it can be applied across a wide range of use cases. A foundation model can capture high-level semantic features with minimal supervision, achieving robust performance even in complex anatomical regions. A set of pseudo-labeled and unlabeled DENSE data generated based on a segmentation generated by the foundation model can be input into the MGSS network to generate a segmentation of an input unlabeled DENSE MR image of a region of interest (e.g., the left ventricle) of a subject. In someQB\ 100934698.1 4790482.00574embodiments, the segmentation prediction (e.g., a segmentation map) generated by the segmentation system (e.g., using the three stages of the neural network-based UDA model, the foundation model and the MGSS network) can be used for strain measurements and analysis.

[0017] The disclosed deep learning framework can address the critical challenge of limited labeled DENSE data for training. By integrating unsupervised learning, domain adaptation, a foundation model, and mask-guided self-training, the framework for the segmentation system enables accurate and reliable segmentation, for example, left ventricle (LV) segmentation, without requiring manual annotations in the target DENSE MRI domain. Rather, the disclosed framework leverages the knowledge transfer from the source domain of abundant labeled cine SSFP data. In some embodiments, the disclosed segmentation system is generalizable to DENSE MR images acquired at different sites, with MR systems from different manufacturers, etc. The foundation model can provide high-quality initial masks of the region of interest by leveraging pre-trained knowledge learned from widely available datasets, often with expert annotations, which can significantly reduce the need for a large amount of labeled DENSE MR datasets for training. The integration of foundation models with UDA enables a synergistic learning process that combines high-level semantic information from the foundation model with domain-transferred features from SSFP data.

[0018] FIG. 1 is a block diagram of a system for segmentation of DENSE magnetic resonance (MR) images of a subject in accordance with an embodiment. The system can include an input magnetic resonance (MR) image 102 of a subject, a trained segmentation system 104, an output segmentation 106, a strain analysis module 108, data storage 110 and a display 112. In some embodiments, the MR image of the subject can be an image of anatomy (e.g., the heart) acquired using a DENSE MRI technique (referred to herein as a DENSE MR image). In some embodiments, the DENSE MR image 102 of the subject can be retrieved or received from data storage (or memory) 110, data storage of an imaging system (e.g., disc storage 638 of MRI system 600 shown in FIG. 6), or data storage of other computer systems (e.g., storage device 516 of computer system 500 shown in FIG. 5). In some embodiments, the DENSE MR image 102 may be acquired in real time from the subject using an MRI system (e.g., MRI system 600 shown in FIG. 16. For example, MR data can be acquired from the subject using the DENSE MRI technique performed on the MRI system and reconstructed into a DENSE MR image. As mentioned, in some embodiments, the MR image may be acquired using otherQB\ 100934698.1 5790482.00574MR techniques (e.g., flow imaging) and the segmentation system can be trained to generate a segmentation of images acquired with other MR techniques.

[0019] The DENSE MR image 102 of the subject can be provided as an input to the trained segmentation system 104, which can be configured to generate a segmentation of a region of interest in the DENSE MR image 102 of the subject. For example, in embodiments where the DENSE MR image 102 is an image of the heart, the region of interest to be segmented can be a left ventricle (LV), a right ventricle (LV), a blood pool, etc. In some embodiments, the segmentation system 104 can be implemented as a deep learning neural network.Advantageously, the various components of the segmentation system 104 can be trained using a combination of labeled cine SSFP images and unlabeled DENSE MR images. In some embodiments, the segmentation system 104 can include a three-stage deep learning (DL) framework that utilizes a neural network-based UDA model 103, a foundation model 105, and an MGSS network 150, as discussed further below with respect to FIGs. 2A-2C.

[0020] In some embodiments, the input MR image 102 of the subject can be provided to the neural network-based UDA model 103 and the foundation model 105. The neural network-based UDA model can be configured to generate a first segmentation of the MR image 102. The first segmentation can then be provided to a post-processing module 116 which can be configured to create bounding boxes and points (as discuss further below with respect to FIG. 2A). the bounding boxes and points can be used as prompts for the foundation model 105 (as discussed further below with respect to FIG. 2B). The foundation model 105 can be configured to generate a second segmentation and mask(s) of the MR image 102 using the bounding boxes and points and prompts. The second segmentation can be provided to a partitioning module 138 which can be configured to partition the second segmentation into a pseudo labeled set of data and a labeled set of data. The mask(s), pseudo-labeled set of data and the unlabeled set of data can then be provided to the MGSS network 150. The MGSS network 150 can be configured to generate a final segmentation of the MR image 102 as an output 106 of the segmentation system 104. The segmentation 106 may be, for example, one or more segmentation maps, contours, etc. of the region of interest (e.g., the left ventricle, the right ventricle, blood pool, etc.). The output segmentation 106 may be stored in data storage, for example, data storage 110 (e.g., device storage 516 of computer system 500 shown in FIG. 5). The output 106 segmentation may also be displayed on a display 112 (e.g., display 518 of the computer system 500 shown in FIG. 5).QB\ 100934698.1 6790482.00574

[0021] The output segmentation 106 may also be provided to a strain analysis module 108 which can be configured to perform strain quantification and measurements such as, for example, global and regional circumferential Eccstrain values, radial, and longitudinal strain. In some embodiments, the strain analysis module 108 may be configured to perform various processes including, for example, phase unwrapping, tissue tracking, and strain computation. For example, for a segmentation of the left ventricle (LV), a spatiotemporal phase-unwrapping algorithm can be applied to the LV segmentation 106 to convert the LV myocardial phase data into 2D Eulerian displacement fields, and tissue tracking and temporal trajectory fitting can subsequently be applied to yield Lagrangian displacement and strain measurements. In some embodiments, the Lagrangian strain can then be decomposed into a circumferential direction (Ecc), relative to the LV center of mass. Global and regional strain analyses can be performed at each cardiac phase. In some embodiments, regional strain analysis can be conducted using the 17-segment model recommended by the American Heart Association, focusing on six mid-ventricular short-axis segments: anterior, anterolateral, inferolateral, inferior, inferoseptal, and anteroseptal. In some embodiments, the strain analysis module can be configured to generate displacement maps, strain maps, and strain-time curves. The strain quantification and measurements generated by the strain analysis module 108 may be stored in data storage, for example, data storage 110 (e.g., device storage 516 of computer system 500 shown in FIG. 5. The strain quantification and measurements may also be displayed on a display 112 (e.g., display 518 of the computer system 500 shown in FIG. 5).

[0022] In some embodiments, the segmentation system 104 (including the neural network-based UDA model 103, the foundation model 105, the MGSS model 150, the post-processing module 116, and the partitioning module 138), strain analysis module 108, and data storage 110 may be implemented on one or more processors (or processor devices) of a computer system such as, for example, any general purpose computer system or device, such as a personal computer, workstation, cellular telephone, smartphone, laptop, tablet, or the like. As such, the computer system may include any suitable hardware and components designed or capable of carrying out a variety of processing and control tasks, including, for example, steps for implementing the segmentation system 104 (including the neural network-based UDA model 103, the foundation model 105, the MGSS model 107, the post-processing module 116, and the partitioning module 138), implementing the strain analysis module 108, and receiving one or more inputs including,QB\ 100934698.1 7790482.00574for example, a DENSE MR image of a subject 102, training data, etc. For example, the computer system may include a programmable processor or combination of programmable processors, such as central processing units (CPUs), graphic processing units (GPUs), and the like. In some implementations, the one or more processors of the computer system may be configured to execute instructions stored in a non-transitory computer readable-media. In this regard, the computer system may be any device or system designed to integrate a variety of software, hardware, capabilities and functionalities. Alternatively, and by way of particular configurations and programming, the computer system may be a special purpose system or device. For example, such special-purpose system or device may include one or more dedicated processing units or modules that may be configured (e.g., hardwired, or pre-programmed) to carry pout steps, in accordance with aspects of the present disclosure.

[0023] As mentioned, the segmentation system 104 can include a three-stage deep learning (DL) framework that utilizes a neural network-based UDA model 103, a foundation model 105, and a MGSS network 150. FIGs. 2A-2C are block diagrams of components of the system for segmentation of MR images of FIG. 1 in accordance with an embodiment. FIG. 2A illustrates a neural network-based UDA model 203. The neural network-based UDA model 203 can be trained using a set of labeled cine SSFP source images 206, 222 (xSSFP,ySSFP) and a set of unlabeled DENSE MR target images (xDENSE) 208 of the desired region of interest (e.g., an LV, a RV,) for segmentation. The neural network-based UDA model 203 advantageously utilizes UDA to transfer anatomy-aware representations from the abundantly available labeled cine SSFP source images 206 to the unlabeled DENSE target images 208. Once trained, the neural networkbased UDA model 203 can be used to generate a first (or initial) segmentations (PGAN) 214 on the unlabeled target DENSE images 208 (e.g., an input DENSE MR image of a subject 102 shown in FIG. 1). A post-processing module 216 can be configured to use the first segmentation 214 of the unlabeled DENSE images 208 to derive bounding boxes 218 and point 220 prompts.

[0024] In some embodiments, as illustrated in FIG. 2A, the neural network-based UDA model 203 can be implemented as a generative adversarial network (GAN) that can be configured to align feature distributions between the source (cine SSFP) 206 and target (DENSE) 208 domains. The neural network-based UDA model 203 can include a segmentation network or segmentor (1 / 1) 210 and an adversarial discriminator (D1) 212. In some embodiments, the segmentation network (U1) 210 can be based on the classic UNet architecture. In some embodiments, theQB\ 100934698.1 8790482.00574segmentation network (U1) 210 can be trained to generate anatomically accurate segmentation predictions 224, 214 for both source 206 and target 208 domain images, respectively. To enforce domain-invariant segmentation predictions, the adversarial discriminator (D1) 212 can be introduced to distinguish whether an input originates from the source ground truths (ysspp) or the network’s prediction 224. In one example, the discriminator (D1) 212 can adopt a convolutional architecture consisting of seven stacked convolutional layers followed by batch normalization and LeakyReLU activation according to the discriminator structure in the deep convolutional GAN. The training objective for the segmentation network (U1) can combine a supervised segmentation loss 226 on the source domain 206 with an adversarial loss 228 on the target domain 208. In some embodiments, the supervised segmentation loss 226 can incorporate both pixel-wise cross-entropy (CE) loss and soft Dice loss, promoting accurate predictions at both regional and boundary levels.

[0025] In one example, the overall training objectives can be defined as follows:LLU1= Ls_sup+ Ex~D[log D1(U1(xDENSE))] (1) where the supervised segmentation loss 226 is Ls_sup= LCE(U1(xSSFP),ySSFP) + LLDice(U1(xSSFP),ySSFP), and Ex~Ddenotes the expectation over the target domain DENSE data 208. The adversarial component encourages the segmentor (1 / 1) 210 to produce target-domain predictions that are indistinguishable from those in the source domain, thus promoting cross-domain generalization. The adversarial loss 228 for the discriminator LD1can be given by:LD1= -Ex~D[log D1(U1(xSSFP))] - Ex~D[log(1 - D1(U1(xDENSE)))] (2)

[0026] FIG. 2B illustrates an example foundation model 205 that can be used as the second stage of the segmentation system. The foundation model 205 can be employed to produce high-resolution anatomical masks 232. In some embodiments, the foundation model 205 can be a known foundation model such as, for example, the Segment Anything Model (SAM). In addition, in some embodiments, the trained foundation model 205 can be implemented without any fine-tuning. In general, foundation models can be trained on a massive and diverse corpus of natural images and can generate object-level segmentations from various forms of input prompts, including bounding boxes, points, and coarse masks. The foundation model 205 can receive the prompts 218 and 220 derived from the first segmentation (pGAN) 214, and use the prompts 218,QB\ 100934698.1 9790482.00574220 to generate masks (mFM) 232, for example, LV masks, and a second segmentation prediction PFM) 230 for the unlabeled DENSE images 208. In some embodiments, two prompt pathways can be used. In a first pathway, both a bounding box 218 and a central point 220 derive from the first segmentation 214 can be provided to a prompt encoder 234 of the foundation model 205 and used to guide the foundation model 205 to segment the region of interest (e.g., the LV myocardium), producing the second segmentation (PSAM) 230. In the second pathway, the prompt encoder 234 can provide the bounding boxes 218 to a mask encoder 236 and then only the bounding box 218 is used by mask encoder 236 of the foundation model 205 to, for example, extract the epicardial contour and generate a mask (mFM") 232, for example, epicardial LV masks. The foundation model 205 can refine the prompts 218 to generate high-resolution masks that can exhibit strong alignment with anatomical boundaries and segmentation predictions.

[0027] In some embodiments, a partitioning module 238 can be configured to partition the second segmentations 230 based on a criterion into a pseudo-labeled set of data 242 and an unlabeled set of data 244. Partitioning the second segmentations 230 can enable selective supervision. In some embodiments, the partitioning criteria used to select pseudo-labels can be a Dice similarity coefficient (DSC) between pGAN214 and PSAM 230, partitioning the DENSE data 208 into pseudo-labeled (DPLDENSE) 242 and unlabeled (DULDENSE) 244 sets of training data. In some embodiments, other metrics can be used for partitioning the second segmentation 230. In some embodiments, to assess the reliability of each generated prediction, the partitioning criteria can be a confidence metric based on the DSC score between the first segmentation (PGAN) 214 and the second segmentation (pFM) 230. For example, if the overlap between the two predictions exceeds a predefined threshold T of 0.7, the case can be classified as a high-confidence sample and included in the pseudo-labeled subset DPLDENSE242. Otherwise, it can be assigned to the unlabeled subset DULDENSE244 for subsequent self-training.

[0028] In some embodiments, the pseudo-labeled set of data 242, the unlabeled set of data 244, and the masks 232 can be provided to the mask-guided semi-supervised network (MGSS) 250. FIG. 2C illustrates an example MGSS network 250. In some embodiments, the MGSS network 250 can be implemented as a UNet-based mean-teacher model / architecture that can incorporate the masks 232 generated by the foundation model 205 (shown in FIG. 2B), and once trained, can generate a final segmentation 266 (e.g. segmentation 106 shown in FIG. 1) for an input unlabeled DENSE MR image (e.g., image 102 shown in FIG. 1. The masks 232 can be provided as an inputQB\ 100934698.1 10790482.00574to the MGSS network 250, and can also be integrated into an encoder to provide encoder guidance and enhance contour feature extraction and improve segmentation precision. As mentioned, the pseudo-labeled set of data 242 and the unlabeled set of data 244 are also input to the MGSS network 250.

[0029] In FIG. 2C, the illustrated architecture for the MGSS network 250 can adopt a studentteacher design. In some embodiments, both the student network 252 and the teacher network 254 can share the same UNet-based backbone. The mask 232 generated with the foundation model 205, as described above with respect to FIG. 2B, can be concatenated as an additional input channel to both the student network 252 and the teacher network 254 to provide spatial guidance. To further enrich the semantic representation, a feature-level shortcut connection can be introduced by injecting the masks 232 into the final layer of the encoder to modulate the extracted high-level features for decoding the segmentation mask output. This dual incorporation can allow the network 250 to exploit both low-level localization and high-level anatomical structure cues. During training, in some embodiments, only the student network 252 is updated via backpropagation, while the weights of the teacher network 254 can be updated using an exponential moving average of the student network. In some embodiments, the overall training loss can combine a supervised segmentation loss 258 on the pseudo-labeled DENSE data 242 with a consistency loss 260 between the student and teacher predictions 256, 262, respectively, on the unlabeled DENSE data 244 and can be given by:L_MGSS = L_pl_sup+ Lconsis(3)The supervised loss can be defined as:Lpl sup ^CE (PS_PL’ PG AN} + LD ice (p S_PL’ PG AN} 0) where ps_PLdenotes the student’s predictions 256 on the pseudo-labeled DENSE image (xDENSE) 242, using the corresponding foundation model generated mask (mFM_PL) 232. For the unlabeled DENSE data 244, the consistency loss (Lconsis260 can encourage prediction stability between the student network 252 and teacher network 254 under network perturbations. This can be achieved through minimizing the L2 difference between their predictions, ensuring that both networks 252, 254 generate similar predictions despite variations in initialization and training dynamics.

[0030] In some embodiments, the MGSS network 250 can be trained in a semi-supervised fashion using the pseudo-labeled 242 and unlabeled 244 DENSE data. Optimization can beQB\ 100934698.1 11790482.00574performed using, for example, the Adam optimizer with an initial learning rate of 0.001 and a batch size of 8. In some embodiments, a polynomial decay learning rate schedule can be adopted to facilitate stable convergence, following the formula lt= l0(1 — t / T)p, where l0= 0.001, totaltraining steps T = 200, and decay power p = 0.9. As mentioned, once trained, the MGSS 250 can be used to generate a final segmentation for the input MR image of a subject (e.g., input MR image 102 shown in FIG. 1) based on the masks 232, the pseudo-labeled set of data 242 and the unlabeled set of data provided from the foundation model 205.

[0031] The framework illustrated in FIGs. 2A-2C uses a multi-stage design to provide complimentary supervision signals, enabling more stable optimization and improved generalization under cross-dataset shifts. Combining domain alignment with mask-guided refinement can offer greater robustness than prior UDA models.

[0032] In some embodiments, various elements and components of the system and process illustrated in FIGs. 2A-2C may be implemented on one or more processors (or processor devices) of a computer system such as, for example, any general purpose computer system or device, such as a personal computer, workstation, cellular telephone, smartphone, laptop, tablet, or the like. As such, the computer system may include any suitable hardware and components designed or capable of carrying out a variety of processing and control tasks, including, for example, steps for implementing and training the neural network-based UDA model 203, accessing and or implementing the foundation model 205, implementing and training the MGSS network 250, and receiving one or more inputs including, for example, training images, an MR image for segmentation, etc. For example, the computer system may include a programmable processor or combination of programmable processors, such as central processing units (CPUs), graphic processing units (GPUs), and the like. In some implementations, the one or more processors of the computer system may be configured to execute instructions stored in a non-transitory computer readable-media. In this regard, the computer system may be any device or system designed to integrate a variety of software, hardware, capabilities and functionalities. Alternatively, and by way of particular configurations and programming, the computer system may be a special purpose system or device. For example, such special-purpose system or device may include one or more dedicated processing units or modules that may be configured (e.g., hardwired, or pre-programmed) to carry pout steps, in accordance with aspects of the present disclosure.QB\ 100934698.1 12790482.00574

[0033] FIG. 3 illustrates a method for generating a segmentation for an MR image in accordance with an embodiment. The process illustrated in FIG. 3 is described as being carries out -by the system as illustrated in FIGs. 1 and 2A-2C. Although the blocks of the process are illustrated in a particular order, in some embodiments, one or more blocks may be executed in a different order than illustrated in FIG. 3, or may be bypassed. The process may be implemented by a processing system including at least one electronic processor, where the at least one electronic processor may be or include a processor as previously describes (e.g., including one or more individual processor devices) or as described further below with respect to FIGs. 5 and 6.

[0034] At block 302, an MR image 102, 208 of a subject can be retrieved. In some embodiments, the MR image of the subject can be an image of anatomy (e.g., the heart) acquired using a DENSE MRI technique (referred to herein as a DENSE MR image). In some embodiments, the DENSE MR image 102 of the subject can be retrieved or received from data storage (or memory) 110, data storage of an imaging system (e.g., disc storage 738 of MRI system 700 shown in FIG. 7), or data storage of other computer systems (e.g., storage device 616 of computer system 600 shown in FIG. 6). In some embodiments, the DENSE MR image 102 may be acquired in real time from the subject using an MRI system (e.g., MRI system 600 shown in FIG. 6). At block 304, the MR image can be provided to a trained neural network-based UDA model 103,203. As mentioned, in some embodiments, the neural networkbased UDA network 103, 203 can be trained using the set of unlabeled DENSE MR target images 208 and a set of labeled cine SSFP source images 206, 222 of the desired region of interest (e.g., an LV, aRV,) for segmentation. The neural network-based UDA model 103, 203 advantageously utilizes UDA to transfer anatomy-aware representations from the abundantly available labeled cine SSFP source images 206 to the unlabeled DENSE target images 208. In some embodiments, the neural network-based UDA model 103, 203 can be implemented as a generative adversarial network (GAN).

[0035] At block 306, a first (or initial) segmentation 214 can be generated for the input MR image of the subject trained neural network-based UDA model 103, 203. For example, in embodiments where the DENSE MR image 102, 208 is an image of the heart, the region of interest to be segmented can be a left ventricle (LV), a right ventricle (LV), a blood pool, etc. At block 308, a set of bounding box 218 and point 220 prompts can be created (e.g., using a post-processing module 216) based on the first segmentations 214 for the input DENSE MRQB\ 100934698.1 13790482.00574image. At block 310, the input DENSE MR images 102, 208 and the set of bounding box 218 and point 220 prompts can be provided to a trained foundation model 105, 205. As mentioned, the foundation model 105, 205 can be pretrained and implemented without any fine tuning. In some embodiments, the foundation model 105, 205 can be a known foundation model such as, for example, the Segment Anything Model (SAM). At block 312, a second segmentation 230 and a mask 232 can be generated for the input DENSE MR image 102, 208 using the trained foundation model 105, 205. As mentioned, in some embodiments, the bounding box 218 and point 220 prompts can be used to guide the foundation model 105, 205 to segment the region of interest (e.g., the LV myocardium) to produce the second segmentation 230. In some embodiments, the bounding box 218 only can be provided to the foundation model 105, 205 to allow the foundation model 105, 205 to generate a mask 232.

[0036] At block 314, the second segmentations 230 generated by the foundation model 105, 205 can be partitioned (e.g., using a partitioning module 240) into a pseudo-labeled set of data 242 and an unlabeled set of data 244. As mentioned, partitioning the second segmentations 230 to a pseudo-labeled set of data 242 and an unlabeled set of data 244 can enable selective supervision for the MGSS network 150, 250. In some embodiments, a Dice similarity coefficient (DSC) can be used as the criteria for partitioning the second segmentations 230, for example, a DSC score between the first segmentation 214 and the second segmentation 230. In one example, if the overlap between the two predictions 214, 230 exceeds a predefined threshold T, the case can be classified as a high-confidence sample and included in the pseudo-labeled subset 242. Otherwise, the case can be assigned to the unlabeled subset 244.

[0037] At block 316, a MGSS network 150, 250 can be provided with the pseudo-labeled set of data 242, the unlabeled set of data 244, and the masks 232. In some embodiments, the MGSS network 150, 250 can be implemented as a UNet-based mean-teacher model / architecture that can incorporate the masks 232 generated by the foundation model 204. At block 318, a final segmentation 106, 266 for the input MR image 102, 208 can be generated using the MGSS network 150, 250 based on the pseudo-labeled set of data 242, the unlabeled set of data 244, and the masks 232. The segmentation 106, 266 may be, for example, one or more segmentation maps, contours, etc. of the region of interest (e.g., the left ventricle, the right ventricle, blood pool, etc.). In some embodiments, the generated segmentation 106, 266 can be stored, for example, in data storage 110 (e.g., device storage 516 of computer system 500 shown in FIG. 5).QB\ 100934698.1 14790482.00574The output 106, 266 segmentation may also be displayed on a display 112 (e g., display 518 of the computer system 500 shown in FIG. 5).

[0038] FIG. 4 illustrates a method for generating strain measurements using DENSE MR images of a subject in accordance with an embodiment. The process illustrated in FIG. 4 is described as being carried out by the system as illustrated in FIG. 1. Although the blocks of the process are illustrated in a particular order, in some embodiments, one or more blocks may be executed in a different order than illustrated in FIG. 4, or may be bypassed. The process may be implemented by a processing system including at least one electronic processor, where the at least one electronic processor may be or include a processor as previously described, (e.g., including one or more individual processor devices) or as described further below with respect to FIGs. 5 and 6.

[0039] At block 402, an MR image 102 of a subject can be retrieved. In some embodiments, the MR image of the subject can be an image of anatomy (e.g., the heart) acquired using a DENSE MRI technique (referred to herein as a DENSE MR image). In some embodiments, the DENSE MR image 102 of the subject can be retrieved or received from data storage (or memory) 110, data storage of an imaging system (e.g., disc storage 638 of MRI system 600 shown in FIG. 6), or data storage of other computer systems (e.g., storage device 516 of computer system 500 shown in FIG. 5). In some embodiments, the DENSE MR image 102 may be acquired in real time from the subject using an MRI system (e.g., MRI system 600 shown in FIG. 6).

[0040] At block 404, the DENSE MR image 102 of the region of interest of the subject can be provided to a trained segmentation system 104 with a three-stage deep learning framework that utilizes a neural network-based UDA model 103, a foundation model 105, and a MGSS network 150, which can be configured to generate a segmentation of a region of interest in the DENSE MR image 102 of the subject. For example, in embodiments where the DENSE MR image 102 is an image of the heart, the region of interest to be segmented can be a left ventricle (LV), a right ventricle (LV), a blood pool, etc. In some embodiments, the mask-guided segmentation network with domain adaptation 104 can be implemented as a deep learning neural network. As mentioned, the components of the segmentation system 104 can advantageously be trained using a combination of labeled cine SSFP images and unlabeled DENSE MR images.QB\ 100934698.1 15790482.00574

[0041] At block 406, a segmentation 106 of a region of interest in the DENSE MR image 102 of the subject can generated automatically using the trained segmentation system 104. The segmentation 106 may be, for example, one or more segmentation maps, contours, etc. of the region of interest (e.g., the left ventricle, the right ventricle, blood pool, etc.). At block 408, one or more strain measurements can be determined based on the generated segmentation 106 using, for example, a strain analysis module 108. The strain measurements (or quantification) can include, for example, global and regional circumferential Eccstrain values, radial, and longitudinal strain. In some embodiments, the strain analysis module 108 may be configured to perform various processes including, for example, phase unwrapping, tissue tracking, and strain computation. In some embodiments, the strain measurements can be determined using known strain analysis methods. At block 410, the one or more strain measurements can be stored, for example, in data storage 110 (e.g., device storage 516 of computer system 500 shown in FIG. 5). In some embodiments, the one or more strain measurements can be displayed, for example, on a display 112.

[0042] FIG. 5 is a block diagram of an example computer system in accordance with an embodiment. Computer system 500 may be used to implement the systems and methods described herein. In some embodiments, the computer system 500 may be a workstation, a notebook computer, a tablet device, a mobile device, a multimedia device, a network server, a mainframe, one or more controllers, one or more microcontrollers, or any other general-purpose or application-specific computing device. The computer system 500 may operate autonomously or semi-autonomously, or may read executable software instructions from the memory or storage device 516 or a computer-readable medium (e.g., a hard drive, a CD-ROM, flash memory), or may receive instructions via the input device 520 from a user, or any other source logically connected to a computer or device, such as another networked computer or server. Thus, in some embodiments, the computer system 500 can also include any suitable device for reading computer-readable storage media.

[0043] Data, such as data acquired with an imaging system (e.g., a CT imaging system) may be provided to the computer system 500 from a data storage device 516, and these data are received in a processing unit 502. In some embodiment, the processing unit 502 includes one or more processors. For example, the processing unit 502 may include one or more of a digital signal processor (DSP) 504, a microprocessor unit (MPU) 506, and a graphics processing unit (GPU)QB\ 100934698.1 16790482.00574508. The processing unit 502 also includes a data acquisition unit 510 that is configured to electronically receive data to be processed. The DSP 504, MPU 506, GPU 508, and data acquisition unit 510 are all coupled to a communication bus 512. The communication bus 512 may be, for example, a group of wires, or a hardware used for switching data between the peripherals or between any component in the processing unit 502.

[0044] The processing unit 502 may also include a communication port 514 in electronic communication with other devices, which may include a storage device 516, a display 518, and one or more input devices 520. Examples of an input device 520 include, but are not limited to, a keyboard, a mouse, and a touch screen through which a user can provide an input. The storage device 516 may be configured to store data, which may include data such as, for example, acquired MR data, MR images, segmentations, masks, bounding boxes, points, strain measurements, machine learning models, etc., whether these data are provided to, or processed by, the processing unit 502. The display 518 may be used to display images and other information, such as CT images, patient health data, and so on.

[0045] The processing unit 502 can also be in electronic communication with a network 522 to transmit and receive data and other information. The communication port 514 can also be coupled to the processing unit 502 through a switched central resource, for example the communication bus 512. The processing unit can also include temporary storage 524 and a display controller 526. The temporary storage 524 is configured to store temporary information. For example, the temporary storage 524 can be a random access memory.

[0046] FIG. 6 is a block diagram of an example magnetic resonance imaging (MRI) system in accordance with an embodiment. MRI system 600 that may incorporate an apparatus for mitigation of RF emission described herein and may be used to perform the methods described herein. In some embodiments, the disclosed systems and methods may be designed to accompany the MRI system 600. The MRI system 600 includes an operator workstation 602, which may include a display 604, one or more input devices 606 (e.g., a keyboard and mouse), and a processor 608. The processor 608 may include a commercially available programmable machine running a commercially available operating system. The operator workstation 602 provides the operator interface that facilitates entering scan parameters (e.g., a scan prescription) into the MRI system 600. The operator workstation 602 may be coupled to different servers, including, for example, a pulse sequence server 610, a data acquisition server 612, a dataQB\ 100934698.1 17790482.00574processing server 614, and a data store server 616. The operator workstation 602 and the servers 610, 612, 614, and 616 may be connected via a communication system 640, which may include any suitable network connection, whether wired, wireless, or a combination of both.

[0047] The pulse sequence server 610 functions in response to instructions provided by the operator workstation 602 to operate a gradient system 618 and a radiofrequency (“RF”) system 620. Gradient waveforms for performing a prescribed scan are produced and applied to the gradient system 618, which excites gradient coils in an assembly 622 to produce the magnetic field gradients Gx, Gy, and Gzthat are used for spatially encoding magnetic resonance signals. The gradient coil assembly 622 forms part of a magnet assembly 624 that includes a polarizing magnet 626 and a whole-body RF coil 628 and / or a local coil (not shown). In some embodiments, the magnet assembly 624 may also include an RF shield (not shown) that can be positioned around the RF coil 628.

[0048] RF waveforms are applied by the RF system 620 to the RF coil 628, or a separate local coil, to perform the prescribed magnetic resonance pulse sequence. Responsive magnetic resonance signals detected by the RF coil 628, or a separate local coil, are received by the RF system 620. The responsive magnetic resonance signals may be amplified, demodulated, filtered, and digitized under direction of commands produced by the pulse sequence server 610. The RF system 620 includes an RF transmitter for producing a wide variety of RF pulses used in MRI pulse sequences. The RF transmitter is responsive to the prescribed scan and direction from the pulse sequence server 610 to produce RF pulses of the desired frequency, phase, and pulse amplitude waveform. The generated RF pulses may be applied to the whole-body RF coil 628 or to one or more local coils or coil arrays.

[0049] The RF system 620 also includes one or more RF receiver channels. Each RF receiver channel includes an RF preamplifier that amplifies the magnetic resonance signal received by the coil 628 to which it is connected, and a detector that detects and digitizes the I and Q quadrature components of the received magnetic resonance signal. The magnitude of the received magnetic resonance signal may, therefore, be determined at any sampled point by the square root of the sum of the squares of the I and Q components:M = √(I2+ Q2) (5)and the phase of the received magnetic resonance signal may also be determined according to the following relationship:QB\ 100934698.1 18790482.00574φ = tan-1(Q / I) (6)

[0050] The pulse sequence server 610 may receive patient data from a physiological acquisition controller 630. By way of example, the physiological acquisition controller 630 may receive signals from a number of different sensors connected to the patient, such as electrocardiograph (“ECG”) signals from electrodes, or respiratory signals from a respiratory bellows or other respiratory monitoring device. Such signals are typically used by the pulse sequence server 610 to synchronize, or “gate,” the performance of the scan with the subject’s heartbeat or respiration.

[0051] The pulse sequence server 610 may also connect to a scan room interface circuit 632 that receives signals from various sensors associated with the condition of the patient and the magnet system. Through the scan room interface circuit 632, a patient positioning system 634 can receive commands to move the patient to desired positions during the scan.

[0052] The digitized magnetic resonance signal samples produced by the RF system 620 are received by the data acquisition server 612. The data acquisition server 612 operates in response to instructions downloaded from the operator workstation 602 to receive the real-time magnetic resonance data and provide buffer storage, such that no data is lost by data overrun. In some scans, the data acquisition server 612 passes the acquired magnetic resonance data to the data processor server 614. In scans that require information derived from acquired magnetic resonance data to control the further performance of the scan, the data acquisition server 612 may be programmed to produce such information and convey it to the pulse sequence server 610. For example, during pre-scans, magnetic resonance data may be acquired and used to calibrate the pulse sequence performed by the pulse sequence server 610. As another example, navigator signals may be acquired and used to adjust the operating parameters of the RF system 620 or the gradient system 618, or to control the view order in which k-space is sampled. In still another example, the data acquisition server 612 may also process magnetic resonance signals used to detect the arrival of a contrast agent in a magnetic resonance angiography (“MRA”) scan. For example, the data acquisition server 612 may acquire magnetic resonance data and process it in real-time to produce information that is used to control the scan.

[0053] The data processing server 614 receives magnetic resonance data from the data acquisition server 612 and processes it in accordance with instructions downloaded from the operator workstation 602. Such processing may include, for example, reconstructing two-dimensional or three-dimensional images by performing a Fourier transformation of raw k-spaceQB\ 100934698.1 19790482.00574data, performing other image reconstruction algorithms (e.g., iterative or back-projection reconstruction algorithms), applying filters to raw k-space data or to reconstructed images, generating functional magnetic resonance images, or calculating motion or flow images.

[0054] Images reconstructed by the data processing server 614 are conveyed back to the operator workstation 602 for storage. Real-time images may be stored in a database memory cache (not shown in FIG. 6), from which they may be output to operator display 604 or a display 636. Batch mode images or selected real time images may be stored in a host database on disc storage 638. When such images have been reconstructed and transferred to storage, the data processing server 614 notifies the data store server 616 on the operator workstation 602. The operator workstation 602 may be used by an operator to archive the images, produce films, send the images via a network to other facilities, or post-processing of the acquired MR data or reconstructed images.

[0055] The MRI system 600 may also include one or more networked workstations 642. By way of example, a networked workstation 642 may include a display 644, one or more input devices 646 (e.g., a keyboard and mouse), and a processor 648. The networked workstation 642 may be located within the same facility as the operator workstation 602, or in a different facility, such as a different healthcare institution or clinic.

[0056] The networked workstation 642 may gain remote access to the data processing server 614 or data store server 616 via the communication system 640. Accordingly, multiple networked workstations 642 may have access to the data processing server 614 and the data store server 616. In this manner, magnetic resonance data, reconstructed images, or other data may be exchanged between the data processing server 614 or the data store server 616 and the networked workstations 642, such that the data or images may be remotely processed by a networked workstation 642. This data may be exchanged in any suitable format, such as in accordance with the transmission control protocol (TCP), the internet protocol (IP), or other known or suitable protocols.

[0057] Computer-executable instructions for performing processes according to the abovedescribed methods may be stored on a form of computer readable media. Computer readable media includes volatile and nonvolatile, removable, and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer readable media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), electrically erasableQB\ 100934698.1 20790482.00574programmable ROM (EEPROM), flash memory or other memory technology, compact disk ROM (CD-ROM), digital volatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired instructions and which may be accessed by a system (e.g., a computer), including by internet or other computer network form of access.

[0058] The present disclosure has been described in terms of one or more preferred embodiments, and it should be appreciated that many equivalents, alternatives, variations, and modifications, aside from those expressly stated, are possible and within the scope of the invention.QB\100934698.1 21

Claims

790482.00574CLAIMS:

1. A method for segmentation of magnetic resonance (MR) images of a subject, the method comprising:receiving, using a processor device, at least one MR image of the subject; providing, using the processor device, the at least one MR image of the subject to a segmentation system comprising a neural network-based unsupervised domain adaptation (UDA) model, a foundation model, and a mask-guided semi-supervised (MGSS) network;generating, using the segmentation system, a segmentation of a region of interest of the at least one MR image of the subject; anddisplaying, using a display, the segmentation of the at least one MR image of the subject.

2. The method according to claim 1, wherein the at least one MR image of the subject is a DENSE MR image of the subject.

3. The method according to claim 1, wherein generating, using the segmentation system a segmentation of a region of interest of the at least one MR image of the subject comprises:providing, using the segmentation system, the at least one MR image of the subject to the neural network-based UDA model; andgenerating, using the neural network-based USA model, a first segmentation of the at least one MR image of the subject; andgenerating, using a post-processing module, a set of prompts based on the first segmentation.

4. The method according to claim 3, wherein generating, using the segmentation system a segmentation of a region of interest of the at least one MR image of the subject further comprises:providing, using the segmentation system, the at least one MR image of the subject and the prompts to the foundation model;generating, using the foundation model, a second segmentation and a mask based on the at least one MR image of the subject and the set of boundary box and point prompts; andQB\100934698.1 22790482.00574partitioning, using a partitioning module, the second segmentation into a pseudo-labeled set of data and an unlabeled set of data.

5. The method according to claim 4, wherein generating, using the segmentation system a segmentation of a region of interest of the at least one MR image of the subject further comprises:providing, using the segmentation system, the mask, the pseudo-labeled set of data and the unlabeled set of data to the foundation model; andgenerating, using the MGSS network, a final segmentation of the MR image of the subject based on the mask, the pseudo-labeled set of data and the unlabeled set of data.

6. The method according to claim 1, further comprising determining, using the processor device, one or more strain measurements using the segmentation of the at least one DENSE MR image of the subject.

7. The method according to claim 6, wherein the one or more strain measurements is one of circumferential strain, radial strain, or longitudinal strain.

8. The method according to claim 2, wherein the region of interest is a left ventricle.

9. The method according to claim 2, wherein the region of interest is a right ventricle.

10. The method according to claim 1, wherein the trained neural network-based UDA model is implemented as a generative adversarial network.

11. The method according to claim 3, wherein the set of prompts includes bounding boxes and point prompts.

12. The method according to claim 4, wherein partitioning, using the partitioning module, the second segmentation into a pseudo-labeled set of data and an unlabeled set of data furtherQB\100934698.1 23790482.00574comprises determining a Dice similarity coefficient (DSC) between the first segmentation and the second segmentation and comparing the DSC to a predetermined threshold.

13. A system for generating a segmentation of magnetic resonance (MR) images of a subject; the system comprising:a neural network-based unsupervised domain adaptation (UDA) model configured to transfer knowledge from a set of labeled source images to a set of unlabeled target images and generate a first segmentation of at least one MR image of the subject;a post-processing module coupled to the neural-network-based UDA model and configured to generate a set of prompts derived from the first segmentation;a foundation model configured to receive the set or prompts from the post-processing module and the at least one MR image of the subject, and further configured to generate a second segmentation and a mask based on the at least one MR image of the subject and the set of prompts;a partitioning module configured to receive the second segmentation and to partition the second segmentation into a pseudo-labeled set of data and an unlabeled set of data; anda mask-guided semi-supervised (MGSS) network configured to receive the mask, the pseudo-labeled set of data and the unlabeled set of data and to generate a final segmentation of the at least one MR image of the subject.

14. The system according to claim 13, wherein the neural network-based UDA model is implemented as a generative adversarial network.

15. The system according to claim 13, wherein the foundation model is a segment anything model (SAM).

16. The system according to claim 13, wherein the set of prompts includes bounding boxes and point prompts.

17. The system according to claim 13, wherein to partitioning the second segmentation into a pseudo-labeled set of data and an unlabeled set of data comprises determining a Dice similarityQB\100934698.1 24790482.00574coefficient (DSC) between the first segmentation and the second segmentation and comparing the DSC to a predetermined threshold.

18. The system according to claim 13, wherein the at least one MR image of the subject is a DENSE MR image of the subject.QB\100934698.1 25

Images