Systems and methods for image reconstruction

Abstract

A method can include receiving, by one or more processors, under-sampled magnetic resonance imaging (MRI) data and an annotated reference image, generating, by the one or more processors, using a first model, an image based on the under-sampled MRI data. The method can include extracting, by the one or more processors, from at least one second model, at least one reconstructed feature vector and at least one reference feature vector, determining, by the one or more processors, at least one loss value based on at least one of the at least one reconstructed feature vector or the at least one reference feature vector, and updating, by the one or more processors, weights of the first model using the at least one loss value.

Description

Atty Dkt. No.: 046434-0981SYSTEMS AND METHODS FOR IMAGE RECONSTRUCTIONCROSS-REFERENCE TO RELATED APPLICATION

[0001] The present application claims priority to Greek Patent Application No. 20250100134, filed on February 19, 2025, the entire disclosure of which is incorporated herein by reference for all purposes.TECHNICAL FIELD

[0002] The present disclosure relates generally to image reconstruction.BACKGROUND

[0003] Medical image reconstruction networks typically use metrics that originate from computer vision tasks, such as the structural similarity index measure (SSIM), peak signal-to-noise ratio (PSNR), or mean squared error (MSE). While these metrics ensure overall image quality, the metrics lack prioritization of diagnostically relevant features that are crucial for clinical decisionmaking. Radiologists interpret medical images by focusing on subtle features around pathologies, visual cues, and contextual information. Image labels and interpretations provided by radiologists have been used in training networks for pathology detection and classification, segmentation, or medical report generation, using medical images as inputs. However, conventional learning-based approaches for image reconstruction fail to incorporate the radiologist-specific perception into the training process, leading to reconstructed images that meet image quality standards, but may overlook or eliminate diagnostically important features. For example, neural networks for magnetic resonance imaging (MRI) image reconstruction from under-sampled acquisitions (e.g., from parallel imaging) can achieve diagnostic quality up to four -fold undersampling. For larger acceleration factors (e.g., six-fold) the reconstructed images still maintain an overall high SSIM score. Even so, the images may be diagnostically suboptimal since conventional approaches lack the ability to prioritize image features that are important for diagnosis, such as subtle pathologies or anatomical details essential for clinical decision-making.14910-6780-6092.3Atty Dkt. No.: 046434-0981SUMMARY

[0004] At least one aspect of the present disclosure is directed towards a method. The method can include receiving, by one or more processors, under-sampled magnetic resonance imaging (MRI) data and an annotated reference image corresponding to the under-sampled MRI data. The method can include generating, by the one or more processors, using a first model, an image based on the under-sampled MRI data. The method can include extracting, by the one or more processors, from at least one second model, at least one reconstructed feature vector corresponding to the image and at least one reference feature vector corresponding to the annotated reference image after inputting the image and the annotated reference image into the at least one second model. The method can include determining, by the one or more processors, using the at least one second model, at least one loss value based on at least one of the at least one reconstructed feature vector or the at least one reference feature vector and updating, by the one or more processors, weights of the first model using the at least one loss value.

[0005] At least one aspect of the present disclosure is directed towards a system. The system can include one or more processors and at least one memory. The one or more processors can be configured to receive medical imaging data associated with an acceleration factor and an annotated reference image corresponding to the medical imaging data. The one or more processors can generate, using a first model, an image based on the medical imaging data. The one or more processors can determine a similarity metric based on the image and the annotated reference image. The one or more processors can generate, using a second model, a plurality of feature maps for each layer of the second model, the plurality of feature maps corresponding to at least one of the image or the annotated reference image. The one or more processors can extract from at least one of the plurality of feature maps, at least one feature vector. The one or more processors can determine a loss value using a loss function based at least partially on the similarity metric and the at least one feature vector and update the second model based on the loss function.

[0006] At least one aspect of the present disclosure is directed towards another method. The method can include receiving, by one or more processors, medical imaging data associated with a first value and a training image corresponding to the medical imaging data. The method can include generating, by the one or more processors, using a first machine learning model and the medical24910-6780-6092.3Atty Dkt. No.: 046434-0981imaging data, an image. The method can include generating, by the one or more processors, a similarity value based on the image and the training image. The method can include generating, by the one or more processors, using a second machine learning model, a plurality of feature maps for each layer of the second machine learning model, the plurality of feature maps corresponding to at least one of the image or the training image. The method can include extracting, by the one or more processors, from at least one of the plurality of feature maps, a plurality of feature vectors corresponding to at least one of the image or the training image. The method can include determining, by the one or more processors, at least one loss value based on the similarity value and the plurality of feature vectors and updating, by the one or more processors, the first machine learning model using the at least one loss value.

[0007] It should be appreciated that all combinations of the foregoing concepts and additional concepts discussed in greater detail below (provided such concepts are not mutually inconsistent) are contemplated as being part of the subject matter disclosed herein. In particular, all combinations of claimed subject matter appearing at the end of this disclosure are contemplated as being part of the subject matter disclosed herein.BRIEF DESCRIPTION OF THE FIGURES

[0008] The foregoing and other features of the present disclosure will become more fully apparent from the following description and appended claims, taken in conjunction with the accompanying drawings. Understanding that these drawings depict only several implementations in accordance with the disclosure and are therefore not to be considered limiting of its scope, the disclosure will be described with additional specificity and detail through use of the accompanying drawings.

[0009] FIG. 1 is a block schematic diagram of an example system for image reconstruction, according to some implementations of the present disclosure;

[0010] FIG. 2 is a schematic of an example training process for image reconstruction, according to some implementations of the present disclosure;

[0011] FIG. 3 is a table of reconstruction and detection results, according to some implementations of the present disclosure;34910-6780-6092.3Atty Dkt. No.: 046434-0981

[0012] FIG. 4 is a flow diagram of an example method for image reconstruction, according to some implementations of the present disclosure; and

[0013] FIG. 5 is a block schematic diagram of an example computer system, according to some implementations of the present disclosure.

[0014] Reference is made to the accompanying drawings throughout the following detailed description. In the drawings, similar symbols typically identify similar components, unless context dictates otherwise. The illustrative implementations described in the detailed description, drawings, and claims are not meant to be limiting. Other implementations may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented here. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the figures, can be arranged, substituted, combined, and designed in a wide variety of different configurations, all of which are explicitly contemplated and made part of this disclosure.DETAILED DESCRIPTION

[0015] MRI is a medical imaging technique used to visualize internal structures in a body. An MRI machine uses a magnet to generate a strong, uniform magnetic field, gradient coils to generate varying magnetic fields, and radiofrequency (RF) coils to both transmit and receive RF pulses. Both the gradient coils and the RF coils are directed towards different parts of the body. A main magnet of the MRI machine generates a strong, static magnetic field that aligns protons in the body and RF pulses transmitted by the RF coils cause the protons to move from such alignment. Impulses are generated by the gradient coils to vary locally the static magnetic field to encode the spatial location of the protons. Once the RF pulse is stopped, the protons return to their original alignment, RF coils receive the associated RF waves from the protons, and a computer processes such signals to create detailed contrast images. Contrast images highlight different tissues or structures in the body based on radiofrequency signal intensity.

[0016] Parallel imaging is an MRI technique that employs multiple RF coils to complement the spatial encoding provided by the gradient coils to reduce scan times and minimize artifacts due to subject motion in the contrast images, while maintaining a high spatial resolution. In traditional44910-6780-6092.3Atty Dkt. No.: 046434-0981MRI, every line of k-space data is collected sequentially to obtain a full set of k-space data (e.g., frequency domain representation of spatial information in an image). In contrast, parallel imaging leverages the different spatial sensitivity of each RF coils and collects fewer lines of k-space data with multiple RF coils, thereby reducing scan time. The missing k-space data is then reconstructed using a reconstruction algorithm to obtain the full contrast image. Parallel imaging may use a predetermined acceleration factor, which refers to an extent at which scan time is reduced. For example, an acceleration factor of 2 indicates that the scan time is reduced by half, meaning that only half of the k-space lines are acquired. Parallel imaging thus enables to reconstruct high quality MR images from undersampled k-space data to reduce scan time.

[0017] Radiologists analyze and interpret medical images, such as images produced from MRI scans. To analyze the images, radiologists may segment the images corresponding to different pathologies, such as tumors or organs. Radiologists may also perform post processing on the images to improve clarity, delineate tissue boundaries, and remove irrelevant tissue data to enhance visualization of diagnostically relevant regions, among other image analysis techniques. Following analysis, the radiologists generate detailed, diagnostic reports describing the results and findings of the image analysis.

[0018] Neural networks are a type of machine learning model which can process input data and learn patterns. Specifically, neural networks can be trained to identify and extract important attributes (e.g., patterns, features) from data. For example, the networks can extract features from images, such as MR images. These networks can be trained to extract relevant features, such as diagnostically relevant features as determined by radiologists.

[0019] Conventional deep learning (DL)-based MRI reconstruction method have enhanced the cost-efficiency of parallel imaging and compressed sensing techniques (e.g., MR image reconstruction methods), enabling higher acceleration factors and expanding their range of applications in clinic. In particular, DL-based reconstruction methods have enabled clinically accurate MRI scans from four-times accelerated acquisitions (e.g., capturing 25% of full MRI data, 4 times faster than conventional MRI scan methods) compared to the traditionally used two-times accelerated acquisitions relying on non-DL-based image reconstruction methods. However, for higher undersampling rates (for example, five-fold acceleration), existing deep learning methods54910-6780-6092.3Atty Dkt. No.: 046434-0981often fail to reconstruct diagnostically useful images as the structures and features of lesions are not truthfully (e.g., fully, accurately) preserved. This shortcoming is significantly influenced by the image quality assessment metrics used to train the reconstruction networks, which play a key role in the final performance of the reconstruction networks. In fact, the metrics typically used for training networks for MRI tasks are the same used for standard computer vision tasks, and often do not correlate with the metrics used by clinicians to assess the diagnostic quality of MR images. For instance, subtle pathologies can be substantially altered in MR images without a major change in standard image quality metrics such as the structural similarity index measure (SSIM). Such lack of change in image quality metrics indicates that even small changes in SSIM, which do not affect the overall appearance of the image, could yet significantly impact pathology detection if associated with localized changes in image quality. Furthermore, networks yielding quantitatively small differences in SSIM and peak signal-to-noise ratio (PSNR) can have markedly different clinical scores.

[0020] To address these limitations in the broader deep learning field, a perceptual loss (e.g., a loss function that measures difference between two images based on how humans perceive the images) was proposed as an alternative to the pixel-based loss for neural network training. During training, both the ground-truth and the predicted images are passed through a pre-trained loss network with frozen (e.g., fixed) weights, and feature vectors from the loss network layers are extracted. These features encode high-level information, and comparing the feature vectors instead of the images directly can lead to more robust training. In particular for MRI, a perceptual loss can be used for super resolution, image synthesis, segmentation, image denoising, synthesis of absent data, and image reconstruction based on compressed sensing. Networks like VGG (or derivatives) can be used to extract the features for the loss function (e.g., used as the “loss network”) since VGG is pre-trained on large image datasets and the features of VGG correlate well with human perception. Despite the widespread use of VGG networks to inform the perceptual loss in the case of natural images, VGG networks can be less effective as feature extractors for medical images. To address such challenges, application-specific models have be used as the loss network, for example, for models aiming at improving longitudinal image prediction (e.g., predicting future images based on past temporal image sequences) in, for example, infants.64910-6780-6092.3Atty Dkt. No.: 046434-0981

[0021] The loss functions in the existing perceptual loss-based methods may rely on a “general” human perception. As described herein, the systems and methods of the present disclosure include a perceptual loss that aligns directly with medical evaluation from a radiologist. In particular, the systems and methods include a network trained, for example, to detect meniscal tear using a training dataset. The systems and methods include at least one neural network to perform a medical task (e.g., pathology detection, report generation) by fine-tuning on the training dataset. The systems and methods include extracting feature vectors using the at least one neural network trained on a medical task and using such feature vectors to evaluate a medically informed perceptual loss while training an image reconstruction network (e.g., variational network) for, for example, knee MR image reconstruction. The systems and methods of the present disclosure can improve the diagnostic accuracy of specific (e.g., defined) pathologies, such as, meniscal tears for highly accelerated MRI scans (e.g., four-, six-, and eight-fold accelerations), responsive to the loss function as described herein being used in combination with SSIM over solely the SSIM to train the reconstruction network.

[0022] Implementations described herein integrate radiologist expertise into a network to prioritize diagnostically significant features, such as meniscal tears. To integrate the radiologist expertise, the systems and methods of the present disclosure include at least one machine learning model trained on ground truth data labeled by radiologists to produce diagnostically superior images (e.g., compared to conventional reconstruction methods). The systems and methods include a loss function to train (e.g., update) MR image networks from under-sampled acquisitions. The loss function combines an SSIM with a medically informed perceptual loss that utilizes features from a detection network focusing on specific pathologies, such as meniscal tears. The development and utilization of such a loss function addresses a critical gap in conventional learning-based reconstruction training approaches. Individual metrics like SSIM and PSNR, or even perceptual loss that use features from networks trained on natural images (e.g., VGG networks), emphasize overall image fidelity but, at high acceleration factors (e.g., greater than 4), fail to preserve clinically significant features around subtle pathologies, such as meniscal tears.

[0023] FIG. 1 is a block diagram of an example system 100 for reconstructing an image and detecting specific pathologies in the image. The system 100 includes at least one MRI system 102 (e.g., MRI machine, apparatus). The MRI system 102 includes a patient table to position the patient74910-6780-6092.3Atty Dkt. No.: 046434-0981for an MRI scan and uses a magnet as well as gradient coils and RF coils to spatially encode RF signals to excite (e.g., move) nuclei and detect signals emitted by the excited nuclei as the nuclei return to an aligned state. The MRI system 102 can detect and receive signals emitted by the nuclei as the nuclei move from an unaligned state back to the aligned state. The signals may be received by the RF coils. In some implementations, the MRI system 102 is a medical imaging system 102. For example, the medical imaging system 102 can be at least one of an MRI system, a computed tomography (CT) system, ultrasound, or a position emission tomography (PET), among others.

[0024] The system 100 includes a computing system 104 (e.g., controller) communicatively coupled to the MRI system 102. The computing system 104 includes one or more processors 106 and at least one memory 108, which can be implemented as one or more processing circuits. The processor 106 may be a general purpose or specific purpose processor, an application specific integrated circuit (ASIC), one or more field programmable gate arrays (FPGAs), a group of processing components, or other suitable processing components. The processor 106 may be configured to execute computer code or instructions stored in memory (e.g., fuzzy logic, etc.) or received from other computer readable media (e.g., CDROM, network storage, a remote server, etc.) to perform one or more of the processes described herein. The memory 108 may include one or more data storage devices (e.g., memory units, memory devices, computer -readable storage media, etc.) configured to store data, computer code, executable instructions, or other forms of computer-readable information. The memory 108 may include random access memory (RAM), read-only memory (ROM), hard drive storage, temporary storage, non-volatile memory, flash memory, optical memory, or any other suitable memory for storing software objects and / or computer instructions. The memory 108 may include database components, object code components, script components, or any other type of information structure for supporting the various activities and information structures described in the present disclosure. The memory 108 may be communicably connected to the processor 106 and may include computer code for executing (e.g., by processor 106) one or more of the processes described herein. The memory 108 can include various modules (e.g., circuits, engines) for completing processes described herein. The one or more processors 106 and memory 108 may include various distributed components that may be communicatively coupled by wired or wireless connections; for example, various portions of the computing system 104 may be implemented using one or more client devices remote from one or more server devices. The computing system 104 can include any one or more rules,84910-6780-6092.3Atty Dkt. No.: 046434-0981heuristics, logic, code, functions, machine learning models, neural networks, algorithms, or various combinations thereof to implement one or more components of the computing system 104, such as data receiver 110, image generator 112, similarity generator 116, feature generator 118, feature extractor 122, and loss generator 124. The computing system 104 and / or various components thereof can execute various operations described herein and / or combinations thereof as one or more tasks. For example, the data receiver 110 can cause the processor 106 to execute a data receiving task and the image generator 112 can cause the processor 106 to execute an image generating task.

[0025] The computing system 104 controls the MRI system 102, and provides scanning protocols, such as the acceleration factor, parts of the body to scan, etc. For example, the computing system 104 sets the MRI system 102 to a first acceleration factor to enable PI of the MRI scan. The scanning protocol changes based on a type of MRI scan being performed, such as a scan of a knee or scan of an elbow. The computing system 104 can select and adjust the scanning protocol in response to user (e.g., imaging technician) input. The computing system 104 also sets scan times of the MRI system 102. Each parameter of the scanning protocol may be adjusted by a user using the computing system 104. The computing system 104 controls components of the MRI system 102, such as the patient table, and controls a movement and position of the components.

[0026] The computing system 104 includes at least one data receiver 110. The data receiver 110 receives data from at least one of the MRI system 102, the user, or the memory 108. For example, the data receiver 110 can receive signals detected and collected by the RF coils of the MRI system 102 in response to the nuclei moving to their aligned state. The data receiver 110 can receive signals from multiple RF coils in parallel (e.g., simultaneously) when the MRI system 102 is performing PI. The signal can be an under-sampled k-space signal (e.g., acquiring fewer k-space data points than needed for a full MR image). The signal may be under-sampled in response to the MRI system 102 operating at the first acceleration factor. The data receiver 110 can associate the signals with the acceleration factor at which the MRI system 102 is emitting and receiving the signals.

[0027] The data receiver 110 can receive data (e.g., low quality data) from the user. For example, the user can input (e.g., upload) medical imaging (e.g., MRI) data into the computing94910-6780-6092.3Atty Dkt. No.: 046434-0981system 104. The MRI data can include under-sampled, raw multicoil k-space data (e.g., undersampled, raw MRI data). The data receiver 110 can also receive images, such as reference images (e.g., annotated reference images) or training images. The annotated reference images can be annotated by radiologists to indicate, for example, a presence of a specific abnormality, such as a meniscal tear. The annotated reference images can include reports generated by radiologists, and other annotations to indicate segmentation, classification, or other radiologist -related tasks. The images can be at least one of provided by the user or stored in the memory 108. For example, the data receiver 110 can extract the data from the memory 108. Both the medical imaging data and the images can be included in a training dataset. Each of the images can correspond to a set of the medical imaging data, and the images can be a ground-truth (e.g., full, accurate) image associated with a respective set of the medical imaging data.

[0028] The computing system 104 includes at least one image generator 112. The image generator 112 can generate an image (e.g., reconstructed image, MR image) in response to receiving the data (e.g., signal, medical imaging data) from the data receiver 110. For example, the image generator 112 reconstructs an image based on a signal associated with an acceleration factor. As another example, the image generator 112 can reconstruct the image using the medical imaging data provided by the data receiver 110. The image generator 112 can include at least one model 114 (e.g., first model). The model 114 can herein be referred to as the first model, first machine learning model, etc. The first machine learning model 114 can include a reconstruction network (e.g., variational network, reconstruction model). The variational network can predict (e.g., determine) missing k-space data from the data received from the data receiver 110 to generate (e.g., reconstruct) the image. The variational network can be trained end-to-end and can reconstruct the image using spatial sensitive profiles of the RF coils of the MRI system 102. The first model 114 can receive the data as an input, and output the image based on the data provided by the data receiver 110.

[0029] The first model 114 can transform the k-space data provided by the data receiver 110 into an interpretable, visual representation (e.g., image). In some implementations, the model 114 can at least one of: reconstruct an image, perform motion correction, remove image artifacts, enhance a signal-to-noise ratio (SNR), increase a resolution (e.g., super-resolution), generate a synthetic image contrast, or complete the image based on the received data from the MRI system104910-6780-6092.3Atty Dkt. No.: 046434-0981102. In implementations, where the first model 114 performs at least one of motion correction, remove of image artifacts, enhances the SNR, or increases the resolution (e.g., by recovering or estimating details, etc.), the model 114 may not output an image, and may receive data and output modified data. In some implementations, the model 114 can generate at least one of an image reconstructed based on the data or generate a synthetic contrast that may have a different image contrast type (e.g., T1 -weighted, T2-weighted, proton density, etc.) from the received data. In implementations where the model 114 completes an image, the model 114 can estimate or generate data to complete information missing from the received data, or generate the image, and estimate and fill in missing, occluded, or corrupted portions of the image.

[0030] The first model 114 can include a plurality of image generation models. Each of the plurality of image generation models can correspond to an acceleration factor. For example, a first image generation model corresponds to an acceleration factor of 2 while a second image generation model corresponds to an acceleration factor of 4. The first model 114 can select an image generation model to use to generate the reconstructed image based on an acceleration factor associated with the data provided by the data receiver 110. Each signal or medical imaging data can be associated with an acceleration factor. In some implementations, the data receiver 110 receives the data and determines the acceleration factor based on the data. In some implementations, the first model 114 determines the acceleration factor and then selects an image generation model to input the data into.

[0031] The computing system 104 can include at least one similarity generator 116. The similarity generator 116 can receive the reference images from the data receiver 110 and the reconstructed image from the image generator 112. The similarity generator 116 can receive the reconstructed image from the image generator 112, and then receive the associated reference image from the data receiver 110. The reconstructed image and the reference image can correspond to each other based on an association between the data used to reconstruct the image and the reference image stored in the training dataset. Once received, the similarity generator 116 can generate at least one similarity value (e.g., similarity metric) based on the reconstructed image and the reference image. The similarity value can be an SSIM.114910-6780-6092.3Atty Dkt. No.: 046434-0981

[0032] The computing system 104 can include at least one feature generator 118. The feature generator 118 mimics radiologist-related tasks, such as image interpretation, classification, report generation, or detection. The feature generator 118 can include a second model 120 (e.g., machine learning model). The second model 120 can be a neural network, for example, specifically a neural network. The second model 120 can be used to detect specific pathologies, such as meniscal tears, in medical images. In some implementations, the second model 120 can be used to perform classification, segmentation, or report generation. The feature generator 118 can receive the reconstructed image and the reference image from the similarity generator 116 and input the reconstructed image and the reference image into the second model 120. The second model 120 can generate, as an output, feature maps (e.g., feature representation) for each of the reconstructed image and the reference image. For example, the second model 120 outputs at least one reconstructed feature map corresponding to the reconstructed image and at least one reference feature map corresponding to the reference image. The second model 120 can output feature maps for each layer (e.g., block, module) of the second model 120. For example, responsive to the second model 120 including 12 layers, the second model 120 outputs 12 feature maps corresponding to the reconstructed image and 12 feature maps corresponding to the reference image. In some implementations, the second model 120 generates feature maps for a predetermined number of layers. For example, given a predetermined number of 4, even if the second model 120 has 12 layers, the second model 120 generates 4 feature maps for the reconstructed image and 4 for the reference image.

[0033] In some implementations, the second model 120 can perform at least one of classification, detection, segmentation, prediction of diagnosis, or image-to-r eport generation. For example, the second model 120 can generate features maps by extracting feature maps from at least one of the reconstructed or reference image to perform at least one of classification, detection, or segmentation on at least one of the reconstructed or reference image. The second model 120 can detect and classify features of the reconstructed and reference images, and use them to segment the reconstructed and reference images accordingly. Based on at least one of the reconstructed and reference images or corresponding feature maps, the second model 120 can predict a diagnosis or generate a report. For example, the second model 120 can generate a predicted diagnosis of the patient corresponding to the images, such as a type of cancer based on a detected pathology. Based on at least one of the reconstructed or reference images, the second model 120 can generate a124910-6780-6092.3Atty Dkt. No.: 046434-0981report that describes at least imaging techniques, such as parameters of the MRI machine 102, dimensions of the anatomy depicted in the images, or detected pathologies, among others.

[0034] Each of the feature maps generated by the feature generator 118 can include feature vectors. For example, each of the reconstructed feature maps includes reconstructed feature vectors and each of the reference feature maps includes reference feature vectors. The feature vectors represent features at a spatial location (e.g., position) in the image, such as a texture or pattern.

[0035] The second model 120 can have frozen (e.g., fixed) weights. For example, the second model 120 is updated (e.g., trained) prior to being provided with the reconstructed image and the reference image. The second model 120 can be fine-tuned (e.g., trained, tuned, updated) using the training dataset. The training dataset can include a plurality of reference images, and each of the reference images can be tagged (e.g., marked, associated with) a binary value. For example, the training dataset includes a plurality of annotated reference images. The binary value can indicate whether the reference image includes a specific pathology (e.g., abnormality), such as a meniscal tear. For example, 0 indicates that the reference image does not include the specific pathology, and 1 indicates that the reference image includes the specific pathology.

[0036] The second model 120 can be updated by providing the plurality of reference images as inputs into the second model 120. The second model 120 can determine whether the plurality of reference images includes the specific pathology and identify the specific pathology in the image (e.g., position of the specific pathology), and generate a plurality of loss values corresponding to a respective one of the plurality of reference images. The second model 120 can generate the loss values based on a loss function. The loss function can be a combination of a classification loss, objectness loss, and a bounding box regression loss. For example, the second model 120 can generate a bounding box around the specific pathology and indicate a classification and confidence of same for an object within the bounding box. The second model 120 can determine the plurality of loss values based on the output including an indication of a presence of the specific pathology, and a location of the specific pathology within the reference image.

[0037] Using the plurality of loss values generated the, for example, feature generator 118, can update weights of the second model 120 and evaluate a performance of the second model 120 based on the weights. For example, the feature generator 118 can update weights of the second134910-6780-6092.3Atty Dkt. No.: 046434-0981model 120 after each generation of a loss value. The performance can be evaluated by an accuracy of the second model’s 120 indication of the presence and location of the specific pathology. For example, a best performance of the second model 120 can correlate with a lowest loss value. As such, the feature generator 118 can select and store weights for the second model 120 associated with the best performance. The weights stored can be the weights of the second model 120 associated with the lowest comparative loss value. The performance can be evaluated based on a task that the second model 120 is performing. For example, responsive to the task being classification, the performance is evaluated based on precision and recall. As another example, responsive to the task being detection, the performance can be evaluated based on mean average precision or loU. In these cases, the best performance can correlate with a highest precision and recall or a highest mean average precision. Once the feature generator 118 updates and stores the weights of the second model 120, the weights of the second model 120 are frozen. The feature generator 118 can thus generate feature maps that integrate radiologist expertise by training on reference images annotated by radiologists.

[0038] The computing system 104 can include at least one feature extractor 122. The feature extractor 122 can receive the feature maps from the feature generator 118, and extract feature vectors from the feature maps. For example, the feature extractor 122 extracts reconstruction feature vectors from the reconstruction feature map and reference feature vectors from the reference feature map. The feature extractor 122 can extract all the feature vectors from the feature maps. In some implementations, the feature extractor 122 can extract specific feature vectors from the feature vector according to a loss function, described further herein.

[0039] The computing system 104 can include at least one loss generator 124. The loss generator 124 can determine at least one loss value based on at least one of the similarity value, the reconstructed feature vectors, or the reference feature vectors. The loss generator 124 can include a loss function to generate (e.g., calculate, determine) at least one loss value. The loss generator 124 can use the at least one loss value to update weights of the first model 114. The loss generator 124 can determine the at least one loss value using the loss function based on at least one of the similarity value or the feature vectors (e.g., reconstruction feature vectors, reference feature vectors). The loss function can include the similarity value and a perceptual loss (e.g., medically informed perceptual loss, etc.). The perceptual loss can be a difference (e.g., visual144910-6780-6092.3Atty Dkt. No.: 046434-0981difference) between the reconstruction feature map and the reference feature map. For example, the loss generator 124 can compare the reconstruction feature vectors and the reference feature vectors in a latent space that prioritizes diagnostically relevant features. The loss function can include (e.g., determine the loss value by) the feature maps generated by the feature generator 118, a number of layers of the second model 120 used, a number of elements of the feature vectors from the feature maps corresponding to the number of layers, and weight coefficients. The weight coefficients can be configured to prioritize specific feature vectors (e.g., clinically significant features) during training or balance the terms of the loss function. For example, the terms of the loss function can include the SSIM, and the perceptual loss described above. The weight coefficients can be tailored to balance the two terms to determine the loss value. In some implementations, the loss function includes the perceptual loss and does not include the SSIM. For example, the similarity generator 116 does not generate the similarity value, and the loss generator 124 determines the loss value solely on the perceptual loss.

[0040] The feature maps used by the loss generator 124 to determine the loss value can be determined by the number of layers of the second model 120. For example, the loss generator 124 uses a specific number of layers of the second model 120 to determine the loss value, each layer including a feature map. The specific number of layers can be less than or equal to a total number of layers in the second model 120. For example, the number of layers can be in a range between 1 to 25, inclusive. Specifically, the number of layers can be 3, 4, 7, 13, 17, 20, and 23. The number of layers can include corresponding weight coefficients. The weight coefficients can be in a range or 0.3 to 2.7, inclusive. For example, the weight coefficients can be 2.5, 1.2, 0.7, 0.6, 0.5, 0.5, and 0.5. The weight coefficient can increase as the number of layers decrease. The number of layers used by the loss generator 124 to determine the loss value can be predetermined. In some implementations, the loss generator 124 can determine the number of layers to use based on, for example, a quality of the reconstructed image, the similarity value, etc.

[0041] Once the loss value is determined, the loss generator 124 can update the weights of the second model 120. The computing system 104 can continue to provide the reference image and medical imaging data to the data receiver 110 for the loss generator 124 to generate the loss value to update the weights of the second model 120 based on the reference image and a reconstructed image generated by the image generator 112 until the weights of the second model 120 converge154910-6780-6092.3Atty Dkt. No.: 046434-0981(e.g., equilibrate, do not change). Once the weights of the second model 120 converge, the second model 120 can be deployed (e.g., training, updating is complete).

[0042] After the weights of the second model 120 are updated and the weights converge, the data receiver 110 can receive at least one signal associated with an acceleration factor. The signal can be an under-sampled k-space signal provided by the MRI system 102. The signal can be provided during or after a scan performed by the MRI system 102. The data receiver 110 can provide the signal to the image generator 112 to generate the reconstructed image. The first model 114 can receive the signal as an input and output the reconstructed image. The image generator 112 can then provide the reconstructed image to the feature generator 118. The feature generator 118 can provide the reconstructed image as an input into the second model 120, and the second model 120 can output an indication of whether the reconstructed image includes the specific pathology (e.g., meniscal tear). The second model 120 can output both an indication and highlight an area of the specific pathology on the reconstructed image. The second model 120 can detect the specific pathology that the binary values of the plurality of reference images indicate. For example, the second model 120 can detect the specific pathology identified in images of the training dataset.

[0043] In some implementations, weights of the second model 120 can be updated to detect a plurality of pathologies. For example, the training dataset can include sets of the plurality of reference images. Each set of reference images can correspond to a different pathology. For example, a first set of reference images indicates a presence of a meniscal tear, and a second set of reference images indicates a presence of cartilage degeneration. As such, the loss generator 124 can determine the loss value based on the reference image and the reconstructed image provided by the first model 114 using the corresponding medical imaging data. The sets of the plurality of reference images can be processed sequentially, and the second model 120 can detect a plurality of pathologies as a result.

[0044] FIG. 2 depicts an example process 200 that can be implemented by the system 100 as described above. The process 200 can include inputting under-sampled data 202 into the first model 114. For example, the data receiver 110 can receive and provide the under-sampled data 202 to the image generator 112, and the image generator 112 can input the under-sampled data 202 into the first model 114. The first model 114 can have trainable (e.g., updatable) weights. The164910-6780-6092.3Atty Dkt. No.: 046434-0981first model 114 can output a reconstructed image 204 after receiving the under-sampled data 202 as an input. Once output, a similarity value 208 can be determined based on a similarity between the reconstructed image 204 and a reference image 206 (e.g., target image, ground truth image). For example, the data receiver 110 can receive both the under-sampled data 202 and the reference image 206 and provide the under-sampled data 202 to the image generator 112 and the reference image 206 to the similarity generator 116. The reference image 206 can correspond to the undersampled data 202. For example, the reference image 206 can be generated from the under-sampled data 202 using a Fourier transform process. Once the similarity value 208 is determined, both the reference image 206 and the reconstructed image 204 can be provided to the second model 120. The second model 120 can receive and process the reference image 206 and the reconstructed image 204 sequentially or in parallel (e.g., simultaneously). For example, the data receiver 110 can provide the reference image 206 to the feature generator 118 and the image generator 112 can provide the reconstructed image 204 to the feature generator 118. The feature generator 118 can receive and input the reference image 206 and the reconstructed image 204 prior to the similarity generator 116 generating the similarity value 208 or in parallel.

[0045] The second model 120 can receive the reconstructed image 204 and the reference image 206 as inputs and output a plurality of reconstructed feature maps 210 and a plurality of reference feature maps 212. Weights of the second model 120 can be frozen. The plurality of reconstructed feature maps 210 can correspond to the reconstructed image 204, and the plurality of reference feature maps 212 can correspond to the reference image 206. Each of the plurality of reconstructed feature maps 210 and the plurality of reference feature maps 212 include feature vectors. The second model 120 can output the plurality of reconstructed feature maps 210 and the plurality of reference feature maps 212 for each layer of the second model 120, or for each predetermined number of layers of the second model 120. The second model 120 can output the plurality of reconstructed feature maps 210 and the plurality of reference feature maps 212 in sequence or in parallel.

[0046] Once the second model 120 outputs the plurality of reconstructed feature maps 210 and the plurality of reference feature maps 212, a perceptual loss 214 can be determined. The perceptual loss 214 can be determined based at least partially on the plurality of reconstructed feature maps 210 and the plurality of reference feature maps 212 and include a summation of a174910-6780-6092.3Atty Dkt. No.: 046434-0981mean squared error (MSE) of the plurality of reconstructed feature maps 210 and the plurality of reference feature maps 212. The loss generator 124 can determine the perceptual loss 214 and can add the similarity value 208 and the perceptual loss 214 to determine a loss value 216. The loss value 216 can be used to update the weights of the first model 114. For example, the loss generator 124 can update the weights of the first model 114 using the loss value 216.

[0047] The under-sampled data 202 and the reference image 206 can be included in a training dataset. The training dataset can associate the under-sampled data 202 and the reference image 206. For example, the training dataset used to train, validate, and test the first model 114 and second model 120 can include a plurality of medical imaging data and medical images, such as MRI data and MR images of a knee. The training dataset can include, for example, 973 volumes (e.g., three-dimension (3D) data) or 69,487 images for training. A validation dataset included in the training data set can include, for example, 199 volumes and 7,911 images, and can be divided into a plurality of validation datasets at random to create a new validation and an equally large testing dataset. For the annotations of the specific pathologies (e.g., meniscal tears), binary classification can be used (e.g., label equal to 1 if there is the specific pathology in the image or 0 if there was not). The rest of the pathologies (e.g., other than the specific pathology) may not be annotated in the training dataset. In some implementations, additional pathologies are annotated as well in the training dataset.

[0048] To expand the training dataset for updating the weights of the second model 120, data augmentation strategies were implemented. In particular, a a) hue, saturation, and value (HSV) adjustments with hue shift of ±0.015, saturation scale of 0.7, and value scale of 0.4, b random rotations up to ±5 degrees, c) random translations up to 10% of the image size, d) random scaling up to 50% of the image size, e) horizontal and vertical flipping with probability of 50%, and f) mosaic augmentation with a probability of 1 was implemented on the images of the training dataset.

[0049] To train (e.g., update) the first model 114, Cartesian under-sampling was used using the following mask: either 8%, 6%, 4%, or 2% of the central k-space was used as an autocalibration signal (ACS) lines and the rest of k-space was uniformly sampled to achieve an overall acceleration factor of 4, 6, 8, or 10, respectively. Four different versions of the first model 114 can184910-6780-6092.3Atty Dkt. No.: 046434-0981be trained, corresponding to a specific acceleration factor each time. To train the first model 114, random subsets of the training dataset was selected, and each of the random subsets included at least one image including the specific pathology.

[0050] The second model 120 can be fine-tuned prior to freezing the weights of the second model 120. Parameters of the second model 120 can be automatically determined from the training dataset, a learning rate, which was dropped to 0.005 compared to a default 0.01 value, and an intersection over union (loU) threshold, which was set to 0.4 compared to a default 0.2 value. The second model 120 can be trained for 100 epochs, using an image size of 640 x 640 and a batch size of 4. A default loss function of the second model 120 can be used. After fine-tuning, the weights of the second model 120 were stored that had the best performance in the validation dataset, thus, avoiding using overfitted models. The fine-tuning was done on a high-performance cluster using one NVIDIA Al 00 Tensor Core GPU, equipped with 80 GBs of video memory.

[0051] The first model 114 can be trained using a combination of at least one optimization algorithm with a learning rate of 0.0003 and a scheduler based on cosine annealing. The first model 114 can be trained for 210k iteration steps. Until the 7.5k-th iteration, the learning rate can increase linearly from 0 to 0.0003. Afterwards, the learning rate can remain constant for 140k steps. For the remaining steps, the learning rate can drop to lOe - 8 based on a cosine annealing schedule.12 cascades (e.g., sequential process with multiple stages) and 32 channels (e.g., depth of an image tensor) for the first model 114 can be used. The training for all the first models 114 (e.g., four models, plurality of image generation models) was performed on a high-performance cluster using four NVIDIA Al 00 Tensor Core GPU, equipped with 80 GBs of video memory each.

[0052] The loss function used during the training of the first model 114 can be either the SSIM (e.g., similarity value) between the ground-truth image x (e.g., reference image 206) and the predicted image x (e.g., reconstructed image 204) or a weighted combination between SSIM and perceptual loss as:

[0054] Here,. / / (x) and ft(x) are the feature maps (e.g., plurality of reconstructed feature maps 210 and plurality of reference feature maps 212) of a layer I of the second model 120, where x or194910-6780-6092.3Atty Dkt. No.: 046434-0981x can be passed as inputs. The weights of the second model 120 are kept frozen. L can indicate the number of second model 120 layers used for the perceptual loss and Ni can be the number of elements of the output feature vector from layer / . wi are weighting coefficients, which can be selected to either prioritize specific feature vectors during training or balance the terms of the loss function. FIG. 2 presents the process 200 for updating weights of the first model 114 using the above loss function (e.g., equation 1).

[0055] To evaluate an effectiveness of the first model 114 and the second model 120, two types of evaluation metrics can be employed to measure the overall quality of the reconstruction and the bounding box estimation (e.g., second model 120 generates a bounding box around an identified specific pathology). To evaluate the overall image quality, a 3D SSIM and a 3D peak signal to noise ratio (PSNR) can be used. Additionally, a 2D mean loU (mloU) can be used to evaluate the bounding box prediction (e.g., estimation) for a comprehensive comparison. In particular, a predicted bounding box was considered as a positive sample responsive to an mloU of the predicted bounding box with the ground truth bounding box exceeding certain values, such as 0.1 or 0.5. Thresholds of 0.1 and 0.5 can be represented as APio and APso, respectively. The bounding boxes for the reconstructed images were calculated by testing the fine-tuned second model 120 with the reconstructed images.

[0056] FIG. 3 depicts a table 300 of reconstruction and detection results of the first model 114 and the second model 120 following the process 200 which can be implemented by the system 100. By leveraging a feature extraction capability of the second model 120, the perceptual loss can ensure that the reconstructed images (e.g., reconstructed image 204) retain diagnostic integrity, as evidenced by the improved detection accuracy metrics (APio and APso) shown in the table 300.

[0057] The table 300 shows that the APio of the first model 114 trained with the loss function described above for 6-fold acceleration is 0.006 higher than the APio score of a reconstruction network trained solely with SSIM for 4-fold acceleration, despite a 0.167 drop in SSIM. Overall, the enhancement in detection metrics of the second model 120 highlights the added diagnostic value of the proposed PL. Moreover, these results underscore the necessity of task-specific (e.g., detecting specific pathologies) loss functions for improving pathology detection.204910-6780-6092.3Atty Dkt. No.: 046434-0981

[0058] One key advantage of the systems and methods including the perceptual loss approach described above lies in its utilization of features learned by a network (e.g., second model 120) specifically trained to detect specific pathologies, such as meniscal tears. This means that the perceptual loss captures general information about the meniscus and can differentiate between the presence and absence of tears, as well as the indicators associated with these pathologies. In contrast, applying a pixel-wise metric, like PSNR, within the available bounding boxes of the pathology would mean that the reconstruction network may focus solely on the pathology itself, potentially overlooking critical features of healthy cases. This narrow focus increases the risk of hallucinating pathologies in areas without tears, as the reconstruction network may lack a comprehensive understanding of the meniscus’s overall features. By incorporating the PL, the reconstruction process benefits from a broader and more diagnostic-informed feature set, improving the clinical relevance of the results.

[0059] Different neural networks, such as transformers, autoencoders, segmentation, or classification networks can be used as feature extractors (e.g., the second model 120) to form the features (e.g., feature maps, feature vectors) for the perceptual loss. For an approach without data annotation (e.g., without binary value), a vision language models (VLM) for report generation based on MR images can be used. Data for VLM training may be readily available from routine clinical scans and limited post-processing is needed, without requiring input from a radiologist. The systems and methods of the present disclosure can integrate with any existing image reconstruction architecture (e.g., variational network, unrolled optimization network, diffusion model) and can support a wide range of radiologist-informed tasks (e.g., classification, detection, segmentation, report generation, etc.)

[0060] The systems and methods of the present disclosure as described above establish the efficacy of a medically informed perceptual loss in improving the diagnostic quality of MRI reconstructions, such as knee MRI reconstructions particularly for detecting meniscal tears in highly accelerated scans (e.g., acceleration factor of 6). The systems and methods enable a significant step toward clinically meaningful MRI reconstruction methods for higher acceleration factors than currently possible.214910-6780-6092.3Atty Dkt. No.: 046434-0981

[0061] FIG. 4 is a flow diagram of an example method 400 which can be implemented by the system 100 for updating weights of a reconstruction network using a perceptual loss to enable detection of specific pathologies, according to some implementations of the present disclosure. The method 400 can be performed using various systems described herein. Various steps in the method 400 may be repeated, omitted, performed in various orders, or otherwise modified. Various steps in the method 400 may be run concurrently, in parallel, or individually. The method 400 may be implemented by at least one or more processors, such as the processor 106.

[0062] At block 402, the method 400 can include receiving medical imaging data. The medical imaging data can include an under-sampled MRI data (e.g., under-sampled data 202) and a reference image (e.g., training image, reference image 206) corresponding to the under-sampled MRI data. The medical imaging data, such as the under-sampled MRI data, can be associated with a first value, such as an acceleration factor. The medical imaging data can include under-sampled multicoil k-space data.

[0063] At block 404, the method 400 can include generating an image (e.g., reconstructed image 204). The image can be generated using a first model (e.g., first model 114, first machine learning model, variational model, neural network). The image can be a reconstructed image based on the medical imaging data (e.g., under-sampled MRI data). In some implementations, the first model includes a plurality of image generation models. Each of the image generation models can correspond to a different acceleration factor (e.g., 2, 4, etc.) Based on the acceleration factor associated with the medical imaging data, the first model can generate the image using a respective image generation model of the plurality of image generation models.

[0064] At block 406, the method 400 can include extracting at least one feature vector. The at least one feature vector can be extracted from a second model (e.g., second model 120, second machine learning model, neural network, detection machine learning model). Weights of the second model can be frozen. The at least one feature vector can include at least one reconstructed feature vector and at least one reference feature vector. The at least one reconstructed feature vector can correspond to the reconstructed image and the at least one reference feature vector can correspond to the reference image. The at least one feature vector can be output by the second model in response to receiving the reconstructed image and the reference image as an input.224910-6780-6092.3Atty Dkt. No.: 046434-0981

[0065] In some implementations, the at least one feature vector is included in at least one feature map. For example, prior to block 408, the method 400 includes generating a first set of feature maps corresponding to the reconstructed image (e.g., reconstructed feature map 210) and a second set of feature maps corresponding to the reference image (e.g., reference feature map 212). The second model can output the first set of feature maps and the second set of feature maps in response to receiving an input of the reconstructed image and the reference image. The first set of feature maps can include the reconstructed feature vectors, and the second set of feature maps can include the reference feature vectors. The second model can generate a feature map for each layer of the second model.

[0066] In some implementations, to determine the weights of the second model (e.g., prior to fixing the weights), the method 400 can include tuning the second model using a training dataset prior to receiving the medical imaging data. The training dataset can include a plurality of reference images (e.g., training images). Each of the plurality of reference images can be associated with a binary value. The binary value can indicate a presence of a specific (e.g., defined) abnormality in a respective reference image. The binary value may be input for each reference image by one or more radiologists. The second model can be tuned to detect the specific abnormality, and the performance of the network can be based at least partially on an accuracy of detecting the specific abnormality. The method 400 can include providing the plurality of reference images to the second model to update weights of the second model. The second model can detect whether each of the plurality of reference images include the specific abnormality and an estimated location of the specific abnormality. Based on the detection and the estimated location, a plurality of loss values can be generated for each of the reference images. The plurality of loss values can be used to update weights of the second model. The weights of the second model with a best performance (e.g., lowest loss value) compared to the other weights (e.g., responsive to updates by at least one of the loss values of the plurality of loss values) are stored, and the weights of the second model with the best performance are frozen.

[0067] At block 408, the method 400 can determine a loss value (e.g., loss value 216). The loss value can be determined based on the feature vectors (e.g., reconstructed feature vector, reference feature vector). The loss value can be determined using a loss function. For example, the loss function includes (e.g., incorporates) the feature vectors. The loss function can include a234910-6780-6092.3Atty Dkt. No.: 046434-0981number of layers of the second model and at least one weight coefficient. The number of layers can be in a range between 1 to 25, inclusive. The weight coefficient can be in a range of 0.2 to 2.7, inclusive. The weight coefficient can correspond to the number of layers. The loss function can include a perceptual loss. The perceptual loss can be determined using at least the number of layers and the weight coefficient. The perceptual loss can be based at least partially on a number of layers of the second model, the plurality of feature maps associated with the number of layers, and the weight coefficients.

[0068] In some implementations, the loss function can be a weighted combination of a similarity value and the perceptual loss. At least one similarity value, such as an SSIM, can be generated based on the reference image and the reconstructed image. The similarity value can be included in the loss function to determine the loss value. For example, the loss value can be calculated based on a weighted combination of an SSIM and the perceptual loss.

[0069] At block 410, the method 400 can include updating a model (e.g., the first model) based on the loss value. The first model can be updated by updating weights of the model. The first model can be updated until weights of the model converge. In some implementations, after updating the weights, the method 400 can include receiving a signal associated with an acceleration factor. The signal can be input into the first model which can generate the image. Following generation of the image, the second model can detect whether the image includes at least one abnormality (e.g., specific pathology). The second model can also generate a box indicating a location of the at least one abnormality. Following detection of the abnormality, the second model can generate a tag or an alert.Definitions.

[0070] As used herein, the singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise. Thus, for example, the term “a member” is intended to mean a single member or a combination of members, “a material” is intended to mean one or more materials, or a combination thereof.244910-6780-6092.3Atty Dkt. No.: 046434-0981

[0071] As used herein, the terms “about” and “approximately” generally mean plus or minus 10% of the stated value. For example, about 0.5 would include 0.45 and 0.55, about 10 would include 9 to 11, about 1000 would include 900 to 1100.

[0072] It should be noted that the term “exemplary” as used herein to describe various implementations is intended to indicate that such implementations are possible examples, representations, and / or illustrations of possible implementations (and such term is not intended to connote that such implementations are necessarily extraordinary or superlative examples).

[0073] As used herein, the terms “coupled,” “connected,” and the like mean the joining of two additional intermediate members being integrally formed as a single unitary body with one another or with the two members or the two members and any additional intermediate members being attached to one another.

[0074] As shown in FIG. 5 a system 500 includes a computer -accessible medium 520, a processing arrangement 510, and a storage arrangement 540 joined directly or indirectly to one another. Such joining may be stationary (e.g., permanent) or moveable (e.g., removable or releasable). Such joining may be achieved with the two members or the two members and any device such as a hard disk, floppy disk, memory stick, CD-ROM, RAM, ROM, etc., or a collection thereof) can be provided (e.g., in communication with the processing arrangement 510). The computer-accessible medium 520 may be a non-transitory computer-accessible medium. The computer-accessible medium 520 can contain executable instructions 550 thereon. In addition, or alternatively, a storage arrangement 540 can be provided separately from the computer-accessible medium 520, which can provide the instructions to the processing arrangement 510 so as to configure the processing arrangement to execute certain exemplary procedures, processes and methods, as described herein, for example. The instructions may include a plurality of sets of instructions.

[0075] The system 500 may also include a display or output device, an input device such as a keyboard, mouse, touch screen or other input device, and may be connected to additional systems via a logical network. Many of the implementations described herein may be practiced in a networked environment using logical connections to one or more remote computers having processors. Logical connections may include a local area network (“LAN”) and a wide area254910-6780-6092.3Atty Dkt. No.: 046434-0981network (“WAN”) that are presented here by way of example and not limitation. Such networking environments are commonplace in office-wide or enterprise-wide computer networks, intranets and the Internet and may use a wide variety of different communication protocols. Those skilled in the art can appreciate that such network computing environments can typically encompass many types of computer system configurations, including personal computers, hand-held devices, multiprocessor systems, microprocessor -based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, and the like. Implementations of the invention may also be practiced in distributed computing environments where tasks are performed by local and remote processing devices that are linked (either by hardwired links, wireless links, or by a combination of hardwired or wireless links) through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0076] Various implementations are described in the general context of method steps, which may be implemented in one implementation by a program product including computer-executable instructions, such as program code, executed by computers in networked environments. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Computer-executable instructions, associated data structures, and program modules represent examples of program code for executing steps of the methods disclosed herein. The particular sequence of such executable instructions or associated data structures represents examples of corresponding acts for implementing the functions described in such steps.

[0077] Software and web implementations of the present invention could be accomplished with standard programming techniques with rule-based logic and other logic to accomplish the various database searching steps, correlation steps, comparison steps and decision steps. It should also be noted that the words “component” and “module,” as used herein and in the claims, are intended to encompass implementations using one or more lines of software code, and / or hardware implementations, and / or equipment for receiving manual inputs.

[0078] It is important to note that the construction and arrangement of the various exemplary implementations are illustrative only. Although only a few implementations have been described in detail in this disclosure, those skilled in the art who review this disclosure will readily appreciate264910-6780-6092.3Atty Dkt. No.: 046434-0981that many modifications are possible (e.g., variations in sizes, dimensions, structures, shapes and proportions of the various elements, values of parameters, mounting arrangements, use of materials, colors, orientations, etc.) without materially departing from the novel teachings and advantages of the subject matter described herein. Other substitutions, modifications, changes and omissions may also be made in the design, operating conditions and arrangement of the various exemplary implementations without departing from the scope of the present invention.

[0079] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any inventions or of what may be claimed, but rather as descriptions of features specific to particular implementations of particular inventions. Certain features described in this specification in the context of separate implementations can also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation can also be implemented in multiple implementations separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.274910-6780-6092.3

Claims

Atty Dkt. No.: 046434-0981WHAT IS CLAIMED IS:

1. A method, comprising:receiving, by one or more processors, under-sampled magnetic resonance imaging (MRI) data and an annotated reference image corresponding to the under-sampled MRI data;generating, by the one or more processors, using a first model, an image based at least on the under-sampled MRI data;extracting, by the one or more processors, using at least one second model, at least one reconstructed feature vector from the image and at least one reference feature vector from the annotated reference image after inputting the image and the annotated reference image into the at least one second model;determining, by the one or more processors, using the at least one second model, at least one loss value based on at least one of the at least one reconstructed feature vector or the at least one reference feature vector; andupdating, by the one or more processors, weights of the first model using the at least one loss value.

2. The method of claim 1 , wherein weights of the at least one second model are fixed.

3. The method of claim 2, wherein to determine the weights of the at least one second model, the method further comprises:providing, by the one or more processors, a plurality of annotated reference images comprised in a training dataset to the at least one second model prior to receiving the undersampled MRI data and the annotated reference image;generating, by the one or more processors, using the at least one second model, at least one of an indication of a presence of a specific abnormality and a location of the specific abnormality in each of the plurality of annotated reference images;generating, by the one or more processors, a plurality of loss values for each of the plurality of annotated reference images based on at least one of the indication and the location of the specific abnormality for each of the plurality of annotated reference images; and storing, by the one or more processors, the weights of the at least one second model corresponding to a lowest loss value of the plurality of loss values.284910-6780-6092.3Atty Dkt. No.: 046434-09814. The method of claim 3, wherein each of the plurality of annotated reference images are associated with a binary value, the binary value indicative of the presence of the specific abnormality in each of the plurality of annotated reference images.

5. The method of claim 1, further comprising generating, by the one or more processors, using the at least one second model, a first set of feature maps corresponding to the image and a second set of feature maps corresponding to the annotated reference image for each layer of the at least one second model, wherein the first set of feature maps and the second set of feature maps are output by the at least one second model in response to receiving the input of the image and the annotated reference image.

6. The method of claim 5, further comprising:determining, by the one or more processors, at least one similarity value based on the annotated reference image and the image; anddetermining, by the one or more processors, the at least one loss value based on at least one of the at least one similarity value, the at least one reconstructed feature vector, or the at least one reference feature vector.

7. The method of claim 6, wherein:the at least one loss value is determined using a loss function, the loss function comprising a number of layers of the at least one second model and at least one weight coefficient;the number of layers are in a range between 1 to 25, inclusive; andthe at least one weight coefficient is in a range of 0.3 to 2.7, inclusive.

8. The method of claim 7, wherein the loss function comprises the at least one similarity value and a perceptual loss, the perceptual loss determined using at least the number of layers and the at least one weight coefficient, wherein the perceptual loss is a medically informed perceptual loss.294910-6780-6092.3Atty Dkt. No.: 046434-09819. The method of claim 1, further comprising:receiving, by the one or more processors, after updating the weights of the first model, a signal associated with an acceleration factor;generating, by the one or more processors, using the first model and the signal, the image; anddetecting, by the one or more processors, using the at least one second model, whether the image comprises at least one abnormality.

10. A system, comprising one or more processors and at least one memory, the one or more processors configured to:receive medical imaging data associated with an acceleration factor and an annotated reference image corresponding to the medical imaging data;generate, using a first model, an image based on the medical imaging data; determine a similarity metric based on the image and the annotated reference image; generate, using a second model, a plurality of feature maps for each layer of the second model, the plurality of feature maps corresponding to at least one of the image or the annotated reference image;extract, from at least one of the plurality of feature maps, at least one feature vector; determine a loss value using a loss function based at least partially on the similarity metric and the at least one feature vector; andupdate the second model based on the loss value.

11. The system of claim 10, wherein the first model comprises a plurality of image generation models, each of the plurality of image generation models corresponding to a different acceleration factor, the first model generating the image based on the acceleration factor associated with the medical imaging data using a corresponding image generation model.

12. The system of claim 10, wherein the loss function comprises a weighted combination of the similarity metric and a perceptual loss.

13. The system of claim 12, wherein the perceptual loss is based at least partially on a number of layers of the second model, the plurality of feature maps associated with the number of layers, and weight coefficients.304910-6780-6092.3Atty Dkt. No.: 046434-098114. The system of claim 13, wherein:the number of layers is in a range of 1 to 25, inclusive; anda value of the weight coefficients is in a range of 0.3 to 2.7, inclusive.

15. The system of claim 10, wherein the first model is a reconstruction machine learning model, and the second model is a detection machine learning model.

16. A method, comprising:receiving, by one or more processors, medical imaging data associated with a first value and a training image corresponding to the medical imaging data;generating, by the one or more processors, using a first machine learning model and the medical imaging data, an image;generating, by the one or more processors, a similarity value based on the image and the training image;generating, by the one or more processors, using a second machine learning model, a plurality of feature maps for each layer of the second machine learning model, the plurality of feature maps corresponding to at least one of the image or the training image;extracting, by the one or more processors, from at least one of the plurality of feature maps, a plurality of feature vectors corresponding to at least one of the image or the training image;determining, by the one or more processors, at least one loss value based on the similarity value and the plurality of feature vectors; andupdating, by the one or more processors, the first machine learning model using the at least one loss value.

17. The method of claim 16, wherein the medical imaging data comprises under-sampled multicoil k-space data and the first value is an acceleration factor.

18. The method of claim 16, wherein the at least one loss value is determined using a loss function and the loss function comprises a weighted combination of the similarity value and a perceptual loss.314910-6780-6092.3Atty Dkt. No.: 046434-098119. The method of claim 18, wherein the perceptual loss is based at least partially on a number of layers of the second machine learning model and the plurality of feature vectors corresponding to the number of layers of the second machine learning model.

20. The method of claim 16, wherein the second machine learning model is updated using a dataset, the dataset comprising a plurality of training images, each of the plurality of training images associated with a value indicative of a presence of a defined abnormality.324910-6780-6092.3

Images