Systems and methods for assessing embryo viability using artificial intelligence

Abstract

Systems and methods for predicting the viability of one or more embryos are described herein. In some variations, the method may include receiving a single image of an embryo via a real-time communication link with an image capture device, and generating a viability score for the embryo by classifying the single image via at least one convolutional neural network. In some variations, the method may include receiving a plurality of single images, each single image depicting a different individual embryo of the plurality of embryos, and generating a viability score for each embryo by classifying each single image via at least one convolutional neural network, and ranking the plurality of embryos based on the viability score for the plurality of embryos.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims priority to U.S. Provisional Patent Application No. 63 / 256,332, filed October 15, 2021, and U.S. Provisional Patent Application No. 63 / 157,433, filed March 5, 2021, each of which is incorporated herein by reference in its entirety.

[0002] The present invention relates generally to the field of assessing embryo viability. [Background technology]

[0003] In vitro fertilization (IVF) is a widely known assisted reproductive technique. IVF involves several complicated steps, such as ovarian stimulation, oocyte retrieval, oocyte fertilization, embryo culture, embryo selection, and embryo transfer. Typically, embryos are cultured until the blastocyst stage (e.g., embryo transfer stage). That is, following oocyte retrieval and fertilization, the embryos are cultured until there is clear differentiation into inner cell mass and trophectoderm structures. Less competent embryos often stop their development prior to the blastocyst stage. Generally, a group of embryos can reach the blastocyst stage. Therefore, embryos that survive to the blastocyst stage need to be assessed before the embryos are selected for transfer. Based on the assessment, a single embryo (or, in rare cases, multiple embryos) can be selected for transfer.

[0004] Thus, embryo selection is a critical aspect of the IVF process. Traditionally, embryo selection is performed by an embryologist manually inspecting and assessing the embryos. The embryologist may assign a grade to the embryo by inspecting the embryo under a microscope. To grade the embryo, the embryologist may assess features such as the degree of blastocyst expansion, the quality of the inner cell mass, and the quality of the trophectoderm. However, manually grading embryos can be a highly subjective process. Different embryologists may grade embryos differently based on their own individual manual inspection. Research has found that manual inspection and grading can often be an intuitive approach. Thus, grades can vary significantly depending on the embryologist inspecting the embryo.

[0005] More recently, non-manual techniques have been explored to make the process of embryo selection more consistent. However, these existing techniques have not been able to gain widespread adoption. For example, time-lapse imaging, which captures a sequence of images of an embryo in a cyclical manner, has been extensively studied. However, time-lapse imaging requires specialized microscopes, which tend to be expensive. The high cost of its installation has prevented clinics and laboratories from adopting the technology. Another technique that has been recently investigated is preimplantation embryo chromosomal aneuploidy testing (PGT-A). Existing PGT-A tests are invasive tests. Concerns have been raised about the health of embryos following these tests. In addition, existing PGT-A tests only identify euploid and aneuploid embryos. Although it is known that aneuploid embryos are less likely to have normal pregnancy outcomes, not all euploid embryos lead to normal pregnancy outcomes. Thus, existing PGT-A tests still cannot completely solve the problem of assessing embryo viability.

[0006] Thus, there is an unmet need for new and improved methods to standardize embryo grading and improve the accuracy of predicting embryo viability. Further, there is an unmet need for new and improved methods of assessing embryo viability that are cost-effective, easy to implement, and easy to adopt. Summary of the Invention [Means for solving the problem]

[0007] Generally, in some variations, a computer-implemented method for predicting embryo viability may include receiving a single image of an embryo and generating a viability score for the embryo by classifying the single image via at least one convolutional neural network, the viability score representing a predicted viability of the embryo. In some variations, the single image classified via at least one convolutional neural network is not part of a time series of images. The viability score may represent, for example, a predicted likelihood that the embryo will achieve clinical pregnancy (e.g., the likelihood that the embryo will achieve clinical pregnancy may be associated with an outcome of fetal cardiac activity), a likelihood that the embryo will achieve live birth, and / or the like. In some variations, the viability score may be based, at least in part, on data associated with the patient, such as age, body mass index, date of image capture, and donor status. Once generated, the viability score may be stored in a database associated with the patient (e.g., a patient into which the embryo may be implanted) and / or communicated to the patient, a clinician, a user of the image capture device, and the like.

[0008] For example, in some variations, the computer-implemented method may include receiving a single image via a real-time communication link with the image capture device, cropping the single image to a boundary of the embryo via a first convolutional neural network, and classifying the single image via at least a second convolutional neural network to generate a viability score for the embryo. In some variations, the single image to be classified is not part of a time series of images. As described above, the viability score may represent, for example, a predicted likelihood that the embryo will reach clinical pregnancy (e.g., the likelihood that the embryo will reach clinical pregnancy may be associated with an outcome of fetal cardiac activity). As another example, the viability score may represent the likelihood that the embryo will reach live birth. In some variations, the viability score may be based at least in part on data associated with the patient, such as age, body mass index, date of image capture, and donor status. Once generated, the viability score may be stored in a database associated with the patient (e.g., a patient into whom the embryo may be implanted) and / or communicated to the patient, a clinician, a user of the image capture device, etc.

[0009] In some variations, the real-time communications link may be provided by an application executing on a computing device communicatively coupled to the image capture device. The application may cause a display on the computing device to display a capture button such that, in response to a user selecting the capture button, the image capture device captures one or more single images of the embryo.

[0010] Further, in some variations, the method may include performing one or more quality control measures on the single image, such as determining whether the single image depicts an embryo (e.g., via a third convolutional neural network) and / or determining a probability that the embryo in the single image is a single blastocyst. Further, in some variations, the method may include generating a viability score for the embryo in response to determining that the single image depicts an embryo. Additionally or alternatively, in some variations, the method may include providing an alert to a user of the image capture device in response to determining that the single image does not depict an embryo.

[0011] Additionally or alternatively, the method may further include predicting whether the embryo is euploid or aneuploid via a fourth convolutional neural network. The predicting may also depend, in some variations, at least in part, on data associated with the subject (e.g., age, date of biopsy, etc.). The method may include generating a ploidy outcome based on whether the embryo is euploid or aneuploid, and updating at least the fourth convolutional neural network based, at least in part, on the ploidy outcome and the data.

[0012] The method may be used to predict the viability of an embryo that is not frozen and / or that has been frozen and thawed. For example, in some variations, the embryo has been frozen and thawed, and the method may include receiving a single image of the embryo after thawing and determining the viability of the embryo after thawing via a second convolutional neural network. Determining the viability of the embryo after thawing may include classifying the single image into either a first class indicating that the embryo is viable after thawing or a second class indicating that the embryo has reduced viability after thawing (e.g., a lower level of viability, not viable, etc.). In some variations, the method may be used to predict the viability of an embryo that should undergo biopsy and / or freezing. For example, the method may include receiving a single image of an embryo prior to biopsy and / or freezing and determining the viability of the embryo prior to biopsy and / or freezing.

[0013] Additionally or alternatively, the method may include receiving a plurality of single images, each single image depicting an individual embryo of the plurality of embryos, generating a per-embryo viability score for the plurality of embryos by classifying each single image via at least one convolutional network, and ranking the embryos based on the viability scores for the plurality of embryos. Further, in some variations, the method may include displaying the plurality of single images on a display and / or displaying the viability scores for the plurality of embryos according to the ranking of the plurality of embryos.

[0014] In some variations, the single image classified via the at least one convolutional neural network is not part of a time series of images. In some variations, some of the multiple images may originate from different image capture devices (e.g., different instances of image capture devices and / or different types of image capture devices).

[0015] As explained above, the viability score may represent, for example, a predicted likelihood that the embryo will reach a clinical pregnancy (e.g., the likelihood that the embryo will reach a clinical pregnancy may be associated with the outcome of fetal cardiac activity). In some variations, the viability score may be based, at least in part, on data associated with the patient, such as age, body mass index, date of image capture, and donor status. Once generated, the viability score may be stored in a database associated with the patient (e.g., a patient into which the embryo may be implanted) and / or communicated to the patient, a clinician, a user of the image capture device, etc.

[0016] Additionally or alternatively, the method may further include predicting whether the embryo is euploid or aneuploid via a fourth convolutional neural network. The predicting may also depend, in some variations, at least in part, on data associated with the subject (e.g., age, date of biopsy, etc.). The method may include generating a ploidy outcome based on whether the embryo is euploid or aneuploid, and updating at least the fourth convolutional neural network based, at least in part, on the ploidy outcome and the data.

[0017] As described above, the method may be used to predict the viability of an embryo that is not frozen and / or that has been frozen and thawed. For example, in some variations, the embryo has been frozen and thawed, and the method may include receiving a single image of the embryo after thawing and determining the viability of the embryo after thawing via a second convolutional neural network. Determining the viability of the embryo after thawing may include classifying the single image into either a first class indicative of the embryo being viable after thawing or a second class indicative of the embryo not being viable after thawing.

[0018] Generally, in some variations, the method may utilize at least one convolutional neural network trained at least in part with specialized training data. For example, a method for predicting embryo viability may include receiving a single image of an embryo captured using an image capture device and generating a viability score for each embryo by classifying each single image via at least one convolutional neural network, where the at least one convolutional neural network may be configured to generate a viability score for the embryo, for example, and may be trained based on training data comprising a plurality of single images of the embryo captured using a plurality of image capture devices. Additionally or alternatively, the at least one convolutional neural network may be trained at least in part by balancing the occurrence rates of outcomes associated with each individual image capture device. For example, the occurrence rates of outcomes may include a corresponding bias representing the percentage of positive pregnancy outcomes associated with each individual image capture device. In some variations, the single images to be classified are not part of a time series of images. As explained above, the viability score may represent, for example, a predicted likelihood that the embryo will reach a clinical pregnancy (e.g., the likelihood that the embryo will reach a clinical pregnancy may be associated with the outcome of fetal cardiac activity). In some variations, the viability score may be based, at least in part, on data associated with the patient, such as age, body mass index, date of image capture, and donor status. Once generated, the viability score may be stored in a database associated with the patient (e.g., a patient into which the embryo may be implanted) and / or communicated to the patient, a clinician, a user of the image capture device, etc.

[0019] As another example, in some variations, a method for predicting embryo viability may include receiving single images of embryos and generating a viability score for each embryo by classifying each single image via at least one convolutional neural network, which may be trained at least in part based on training data including a plurality of augmented images of a plurality of embryos. The augmented images may include, for example, rotated, flipped, scaled, and / or perturbed (e.g., having changes in contrast, brightness, saturation, etc.) images of the plurality of embryos. In some variations, the single image to be classified is not part of a time series of images. As described above, the viability score may represent, for example, a predicted likelihood that an embryo will reach a clinical pregnancy (e.g., the likelihood that an embryo will reach a clinical pregnancy may be associated with an outcome of fetal cardiac activity). In some variations, the viability score may be based at least in part on data associated with the patient, such as age, body mass index, date of image capture, and donor status. Once generated, the viability score may be stored in a database associated with the patient (e.g., the patient into which the embryo may be implanted) and / or communicated to the patient, a clinician, a user of the image capture device, etc. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1 illustrates an exemplary variation of a system for assessing embryo viability.

[0021] [Diagram 2] FIG. 2 is a flow diagram illustrating an exemplary variation of a method for assessing embryo viability using artificial intelligence.

[0022] [Figure 3A] 3A-3H illustrate example variations of a graphical user interface (GUI) that may be part of the plug-and-play software that is rendered on the display of a computing device for capturing images of an embryo. [Figure 3B] 3A-3H illustrate example variations of a graphical user interface (GUI) that may be part of the plug-and-play software that is rendered on the display of a computing device for capturing images of an embryo. [Figure 3C] 3A-3H illustrate example variations of a graphical user interface (GUI) that may be part of the plug-and-play software that is rendered on the display of a computing device for capturing images of an embryo. [Figure 3D] 3A-3H illustrate example variations of a graphical user interface (GUI) that may be part of the plug-and-play software that is rendered on the display of a computing device for capturing images of an embryo. [Figure 3E] 3A-3H illustrate example variations of a graphical user interface (GUI) that may be part of the plug-and-play software that is rendered on the display of a computing device for capturing images of an embryo. [Figure 3F] 3A-3H illustrate example variations of a graphical user interface (GUI) that may be part of the plug-and-play software that is rendered on the display of a computing device for capturing images of an embryo. [Figure 3G] 3A-3H illustrate example variations of a graphical user interface (GUI) that may be part of the plug-and-play software that is rendered on the display of a computing device for capturing images of an embryo. [Figure 3H] 3A-3H illustrate example variations of a graphical user interface (GUI) that may be part of the plug-and-play software that is rendered on the display of a computing device for capturing images of an embryo.

[0023] [Figure 4]FIG. 4 is a flow diagram illustrating an exemplary variation of a method for assessing embryo viability using a series of convolutional neural networks.

[0024] [Diagram 5] FIG. 5 is an exemplary variation that implements a convolutional neural network on input images of embryos for image cropping and image segmentation.

[0025] [Figure 6] FIG. 6 is an exemplary variation that implements a convolutional neural network to perform quality control.

[0026] [Figure 7] FIG. 7 illustrates an exemplary deployment of a convolutional neural network to implement quality control.

[0027] [Figure 8] FIG. 8 is an exemplary variation implementing a convolutional neural network for image classification and score generation.

[0028] [Figure 9] FIG. 9 illustrates an example variation of a GUI that may be part of the plug-and-play software that is rendered on a display of a computing device to display an overall viability score for an embryo.

[0029] [Figure 10] FIG. 10 illustrates an example variation of a GUI that may be part of plug-and-play software that is rendered on the display of a computing device to display images of embryos in the order in which they are ranked.

[0030] [Figure 11] FIG. 11 illustrates an example of an augmented image following application of a random transformation to the image.

[0031] [Figure 12] FIG. 12 illustrates an overview of the characteristics in the training dataset, which contains images of over 2,000 transferred embryos with pregnancy outcomes from seven different clinics.

[0032] [Figure 13] FIG. 13 illustrates the receiver operating characteristic curve for fresh embryo transfer using the techniques described herein compared to the Gardner grading system.

[0033] [Figure 14] FIG. 14 illustrates an exemplary variation of an image of an aneuploid embryo.

[0034] [Figure 15] FIG. 15 illustrates an overview of the characteristics in the training dataset, which contains images of over 2,000 transferred embryos with ploidy status from seven different clinics.

[0035] [Figure 16] FIG. 16 illustrates post-thaw viability assessment results from a single center.

[0036] [Figure 17] 17A and 17B illustrate receiver operating characteristic (ROC) curves for an exemplary embryo transfer using a CNN described herein compared to the manual Gardner grading system.

[0037] [Figure 18A] FIG. 18A illustrates an example embryo image that is top ranked by the techniques described herein.

[0038] [Figure 18B] FIG. 18B illustrates an example embryo image that is ranked lowest by the techniques described herein.

[0039] [Figure 19A]FIG. 19A illustrates the integrated gradient and occlusion sensitivity for an example embryo image that was scored highly by the techniques described herein.

[0040] [Figure 19B] FIG. 19B illustrates the integrated gradient and occlusion sensitivity for an example embryo image that was scored low by the techniques described herein.

[0041] [Figure 20] FIG. 20 illustrates that the scores assigned by the techniques described herein, based on exemplary embryo images, relate to observed pregnancy rates.

[0042] [Figure 21] 21A-21D illustrate example images and data from a control experiment to depict the bias introduced by the unique optical signatures of images from two different image capture devices in two different clinics.

[0043] [Figure 22] FIG. 22 illustrates a table illustrating the balance of the exemplary training data based on the incidence of outcomes for different clinics.

[0044] [Figure 23] 23A and 23B illustrate data from a control experiment to depict the bias introduced by the presence of a micropipette in the image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0045] Detailed Description Non-limiting examples of various aspects and variations of the present invention are described herein and illustrated in the accompanying drawings.

[0046] In vitro fertilization (IVF) is a complex assisted reproductive technique that involves the fertilization of eggs outside the body in a laboratory setting. Fertilized embryos are cultured in laboratory culture dishes (e.g., petri dishes) and transferred to the uterus after fertilization. Typically, the embryo begins to show clear differentiation between the inner cell mass that will form the fetus and the trophectoderm structures that will form the placenta, about 5-6 days after fertilization. This stage is referred to as the blastocyst stage. During the blastocyst stage, the embryo breaks free of the zona pellucida membrane that surrounds the embryo in preparation for "hatching." The embryo must reach the blastocyst stage and hatch before it can implant in the inner wall of the uterus. Thus, extending embryo culture until the embryo reaches the blastocyst stage allows the embryologist more time to observe and assess the viability of the embryo. Additionally, less viable embryos stop their development prior to the blastocyst stage. Thus, typically, embryos that progress to the blastocyst stage are a selected subset of embryos that have a higher potential to form a pregnancy.

[0047] Embryos that reach the blastocyst stage are evaluated before they are transferred to prioritize which embryos should be transferred first. Traditionally, embryos are manually graded by embryologists using the Gardner or Society for Assisted Reproductive Technology (SART) grading systems. These systems require the embryologist to manually inspect the embryo under a microscope and assess three components of its morphology: degree of blastocyst expansion, quality of the inner cell mass, and quality of the trophectoderm. Grades are assigned to each component to generate a final alphanumeric grade. However, manual grading can be complex and it can be difficult to assign an absolute grade. For example, numerical grades can be assigned in ascending order, i.e., very early blastocyst (having 50-75 cells), expanded blastocyst (having 100-125 cells), hatching blastocyst, and hatched blastocyst, each of which represents the degree of blastocyst expansion. For example, a grade "4" may represent an expanded blastocyst, while a grade "5" may represent a hatching blastocyst, and a grade "6" may represent a hatched blastocyst.

[0048] However, the quality of the inner cell mass and the quality of the trophectoderm at each of these stages may complicate the scoring system. For example, alphabetical grades may be assigned to represent both the quality of the inner cell mass and the quality of the trophectoderm. Thus, the grade "AA" may represent a good quality inner cell mass and a good quality trophectoderm. However, the grade "AB" may represent a good quality inner cell mass and a lower quality trophectoderm. Thus, the grade "4AA" may represent an expanded blastocyst with a good quality inner cell mass and a good quality trophectoderm. That is, it may be possible that the expanded blastocyst has the highest quality inner cell mass and trophectoderm. Similarly, it may be possible that the hatching blastocyst (e.g., a blastocyst in the process of hatching from the zona pellucida) has a slightly lower quality trophectoderm than the expanded blastocyst. In such a situation, it is difficult to determine, for example, whether a 4AA embryo (representing an expanded blastocyst with the highest quality inner cell mass and trophectoderm) should be considered less viable than a 5AB embryo (representing a hatching blastocyst with slightly lower quality trophectoderm). Thus, embryologists use intuition to make such decisions. It is possible that different embryologists will select different embryos, thereby making it difficult to standardize the selection process.

[0049] Over the years, there have been several techniques that have been introduced with the aim of improving embryo selection. One of those techniques is time-lapse imaging. Using time-lapse imaging, a microscope may capture a sequence of images of an embryo in a cyclical manner. More specifically, a sequence of microscopic images of an embryo may be captured at regular intervals (e.g., 5-20 minute intervals). The idea is to observe cell dynamics and cell behavior by analyzing the cyclical sequence of images captured over time. For example, measurements of events such as cell division timing, multinucleation, and reverse cleavage may be made by observing the cyclical sequence of embryo images. These measurements may be used to select embryos for transfer. While this provides a somewhat more standardized approach to the embryo selection process compared to manual grading, time-lapse imaging requires specialized time-lapse imaging systems, which tend to be expensive. Not all existing microscopes can be adapted for time-lapse imaging. Hence, time-lapse imaging techniques may be hardware-driven; that is, without specialized instrumentation, the technique is difficult to implement. In addition, time-lapse imaging may require that embryos be cultured in specialized Petri dishes. Loading and unloading embryos from such specialized Petri dishes may take longer, thereby increasing the risk of damaging the embryos. The high cost of such instrumentation and other required changes to existing workflows in clinics and laboratories (e.g., using specialized Petri dishes) make it difficult for time-lapse imaging to gain widespread clinical adoption.

[0050] Another technique that has been introduced more recently with the aim of improving embryo selection is preimplantation embryo chromosomal aneuploidy testing (PGT-A). PGT-A may involve performing a biopsy of the trophectoderm and then sequencing the biopsy to determine whether the embryo has the correct number of chromosomes. This may eliminate aneuploid embryos (leading to a failed pregnancy outcome) for transfer, but it does not fully characterize the viability of euploid embryos, as not all euploid embryos may lead to a normal outcome (e.g., normal pregnancy). Studies have shown that within a group of euploid embryos, those with higher quality morphology have a higher likelihood of a normal outcome. Thus, even in PGT-A cycles, euploid embryos may need to be graded to identify suitable embryos for transfer.

[0051] In contrast to existing technologies, the technology described herein provides a data-driven, standardized approach to evaluating embryos that is easy to adopt and cost-effective. For example, the technology described herein can be hardware-independent. Plug-and-play software that can be compatible with all imaging devices, microscopes, microscopic imaging devices, and / or the like (collectively referred to herein as "image capture devices") can enable the image capture device to capture images of embryos in real time. Thus, the technology can be adopted by any clinic or laboratory with already existing hardware (e.g., microscopes) without any additional hardware installation and / or cost burden.

[0052] The techniques described herein may implement deep learning to score embryos according to their likelihood of reaching a clinical pregnancy. For example, the techniques may implement a series of one or more convolutional neural networks to analyze and classify images of embryos. A series of convolutional neural networks may also improve the accuracy of scoring embryos. In some variations, a first convolutional neural network may be trained to segment and crop embryos in images. A second convolutional neural network may be trained to perform quality control. A third convolutional neural network may be trained to perform image classification and scoring. As discussed above, the techniques described herein may be hardware independent. Thus, the convolutional neural networks described herein may be trained to adapt to any type of image capture device and fit into the existing workflow of all clinics and laboratories while improving the accuracy of assessing embryo viability.

[0053] In some variations, to allow the techniques described herein to be compatible with a wide range of image capture devices, the convolutional neural network may be trained with images from different image capture devices (e.g., different microscopes). Images from different image capture devices may have different optics and different resolutions. Thus, when the convolutional neural network is trained with images having different optics and different resolutions, it may be possible that biases are introduced with respect to images from a specific image capture device compared to certain other image capture devices. To overcome this, the techniques described herein extend the training data as further described herein. For example, the training data may include images that may be randomly flipped, rotated, scaled, and / or perturbed (e.g., changing brightness, contrast, and / or saturation) to accommodate different optics and different resolutions.

[0054] As another example, to accommodate more subtle differences in image capture devices used across different clinics (e.g., subtle differences in microscopes), the convolutional neural network may be trained by balancing the occurrence of outcomes per clinic and / or image capture device. For example, if the training data from clinic A has 60% positive pregnancy outcomes, while the training data for the remaining clinics each has only 40% positive pregnancy outcomes, it may be likely that the convolutional neural network may learn to apply a positive bias (e.g., higher scores) for all images from clinic A. This may in turn lead to suboptimal analysis of embryos per facility. Thus, the techniques described herein may resample the training data such that all clinics and / or all image capture devices have the same ratio of positive to negative images to balance the occurrence of outcomes per clinic and / or image capture device.

[0055] The techniques described herein may identify and mitigate other biases in a similar manner (e.g., by balancing the occurrence rates of outcomes). For example, some captured images of an embryo may include an image of a micropipette holding an embryo (e.g., an embryo-holding micropipette). The presence of the micropipette in the images, which may be used as training data, may introduce bias. For example, if the training data includes images with a micropipette and images without a micropipette, it may be likely that the convolutional neural network may learn to apply either a positive bias (e.g., higher score) or a negative bias (e.g., lower score) for all images with a micropipette. The convolutional neural network may, for example, focus almost exclusively on the micropipette in the image, rather than the embryo, to classify and score the image. This may in turn lead to a suboptimal analysis of the embryo, which may be held by the micropipette during imaging. The techniques described herein may resample the training data so that images with a micropipette may have the same ratio of positive to negative images (i.e., the ratio of positive pregnancy training images to negative pregnancy training images) to balance the incidence of outcomes. In a similar manner, bias introduced based on the stage of the blastocyst in the image (e.g., early blastocyst, expanding blastocyst, hatching blastocyst, hatched blastocyst, etc.) may also be identified and mitigated by balancing the incidence of outcomes.

[0056] Optionally, to further improve the accuracy of prediction, the techniques described herein may include patient data such as age, body mass index, donor status, and / or the like. For example, patient age may significantly affect the outcome of implantation, regardless of embryo viability. Thus, incorporating patient data improves the accuracy of assessing embryo viability. Additionally or alternatively, the techniques described herein may include results from genetic testing results, such as prenatal genetic testing, parental genetic testing, etc., to further improve the accuracy of prediction.

[0057] Furthermore, the techniques described herein may analyze, classify, and score single images of an embryo. This is a significant difference from existing time-lapse imaging techniques that collectively analyze a time series of embryo images to score the embryo. In contrast to the analysis of a time series of images, even if multiple images (e.g., not necessarily in time series) of an embryo are captured (such as at different focal planes and / or rotations), the techniques described herein analyze each image individually to generate an overall score for the embryo. For example, each individual image may be classified and scored. The median of the scores across all images may be the final score of the embryo, which represents the viability of the embryo. Alternatively, each individual image may be classified and scored. The median and / or mode of the scores across all images may be the final score of the embryo, which represents the viability of the embryo. This may improve the accuracy of assigning a score to the embryo. For example, even if one image of an embryo is not captured well (e.g., due to the embryologist's choice of focal plane, variations in lighting, etc.), the overall score assigned to the embryo cannot be significantly affected because all images can be individually classified and scored.

[0058] In addition, the Plug and Play software may allow each individual image to be analyzed in real time. For example, the Plug and Play software may allow an embryologist to capture images of multiple embryos in real time. These images may be analyzed and scored in real time. The Plug and Play software may then display an overall viability score of the embryos in real time. In some variations, the images may also be ranked in real time based on the overall viability scores of the embryos in that image. The Plug and Play software may then display the images in the order in which they are ranked. This is in contrast to existing technologies that do not display images of embryos in the order in which they are ranked. Thus, the most viable embryos are displayed first, making it faster to identify and select the most viable embryos for transfer.

[0059] In addition to the above, the technology may perform aneuploidy prediction. Additionally and / or alternatively, the technology may provide for the assessment of frozen and thawed embryos to determine whether the embryo survives the freeze-thaw process. System Overview

[0060] FIG. 1 illustrates an overview of an example variation of a system 100 for assessing embryo viability. The system 100 may be employed by any clinic and / or laboratory 102 (referred to herein as a "clinic") with existing hardware, such as an image capture device 104. The image capture device 104 may capture one or more images of an embryo. An application 106 running on a computing device in the clinic 102 may provide a real-time communication link between the image capture device 104 and a controller 108. The controller 108 may use artificial intelligence to analyze the images of the embryo to assess embryo viability. In some variations, the controller 108 may optionally incorporate data 110 to improve the accuracy of the assessment. The controller 108 may score the embryo in each individual image. The controller 108 may assess the viability of the embryo based on the score assigned to the embryo in each image. The overall viability score of the embryo may be used to rank the images of the embryo. The overall viability score and the order in which the images are ranked may be transmitted to the patient application 112, the clinician application 114, and the data portal 116. The patient application 112, the clinician application 114, and the data portal 116 may each display the images in the order in which they are ranked.

[0061] As discussed above, any clinic 102 may adopt the system 100 into their existing workflow. The clinic 102 may be, for example, any laboratory, fertility center, or clinic that provides IVF treatment. The clinic 102 may include infrastructure for culturing embryos. For example, the clinic 102 may include critical equipment required for assisted reproductive technology, such as incubators, micromanipulator systems, medical refrigerators, freezing machines, petri dishes, test tubes, 4-well culture dishes, pipettes, embryo transfer catheters, needles, etc. In addition, the clinic 102 may provide a stable, non-toxic, pathogen-free environment for culturing embryos.

[0062] An existing image capture device 104 in the clinic 102 may capture one or more images of the embryo. The image capture device 104 may have any suitable optical system and any suitable resolution. The image capture device 104 may be a microscope, a microscope imaging device, or any other suitable imaging device capable of capturing images of the embryo. For example, the image capture device 104 may be any suitable microscope, such as a bright field microscope, a dark field microscope, an inverted microscope, a phase contrast microscope, a fluorescent microscope, a confocal microscope, an electron microscope, etc. Additionally or alternatively, the image capture device 104 may be any suitable device operably coupled to a microscope camera capable of capturing digital images of the embryo. For example, the image capture device 104 may include a microscope camera operably coupled to a handheld device (e.g., a computer tablet, a smartphone, etc.), a laptop, a desktop computer, etc. In yet another alternative variation, the image capture device 104 may be any suitable computing device (e.g., a computer tablet, a smartphone, a laptop, and / or the like) running a microscope application capable of capturing images of the embryo.

[0063] Application software 106 (referred to herein as an "application") executing on a computing device in the clinic 102 may enable the image capture device 104 to capture one or more images of the embryos. Some non-limiting examples of computing devices include computers (e.g., desktop, personal computers, laptops, etc.), tablets and e-readers (e.g., Apple iPad®, Samsung Galaxy® Tab, Microsoft Surface®, Amazon Kindle®, etc.), mobile devices and smartphones (e.g., Apple iPhone®, Samsung Galaxy®, Google Pixel®, etc.), and the like.

[0064] In some variations, the application 106 (e.g., a web app, a desktop app, a mobile app, etc.) may be pre-installed on the computing device. Alternatively, the application 106 may be rendered on the computing device in any suitable manner. For example, in some variations, the application 106 (e.g., a web app, a desktop app, a mobile app, etc.) may be downloaded onto the computing device from a digital distribution platform, such as an app store or application store (e.g., Chrome® Web Store, Apple® Web Store, etc.). Additionally or alternatively, the computing device may render a web browser (e.g., Google®, Mozilla®, Safari®, Internet Explorer®, etc.) on the computing device. The web browser may include a browser extension, browser plug-in, etc. that may render the application 106 on the computing device. In yet another alternative variation, the browser extension, browser plug-in, etc. may include installation instructions for installing the application 106 on the computing device.

[0065] The application 106 may be plug-and-play software that may be compatible with any type of computing device. Additionally, the application 106 may be compatible with any type of image capture device 104. In some variations, the application 106 may include live viewer software that may display an image of the embryo as seen through the image capture device 104. For example, traditionally, an image capture device 104, such as a microscope, is used to view the embryo. However, with the live viewer software (included within the application 106), the computing device may display an image (e.g., a two-dimensional image) of the embryo as seen through the image capture device 104. A user (e.g., an embryologist) may view the image of the embryo on a display of a computing device running the application 106. The application 106 may allow a user to select an image that the user desires to capture. The application 106 may transmit instructions to the image capture device 104 to capture the selected image. In some variations, the application 106 may perform a quality check before transmitting instructions to the image capture device 104 to capture the selected image. For example, application 106 may analyze the selected image to determine whether the characteristics (e.g., resolution, brightness, etc.) of the selected image would allow for further analysis. In response to satisfying the quality check, application 106 may transmit a command to image capture device 104 to capture the selected image. Alternatively, application 106 may first transmit a command to image capture device 104 to capture the selected image. Once the selected image is captured, application 106 may analyze the captured image to determine whether the characteristics (e.g., resolution, brightness, etc.) of the captured image would allow for further analysis.

[0066] To allow a user to select one or more images for capture, the application 106 may render a widget (e.g., a capture button) when the application 106 is executed on a computing device. The widget may be designed such that a user may interact with the widget. The widget may be pressed or clicked (e.g., via a controller such as a touch screen or a mouse). When a user desires to select an image for capture, the user may press or click on the widget. In response to the press or click, the image capture device 104 may capture that particular image of the embryo. The user may choose to capture multiple images by repeatedly pressing or clicking the widget. In some variations, the widget (e.g., the capture button) may be a standalone button in any suitable shape (e.g., in the form of a circle, oval, rectangle, etc.). In some variations, an object-oriented programming language such as C++ may be used to design and execute the application 106.

[0067] As discussed above, the application 106 may provide a real-time communication link between the image capture device 104 and the controller 108. Thus, captured images of the embryo may be transmitted in real-time to the controller 108 via the application 106. In some variations, the controller 108 may include one or more servers and / or one or more processors running on a cloud platform (e.g., Microsoft Azure®, Amazon® Web Services, IBM® Cloud Computing, etc.). The servers and / or processors may be any suitable processing device configured to run and / or execute a set of instructions or code and may include one or more data processors, image processors, graphics processing units, digital signal processors, and / or central processing units. The servers and / or processors may be, for example, general purpose processors, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and / or the like.

[0068] In some variations, the controller 108 may be included within a computing device on which the application 106 may be executed (e.g., to locally perform one or more processes described herein). Alternatively, the controller 108 may be separate from and operatively coupled to a computing device on which the application 106 may be executed either locally (e.g., the controller 108 located at the clinic 102) or remotely (e.g., as part of a cloud-based platform). In some variations, the controller 108 may include a processor (e.g., a CPU). The processor may be any suitable processing device configured to run and / or execute a set of instructions or code and may include one or more data processors, image processors, graphics processing units, physical processing units, digital signal processors, and / or central processing units. The processor may be, for example, a general-purpose processor, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), and / or the like. The processor may be configured to launch and / or execute application processes and / or other modules, processes, and / or functions associated with the system and / or its associated networks. The underlying device technologies may be provided in a variety of component types (e.g., MOSFET technologies such as complementary metal oxide semiconductor (CMOS), bipolar technologies such as emitter coupled logic (ECL), polymer technologies (e.g., silicon-conjugated polymer and metal-conjugated polymer-metal structures), hybrid analog and digital, and / or the like).

[0069] The controller 108 may use artificial intelligence to assess embryo viability. For example, the controller 108 may implement one or more convolutional neural networks to analyze and classify the captured images. More specifically, the convolutional neural networks may analyze and classify each captured image to assess embryo viability.

[0070] It should be readily apparent that a user may choose to capture multiple images of a particular embryo, but these images may not necessarily be time-lapse images (e.g., in a time series). For example, in time-lapse imaging, two or more images of an embryo are captured in a sequence at periodic or intermittent time intervals (e.g., 5-20 minute time intervals). To capture time-lapse images, a time-lapse microscope may be required. In contrast, as discussed above, system 100 is compatible with any type of image capture device 104. Thus, system 100 may not necessarily capture images in a sequence at periodic time intervals. While this may be one possible variation of system 100, system 100 described herein may capture multiple images in any suitable manner (due to its compatibility with various types of image capture devices). For example, a second image of an embryo may be captured 3 seconds after a first image of the embryo is captured. However, a third image of the embryo may be captured 5 seconds after the second image, and a fourth image of the embryo may be captured 2 seconds after the third image. Thus, even if multiple images of an embryo may be captured and analyzed, these images may not be time lapse images. In an exemplary variation, multiple images of an embryo (e.g., at least two consecutive images) may be captured within a time interval of about 60 seconds or less. For example, two or more consecutive images of an embryo may be captured within a time interval ranging from about 1 second to about 60 seconds, about 5 seconds to about 60 seconds, about 10 seconds to about 60 seconds, about 20 seconds to about 60 seconds, about 30 seconds to about 60 seconds, about 1 second to about 30 seconds, about 1 second to about 20 seconds, about 1 second to about 10 seconds, or about 1 second to about 5 seconds. Additionally, unlike time lapse images, which capture a set number of images in a sequence at periodic time intervals for each embryo, system 100 may capture a different number of images for each embryo. For example, time lapse imaging may capture three images in a sequence at periodic intervals for each embryo to determine the viability of the embryo. In contrast, system 100 may capture three images of a first embryo and two images of a second embryo. System 100 may determine the viability of the first embryo from the three images and the viability of the second embryo from the two images.Thus, the number of images captured for an embryo may vary for different embryos.

[0071] The convolutional neural network may individually analyze and classify each captured image to generate an overall viability score for the embryo. For example, if the application 106 uses the image capture device 104 to capture three images of a day 5 embryo, each of the three images may be individually and separately evaluated by the convolutional neural network. For purposes of discussion herein, unless expressly indicated otherwise, the terms "an image," "each image," "the image," "a captured image," "each captured image," "the captured image," "a separate image," and "an individual image" may be considered to be a single individual image. The evaluation of each image may generate an individual score for the embryo that may be associated with each individual image of the three images. The overall score of the embryo may be a function of each individual score associated with each individual image. For example, the overall score of the embryo may be a representative value of the three individual scores associated with the three individual images. Alternatively, in another embodiment, the overall score for the embryo may be the median of the three individual scores associated with the three individual images. In some variations, the overall viability score for the embryo generated by the convolutional neural network may indicate the likelihood that the embryo will result in a clinical pregnancy if transferred (e.g., the likelihood of a normal outcome).

[0072] The convolutional neural network may include a series of convolutional neural networks to perform one or more of the following: (1) image segmentation and image cropping, (2) quality control, (3) image classification, and (4) optionally incorporating data 110 to generate an accurate overall embryo viability score. The series of convolutional neural networks may be implemented by a server and / or processor. For example, the server and / or processor may include software code for implementing each of the convolutional neural networks. More specifically, each convolutional neural network may be included in the software code as a separate module. When the server and / or processor executes the software code, the individual modules may generate instructions for performing (1) image segmentation and image cropping, (2) quality control, (3) image classification, or (4) incorporating data 110. Additionally or alternatively, the software code may include calls to separate modules implementing individual convolutional neural networks. A call to a specific module may redirect processing performed by the server and / or processor to implement the specific convolutional neural network contained within that module. In some variations, two or more convolutional neural networks may be implemented simultaneously by the server and / or processor. Alternatively, convolutional neural networks may be implemented in series, one after the other. In some variations, the convolutional neural networks may be implemented and / or trained using PyTorch or Tensorflow.

[0073] As discussed above, in some variations, the system 100 may incorporate data 110 (e.g., patient data and / or embryo data) to refine the overall viability score assigned to the embryo. The data 110 may include patient data associated with one or more patients, such as, for example, the patient's age, the patient's body mass index, and / or the like, and / or embryo data. For example, the patient data may include the age of the patient undergoing IVF treatment. This may have an impact on the pregnancy outcome. It may be possible that two similarly scored embryos may lead to different pregnancy outcomes depending on the patient's age. Additionally or alternatively, the patient data may include data related to one or more donors associated with the patient, such as the donor's age, the donor's status, the donor's body mass index, and / or the like. For example, the patient data may include the patient's body mass index. This may act as a contributing factor to the health of the embryo. As another example, the patient data may include an indication of whether the patient is a first-time patient. If not, the patient data may additionally include whether an embryo associated with the patient has ever had a normal outcome. In this manner, the patient data may enable the fertility specialist, embryologist, and / or clinician to personalize the IVF treatment. In some variations, the patient data may include data related to one or more genetic test results, such as prenatal genetic test results, embryo level genetic test results, parental genetic test results, etc. Additionally or alternatively, in some variations, the data 110 may include embryo data, such as genetic test results for aneuploidy, pharmacokinetics for disease, potential future traits, gender, etc. Furthermore, in some variations, other embryo-specific data, such as the date an image of the embryo was captured, the date the embryo is implanted, and / or the like, may be used to further refine the accuracy of the prediction.

[0074] For purposes of discussion herein, data 110 may refer to, for example, (1) data associated with one or more patients and / or one or more donors, which may include description, content, record value, combinations thereof, and / or the like, and / or (2) metadata that provides context for the data. For example, data 110 may include one or both of data and metadata associated with patient records and / or donor records. Data 110 may be extracted from trusted electronic medical records. For example, system 100 may include, for example, a system for providing eIVF TM Patient Portal, Artisan TM Infertility treatment portal, Babysentry TM Management system, EPIC TM IDEAS, the patient portal from Mellowood Medical TM The medical record manager may access one or more third party databases, which may include electronic medical records such as, for example, the medical record management software, or any suitable electronic medical record management software.

[0075] As discussed above, the convolutional neural network implemented on the controller 108 may score the embryos according to their likelihood of reaching a clinical pregnancy (e.g., generating an overall viability score) and, in some variations, may also rank the images (e.g., ranking each image based on the overall viability score of the embryo in that image). The individual overall viability scores and the order in which the images are ranked may be transmitted to the patient application 112, the clinician application 114, and the data portal 116. In some variations, the patient application 112 may run on a computing device (e.g., a computer, tablet, e-reader, smartphone, mobile device, and / or the like) associated with the patient. The patient may access the patient application 112 on the computing device to view the overall viability scores for the embryos and the ranks of the images. The patient application 112 may display the images of the embryos in order of their rank. Thus, the most viable embryos may appear first on the display. This makes it easier for patients to identify the most viable embryos and make important decisions related to their IVF treatment.

[0076] In a similar manner, the clinician application 114 may execute on a computing device (e.g., a computer, tablet, e-reader, smartphone, mobile device, and / or the like) associated with a clinician (e.g., an embryologist, a fertility specialist, etc.). The clinician may access the clinician application 112 on the computing device to view the overall viability score of the embryos and the rank of the images. The clinician application 112 may display the embryos in order of their rank. In some variations, the clinician application 114 may be identical to the application 106 described above. For example, the application 106 that enables the image capture device 104 to capture images of the embryos may also display the overall viability embryo score and the order in which the images are ranked after the embryos are evaluated by the controller 108. Equivalently, in addition to displaying the overall viability and the order in which the images are ranked, the clinician application 114 may enable the image capture device 104 to capture images of the embryos. Alternatively, the clinician application 114 may be different from the application 106 described above. For example, the clinician application 114 may run on a computing device that may be different from the computing device that runs the application 106.

[0077] The data portal 116 may be data collection software that may store the scores (e.g., scores associated with each individual image and overall viability score for the embryo) and / or ranks generated by the controller 108. The collected data may be analyzed in the data portal 116 to further improve the accuracy of the system 100. For example, the collected data may be processed and provided as additional training data to the convolutional neural network implemented by the controller 108. Thus, the convolutional neural network may become more intelligent and further enhance the accuracy of predicting embryo viability. In some variations, the data portal 116 may be connected to one or more databases. The databases may store the scores (e.g., scores associated with each individual image and overall viability score for the embryo), ranks, patient data, and / or other relevant data related to the embryo. In some variations, the data portal 116 may be connected to a memory that stores these databases. Alternatively, the data portal 116 may be connected to a remote server that may store these databases. In some variations, the results from the controller 108 are stored in the eIVF TM Patient Portal, Artisan TM Infertility treatment portal, Babysentry TM Management system, EPIC TM IDEAS, the patient portal from Mellowood Medical TM The results may be transmitted to one or more third party databases, which may include electronic medical records, etc. These results may include an overall viability score, rank, etc. for the embryo. Exemplary Methods for Assessing Embryonic Viability

[0078] FIG. 2 is a flow diagram illustrating an example variation of a high-level overview of a method 200 for assessing embryo viability using artificial intelligence. In some variations, the method 200 may be implemented using a system such as the system 100 described in FIG. 1. At 202, the method 200 may include capturing one or more images of the embryo. In particular, the image of the embryo may be an image captured once the embryo reaches the blastocyst stage (e.g., 5, 6, and / or 7 days after fertilization). That is, the captured image may be a microscopic image of the blastocyst. In some variations, the embryo may be evaluated at any of the different stages of the freeze-thaw process, such as prior to freezing or after freezing and thawing. Additionally or alternatively, the embryo may be evaluated prior to biopsy. In other variations, the embryo is not intended to undergo biopsy or freezing.

[0079] At 204, each image may be individually analyzed and classified in real time by at least one deep convolutional neural network (D-CNN). The D-CNN may be implemented on a controller, such as controller 108 of FIG. 1. For example, the D-CNN may evaluate a single image in real time. If multiple images of an embryo (e.g., multiple images of a blastocyst) are captured, each image may be individually analyzed and classified to generate an overall viability score for the embryo. The D-CNN may also rank the images based on the overall viability score of the embryo in the image. In some variations, the D-CNN may incorporate patient data 206 (e.g., patient-specific metadata), such as age, body mass index, donor status, etc., to improve the accuracy of the overall score assigned to the embryo.

[0080] At 208, the method 200 may predict the likelihood of the embryo achieving a clinical pregnancy based on the embryo's overall viability score generated by the D-CNN. In some variations, the overall viability score indicating the likelihood of a clinical pregnancy (e.g., a normal outcome) may be displayed on one or more displays. In some variations, the image rank (e.g., the image rank determined by the D-CNN) may also be displayed. For example, the embryo images may be displayed in order of their rank. In this manner, a clinician (e.g., an embryologist, a fertility specialist, a clinician, etc.) may select an embryo for transfer in real time in consultation with a patient based on the embryo's overall viability score generated by the D-CNN and the image rank. Embryo image capture

[0081] As discussed above, the techniques disclosed herein may be adopted by any clinic with already existing hardware (e.g., clinic 102 of FIG. 1). The techniques disclosed herein may be compatible with any type of image capture device (e.g., image capture device 104 of FIG. 1) and may be adopted by clinics without the need to change their existing workflow. Thus, plug-and-play software such as application 106 of FIG. 1 may enable clinicians to capture one or more images of an embryo using an existing image capture device. FIGS. 3A-3H illustrate example variations of a graphical user interface (GUI) that may be part of the plug-and-play software rendered on a display of a computing device to capture images of an embryo. In some variations, the computing device may be pre-installed with the application. Alternatively, the application may be downloaded onto the computing device from a web store. In yet another alternative variation, rendering a web browser with a specific browser extension may render the application on the computing device.

[0082] The application may cause the computing device to display various functionality. For example, the application may cause the computing device to manage image capture and other actions associated with a patient. As an illustrative example, as shown in FIG. 3A, the application may render a display 350, which may include a dashboard 351. The dashboard 351 may represent an initial step in a workflow toward determining embryo viability. In some variations, the dashboard 352 may include some information of various patients (e.g., “Ashley Smith” 353a, “Jane Doe” 353b, etc.). For example, the dashboard 351 may include the patient ID of the patient (e.g., represented in FIG. 3A as “Patient ID”), the number of transfer cycles associated with the patient (e.g., represented in FIG. 3A as “Cycles”), the status of the embryo transfers associated with the patient (e.g., represented in FIG. 3A as “Status”), etc.

[0083] In some variations, the display 350 may include a widget designed for user interaction, such as a widget "New Patient" 352. For example, by clicking and / or pressing on "New Patient" 352, a user may add a patient not yet listed on the dashboard 351, including information associated with the patient (e.g., patient ID, number of cycles, status, etc.). In some variations, the information may be manually entered by a user interacting with the display 350. Alternatively, the information may be extracted from a database (e.g., a third-party electronic medical record database). For example, an application may interact with an electronic medical record database to access and extract information related to the patient. The widget "New Patient" 352 in FIG. 3A may be illustrated as a button that is oval in shape, but it should be readily apparent that any suitable widget may be provided. For example, "New Patient" may be a circular shaped widget, a triangular shaped widget, etc.

[0084] To access information for a specific patient, a user may press and / or click on a row containing the patient's information. For example, by pressing and / or clicking on a row containing information related to "Ashley Smith" 353a, the application may transition from display 350 of FIG. 3A to another display (e.g., display 360 of FIG. 3B) that may include more specific information associated with "Ashley Smith" 353a.

[0085] The display 360 of FIG. 3B may include additional information (e.g., represented as “Cycle Info” 363) associated with the cycle of the patient (e.g., “Ashley Smith” 353a). This information may include patient ID, age, body mass index, whether the eggs belong to a donor, etc. The display 360 may further include past data and / or history (e.g., represented as “Cycle History” 364) associated with the current cycle and / or previous cycles. For example, this may include embryo scores associated with past cycles and corresponding outcomes of the cycles. If no embryo images are associated with the current cycle (e.g., the application does not have any embryo images for “Ashley Smith” 353a in the current cycle), the display 360 may include an option to capture a new image or upload a new image (e.g., from an existing database). For example, the display 360 may include one or more widgets designed for user interaction, such as the widget “Capture New Image” 361, “Upload New Image” 362, etc. By clicking and / or pressing on "Capture New Image" 361, the user may use the application to instruct the image capture device to capture an image of the embryo. More specifically, the application may transition from display 360 of FIG. 3B to display 370 of FIG. 3C. By clicking and / or pressing on "Upload New Image" 362, the user may upload an already captured image of the embryo to the application. Although the widget "Capture New Image" 361 and widget "Upload New Image" 362 of FIG. 3B may be illustrated as buttons that are oval in shape, it should be readily apparent that any suitable widgets may be provided.

[0086] The application may cause the computing device to display one or more user interface elements to facilitate capture of an embryo image. For example, the application may include live viewer software for displaying an image of an embryo as viewed through an image capture device (e.g., a microscope). As an illustrative example, as shown in FIG. 3C, image 302 is an image of an embryo as viewed through a microscope. A widget designed for user interaction, such as capture button 313, may enable a user to capture image 302 via an image capture device. For example, in FIG. 3C, the widget may be capture button 313 including the text "Capture". For example, by clicking and / or pressing on capture button 313, an image, such as image 302, may be captured. In this manner, one or more images of an embryo may be captured in real time using the plug-and-play software. Display 370 may also include previously captured images and their individual scores. For example, as seen in FIG. 3C, image 302 may be the fourth image to be captured for a specific patient (e.g., "Ashley Smith" 353a). Three other images may be displayed below the captured image 315. Some of these three images may be of the same embryo, while other images below the image capture 315 may be of different embryos. Additionally or alternatively, some of the three images may be captured on the same day, while some other of the three images may be captured on different days. In FIG. 3C, the three images are of different embryos. The viability score for each embryo is represented adjacent to the individual images. The images 315 may include information regarding the embryo ID and individual viability score associated with the image. Additionally, the user may type notes related to the image via display 370.

[0087] 3D illustrates another variation of a display 375 with one or more user interface elements to facilitate capture of an embryo image. As discussed above, the application may include live viewer software for displaying images of the embryo as viewed through an image capture device (e.g., a microscope). In FIG. 3D, image 302a is a first image of the embryo as viewed through the microscope.

[0088] Unlike Figure 3C, widgets designed for user interaction, such as capture button 313 in Figure 3D, may include an icon (e.g., a camera icon) instead of the text "Capture." By clicking and / or pressing on capture button 313, a first image, such as image 302a, may be captured. Alternatively, the user may click and / or press widget 314 to upload a previously captured image of an embryo.

[0089] To capture multiple images of the same embryo, the user may click and / or press the capture button 313 multiple times. Additionally or alternatively, in response to the user clicking, pressing, and / or holding the capture button 313 for at least a predetermined duration, the application may capture multiple images of the embryo in succession (e.g., burst mode). Additionally or alternatively, the capture button 313 may be associated with a timer such that in response to the user clicking, pressing, and / or holding the capture button, the application may capture multiple images within a time period set, apportioned, or predetermined by the timer.

[0090] As discussed above, the application may capture multiple images of the same embryo. Each image may be scored individually. In some variations, one or more outlier images may be flagged, rejected, and / or eliminated. For example, images of an embryo with a viability score that is significantly different (e.g., differs by at least a predetermined threshold from the viability score associated with all other images of the same embryo, the average viability score of other images of the same embryo, etc.) may be automatically flagged for scrutiny (e.g., by a user), automatically rejected, excluded from the characterization of the embryo but still present for viewing, and / or automatically discarded or deleted entirely. The overall viability score of the embryo may be a mathematical function (e.g., mean, median, mode, etc.) of each of the individual scores. As the user captures images of the embryo, the application scores the images in real time. In some variations, these captured images and / or scores may be displayed to the user in real time. For example, the first image 302a captured in FIG. 3D may be displayed as the already captured image 315a in FIG. 3E. The score of the first image 302a captured in Figure 3D may be displayed adjacent to image 315a (e.g., identical to image 302a) in Figure 3E. In Figure 3E, image 315a has a score of 0.52. A second image 302b of the embryo may be viewed through the Live Viewer software. The second image 302b may be captured by clicking and / or pressing on the capture button 313 or by clicking and / or pressing on the widget 314 to upload a previously captured image of the embryo.

[0091] The first image 302a captured in FIG. 3D and the second image 302b captured in FIG. 3E may be displayed as image 315a and image 315b in FIG. 3F, respectively. The score of the first image 302a captured in FIG. 3D may be displayed adjacent to image 315a (e.g., identical to image 302a) in FIG. 3F. As seen in FIG. 3E, the score of image 315a is 0.52. Similarly, the score of the second image 302b captured in FIG. 3E may be displayed adjacent to image 315b (e.g., identical to image 302b) in FIG. 3F. In FIG. 3F, the score of the second image 315b is 0.62. The overall viability score of the embryo may be displayed below image 315a and image 315b. For example, in FIG. 3F, the overall viability score (e.g., 0.57) is representative of the score of image 315a (e.g., 0.51) and the score of image 315b (e.g., 0.62). In other words, a running average viability score for the embryo may be calculated in real time based on multiple captured images of the embryo as the images are captured and uploaded as images are captured and / or deleted. A third image 302c of the embryo may be viewed through the live viewer software. The third image 302c may be captured by clicking and / or pressing on the capture button 313 or by clicking and / or pressing on the widget 314 to upload a previously captured image of the embryo.

[0092] The first image 302a captured in FIG. 3D, the second image 302b captured in FIG. 3E, and the third image 302c captured in FIG. 3F may be displayed as image 315a, image 315b, and image 315c in FIG. 3G, respectively. The score of the first image 302a captured in FIG. 3D may be displayed adjacent to image 315a (e.g., identical to image 302a) in FIG. 3G. As seen above, the score of image 315a is 0.52. Similarly, the score of the second image 302b captured in FIG. 3E may be displayed adjacent to image 315b (e.g., identical to image 302b) in FIG. 3F. As seen above, the score of image 315b is 0.62. Similarly, the score of the third image 302c captured in FIG. 3F may be displayed adjacent to image 315c (e.g., identical to image 302c) in FIG. 3G. The score for image 315c is 0.63. The updated overall viability score of the embryo may be displayed below image 315a, image 315b, and image 315c. For example, in FIG. 3G, the overall viability score (e.g., 0.59) is a representation of the score for image 315a (e.g., 0.51), the score for image 315b (e.g., 0.62), and the score for image 315c (e.g., 0.63). A fourth image 302d of the embryo may be viewed through the live viewer software. The fourth image 302d may be captured by clicking and / or pressing on the capture button 313 or by clicking and / or pressing on the widget 314 to upload a previously captured image of the embryo. Although FIG. 3G illustrates a fourth image 302d that may be captured, it should be understood that any suitable number of images (e.g., one, two, three, four, five, or more) may be captured and displayed for the embryo and / or provide the basis for the embryo's overall viability score.

[0093] If the user chooses to capture an image of a different embryo, the user may press and / or click on the new embryo widget 316. The application may transition from display 380 of FIG. 3G to display 381 of FIG. 3H. The live viewer software may display an image of another embryo (e.g., image 302e) as seen through an image capture device (e.g., a microscope). The overall viability score of an already scored embryo (e.g., an embryo of FIGS. 3D-3F) may be displayed adjacent to image 382a. For example, image 382a may be one of the first image 302a of a previously scored embryo captured in FIG. 3D, the second image 302b of a previously scored embryo captured in FIG. 3E, or the third image 302c of a previously scored embryo captured in FIG. 3F. The overall viability score (e.g., 0.59 as seen in FIG. 3G) may be displayed adjacent to image 382a. The user can repeat the process as outlined in Figures 3D-3G to determine an overall viability score for the embryo seen in Figure 3G. Assessment of embryo viability

[0094] Once an image is captured, it may be transmitted in real-time to a controller (e.g., controller 108 of FIG. 1) for evaluation. The controller may implement one or more convolutional neural networks (CNNs) to assess and classify the image in real-time. In some variations, the CNN may be a series of CNNs. FIG. 4 is a flow diagram illustrating an example variation of a method 400 for assessing embryo viability using a series of CNNs.

[0095] At 402, the controller may receive an input image, such as image 302 of FIG. 3 captured by the plug-and-play software. The input image may be analyzed by a series of CNNs implemented to perform (1) image segmentation and image cropping at 404, (2) optionally quality control at 406, (3) image classification at 408, and (4) optionally incorporating patient data at 410. At 412, the method 400 may predict embryo viability based on output from image classification (e.g., at 408) and optionally based on output from incorporating patient data (e.g., at 410). In some variations, embryo viability represents the likelihood (e.g., a value between 0 and 1) that the embryo will achieve clinical pregnancy. In some variations, positive or negative fetal cardiac activity may be considered an indicator of clinical pregnancy.

[0096] A CNN typically consists of one or more convolutional layers for extracting features from an image. The convolutional layers may include filters (e.g., weight vectors) for detecting specific features. The filters may be shifted stepwise across the height and width dimensions of the input image to extract features. The shifting of the filters (i.e., application of the filters at different spatial locations) provides translation invariance. For example, if features representing the boundaries of an embryo appear at a first spatial location in one image, and the same features appear at a second, different spatial location in another image, these features can be extracted from both the first and second spatial locations due to the translation invariance of the CNN. Thus, translation invariance provides a feature space in which the encoding of the input image may have enhanced stability against visual variations. That is, even if the embryo is translated and / or rotated slightly from one image to another, the output values ​​do not vary significantly.

[0097] As discussed above, a series of CNNs may be implemented to extract features and / or classify the input image. These CNNs and their architectures are further described below. Image Cropping

[0098] A CNN implementing image segmentation and image cropping (e.g., at 404 in FIG. 4) may take an input image of an embryo to generate an output image that is cropped to the boundaries of the embryo. FIG. 5 is an example variation of implementing a CNN on an input image of an embryo for image cropping and image segmentation. In an example variation, a U-Net501 architecture may be used for image segmentation and image cropping. The U-Net501 architecture may provide precise and fast segmentation of the embryo. The left portion of the U-Net501 architecture may include a contraction path that may generate a low-dimensional representation of the input image, and the right portion of the U-Net501 architecture may include an augmentation path that may upsample the low-dimensional representation to generate a segmentation map.

[0099] The trained U-Net501 architecture may generate a U-Net mask 504 for embryo segmentation. The U-Net mask 504 may be a ground truth binary segmentation mask. The U-Net501 may compare the input image 502 to the U-Net mask 504 and create a square crop around the embryo in the input image. The U-Net501 may then generate an output image 506 cropped to the embryo boundary. In an exemplary variation, the hyperparameters for the U-Net501 architecture may include 40 epochs, lr=0.0005, and batch=32.

[0100] In alternative variations, the CNN architecture for image cropping and image segmentation may include any suitable CNN, such as Mask R-CNN, a fully convolutional network (FCN), etc. Quality Control

[0101] The output image (e.g., output image 506 of FIG. 5) generated by the CNN implementing image cropping and image segmentation may act as an input to a CNN performing quality control. Quality control of the image may include verifying whether the image contains an embryo and / or determining the probability that the embryo is a blastocyst. As a first step to performing such verification, the CNN performing cropping may eliminate images that do not have a foreground result (e.g., do not depict any embryo) since the image has already been cropped and segmented (e.g., using U-Net 501 of FIG. 5) to the boundaries of the embryo. Alternatively, the CNN performing cropping may transmit an alert, such as a warning message and / or a warning signal, to the user (e.g., via the application 106 executing on the computing device) to eliminate images that do not have a foreground result (e.g., do not depict any embryo). In yet another alternative variation, the CNN performing cropping may transmit a modification (e.g., via the application 106 executing on the computing device) to change the image to include an embryo in the image and improve the quality of the image. This may act as a first filter. The remaining images (i.e., those that were not rejected) may be verified by implementing the quality control CNN shown in FIG. 6.

[0102] FIG. 6 is an exemplary variation of implementing a CNN for performing quality control. In an exemplary variation, an autoencoder may be used to perform quality control. The architecture may include five layers of two-dimensional convolution, including a convolution layer, a pooling layer, an input layer, and an output layer. For example, the five layers of two-dimensional convolution may include stride, batch normalization, and ReLU activation. Layer 601a may represent an encoder, 601b may represent a latent space, and layer 601c may represent a decoder of the autoencoder. As seen in FIG. 6, the dimensionality of the image may first be reduced (e.g., encoder layer 601a). From this reduced encoding, the CNN may reconstruct the image (e.g., using decoder layer 601c). The latent space 601b may represent a compressed state of the data. The autoencoder may identify outliers in the latent space 601b. Due to the compression and subsequent reconstruction, the model can be cleaned of any extraneous noise and thereby focus on important features. Figure 6 shows an example input 602 and a corresponding reconstructed output 606 generated by implementing the autoencoder shown in Figure 6.

[0103] In an example variation, the hyperparameters for the autoencoder may include 200 epochs, lr=0.003, and batch=32. The trained latent space may be N=4096.

[0104] FIG. 7 illustrates an example deployment of a convolutional neural network (e.g., the autoencoder of FIG. 6) performing quality control. The autoencoder may extract a learned latent space for 100 random training images. The autoencoder may then take a representative of the latent space. For example, in the example provided in FIG. 7, the representatives generate N=2000. A latent space clustering method may be used to identify outliers. For example, the distance (e.g., cosine similarity) between sample points and reference points in the latent space may be calculated. It may be observed that similar data points cluster together. For example, points with a cosine distance approximately equal to 1 may be similar data points that cluster together in graph 702 (e.g., as cluster 702a). Points with a cosine distance below a threshold (e.g., below 0.87 in FIG. 7) may be outliers (e.g., outlier 702b). Images of embryos with high similarity that cluster together (e.g., corresponding to point 702a) are shown in 704a. An image of an embryo that may be an outlier (e.g., corresponding to point 702b) is shown in 704b. Any suitable CNN capable of performing quality control as described above may be used (e.g., a regular CNN classifier). Image Classification

[0105] After cropping and segmenting the input image, and optionally performing quality control in some variations, the image may be classified using another CNN. Figure 8 is an exemplary variation that implements a CNN for image classification and score generation. The CNN may be trained to classify the image to generate an image score 801d (e.g., a viability score). The image score 801d may indicate the probability that an embryo in the image will achieve a clinical pregnancy, the probability that an embryo in the image will achieve a live birth, and / or the like, based on an evaluation of the image.

[0106] In an exemplary variation, the resnet-18 model 801a architecture may be used in conjunction with transfer learning for image classification. The resnet-18 model 801a may be a residual network that is 18 layers deep. The resnet-18 model 801a may include one or more residual blocks. The discriminative mapping created by the residual blocks may allow the residual blocks to omit connections without affecting the performance of the residual network. Transfer learning may allow the resnet-18 model 801a to transfer knowledge learned by performing similar tasks for different datasets. That is, the resnet-18 model 801a may be a pre-trained model that has been pre-trained on a different dataset (e.g., ImageNet). Transfer learning may be performed to fine-tune the resnet-18 model 801a with respect to some or all layers and reuse the resnet-18 model 801a to classify images of embryos. In some variations, a shallow architecture of the resnet-18 model 801a as shown in Figure 8 may be implemented. This may minimize the risk of overfitting and reduce computational requirements. In alternative variations, modified versions of DenseNet, Inception, Alexnet, and / or GoogLeNet may be used for image classification instead of the resnet model.

[0107] In some variations, as an optional step, patient data may be incorporated to improve the accuracy of predicting embryo viability. In some variations, variables such as patient age, body mass index, and / or donor status may be obtained from an electronic medical record. Patient data may be incorporated by concatenating each image score 801d (e.g., a viability score generated for an embryo in a specific image) with corresponding patient data and / or patient metadata. A small feedforward neural network or logistic regression model 801c may incorporate these concatenated image scores and patient data. For example, a feedforward neural network 801c may be trained on the concatenated values ​​of image scores and patient data (more details on training a CNN below). The feedforward neural network 801c may include layers with batch normalization, ReLU, and dropout. The feedforward neural network 801c may then generate a final score representing the likelihood of a normal pregnancy for the embryo in the cropped and classified specific image. In other implementations, the patient data can be concatenated with the final feature vector layer in the image classification model 801a for parallel training on image and patient data.

[0108] If more than one image of an embryo is captured, the overall viability of the embryo may be a function of individual viability scores that may be generated for each image. For example, the overall viability score may be the average and / or median of the individual viability scores generated for each image. In this manner, a series of CNNs may be used to assess embryo viability. Viewing the output

[0109] As discussed above, plug-and-play software, such as application 106 of FIG. 1, may capture images of the embryo in real time. Each image may be evaluated in real time by one or more convolutional neural networks (CNNs). Thus, the CNN may generate a score for each image in real time. This score generated in real time for the captured images may be transmitted to application 106 for displaying the score to a clinician (e.g., an embryologist, a fertility specialist, etc.). In some variations, multiple images of the embryo may be captured within the span of a few seconds. In such variations, the CNN may generate an individual score for each captured image in real time. However, an overall viability score for the embryo may be a function of each individual score. For such variations, the overall viability score may be transmitted in real time to application 106 for displaying the overall viability score to a clinician.

[0110] The score of the embryo may be displayed in any suitable manner. For example, the score may be displayed as a percentage indicating the likelihood of a normal clinical pregnancy (e.g., 90% indicates that the embryo has a 90% chance of a normal clinical pregnancy). Alternatively, the score may be displayed as a number from a numerical scale (e.g., a number from 0 to 10, with 0 representing the least viable embryo and 10 representing the most viable embryo, a number from 0 to 100, with 0 representing the least viable embryo and 100 representing the most viable embryo, etc.). In yet another alternative variation, the score may separate the embryos into a letter scale (e.g., "A", "B", "C", "D", etc., with "A" representing the least viable embryo). In yet another alternative variation, the score may separate the embryos into categories (e.g., "good", "bad", etc.). In yet another alternative variation, at least some of the images of the embryos that may be displayed may be color coded with a color representing the viability of the embryo. For example, a frame or border of the image of the embryo may be color coded such that the color may be mapped onto the numerical score. In some examples, embryos may be separated into letter scales, categories, colors, and / or the like, at least in part, by comparing the numerical viability score to one or more predetermined thresholds.

[0111] 9 illustrates an example variation of the plug-and-play software rendered on a display 900 of a computing device to display an overall viability score for an embryo. Display 900 may include capture 903, review 905, and export 907. A user may interact with a particular option to cause the application to display widgets and information associated with that option. In FIG. 9, clicking on the review 905 option may cause the application to display widgets and information associated with the review 905 functionality.

[0112] FIG. 9 illustrates an example of a workup 905 for patient "Jane Doe." In FIG. 9, an image 10 may be captured in real time and transmitted to a controller for analysis. The controller may classify the image and generate an overall viability score of 90% for the embryo shown in image 10 (e.g., an estimated 90% likelihood of resulting in a normal clinical pregnancy if implanted in a patient). The application may cause a display 900 to display an image (e.g., image 902) and the overall viability score for the embryo in image 902. Display 900 may also include an analysis 909 associated with the embryo. For example, analysis 909 may include information such as embryo ID, days post fertilization, date the image was taken, and embryo grade (e.g., overall viability score for the embryo).

[0113] In addition, a user (e.g., an embryologist) may select what to do with each embryo (e.g., representing the embryo status with respect to transfer, freezing, discard, etc.) based on each embryo's overall viability score, and the embryo status may be indicated in any suitable manner (e.g., icon, text, color coding, etc.). For example, in FIG. 9, the user may select to discard the embryos in image 1, image 6, and image 8 (indicated with an "X" on the image). The user may select to freeze the embryos in image 2, image 3, image 4, image 5, image 7, and image 9 (indicated with a snowflake icon). The user may select to transfer the embryo in image 10 (indicated with a "T"). Thus, the user may be presented with options to decide what to do with each image and indicate the desired action for each embryo. Other variations of user interfaces for facilitating these actions are further described below with respect to FIG. 10.

[0114] In addition to displaying an overall viability score for the embryo (e.g., an indication of the likelihood of clinical pregnancy) in real time, the application may also display the images in the order in which they are ranked (e.g., from images with embryos with the highest likelihood of clinical pregnancy to images with embryos with the lowest likelihood of clinical pregnancy). FIG. 10 illustrates an example variation of the plug-and-play software rendered on a display 1000 of a computing device to display images of embryos in the order in which they are ranked. In FIG. 10, the images displayed on the display 1000 are images of different embryos (e.g., an image with embryo ID 31HG201-3-3, an image with embryo ID 31HG201-3-2, and an image with 31HG201-3-1). All three images in FIG. 10 are captured on the same day (e.g., day 5). However, the images may be captured on any suitable day (e.g., day 3, day 4, day 6, day 7, etc.).

[0115] As the images of each embryo are sent to the controller implementing the CNN for analysis, the controller may generate an overall viability score for each embryo. For example, the controller may individually analyze and individually score each image captured for the embryo with embryo ID 31HG201-3-3. The overall viability score for the embryo with embryo ID 31HG201-3-3 may be a function of the score of each captured image. In FIG. 10, the overall viability score for the embryo with embryo ID 31HG201-3-3 was determined to be 0.8. In a similar manner, the controller may individually analyze and individually score each image captured for the embryo with embryo ID 31HG201-3-2. The overall viability score for the embryo with embryo ID 31HG201-3-2 may be a function of the score of each captured image. In FIG. 10, the overall viability score for the embryo with embryo ID 31HG201-3-2 was determined to be 0.62. Similarly, the overall viability score for the embryo with embryo ID 31HG201-3-1 was determined to be 0.12.

[0116] The controller may then rank the embryos based on the overall viability scores of the embryos in the images. For example, in the example of FIG. 10, the controller may rank the embryo with embryo ID 31HG201-3-3 and an overall viability score of 0.8 as first, the embryo with embryo ID 31HG201-3-2 and an overall viability score of 0.62 as second, and the embryo with embryo ID 31HG201-3-1 and an overall viability score of 0.12 as third. In this manner, at least one image of each embryo may be displayed in the order in which they are ranked.

[0117] In some variations, an overall viability score of the embryo may be displayed adjacent to each image of the embryo. For example, in FIG. 10, an overall viability score of 0.8 is displayed adjacent to image 1. Additionally, display 1000 may include a drop-down menu below each image of the embryo with an option to select an embryo status (e.g., drop-down menu 1002a for image 1, 1002b for image 2, and 1002c for image 3). A user may select a status (e.g., "frozen transfer", "frozen", "frozen and biopsy", "discard", etc.) via drop-down menu 1002a. The status may be indicated using text, symbols or other icons, color coding, and / or in any suitable manner. Display 1000 may also include notes below each image that display clinician notes about the embryo. Display 1000 may also show an embryo ID that indicates the embryo being viewed on display 1000. Training a Convolutional Neural Network

[0118] To implement a truly data-driven approach to embryo assessment, the CNN described herein may be trained on a large amount of data. The data may be collected from various sources, including a consortium of clinical partners, databases with microscopic images of embryos, electronic medical records, and / or the like. The collected data may include microscopic images of embryos along with electronic medical record data, which may contain pregnancy outcomes for transferred embryos, Gardner grades, preimplantation embryo chromosomal aneuploidy testing (PGT-A) results for embryos that may have been biopsied and tested, and patient data. After collecting the data, the microscopic images in the collected data may be distributed into two groups based on their pregnancy outcomes. For example, the microscopic images in the collected data may be distributed into positive fetal cardiac activity (FCA), which represents a positive pregnancy outcome, and negative fetal cardiac activity (FCA), which represents a negative pregnancy outcome.

[0119] After the collected microscopic images are distributed into positive FCA and negative FCA, the images in these individual groups may be further divided into any suitable ratio to form training data and test data, such as 70% training data and 30% test data. For example, the microscopic images with positive FCA may be distributed into 70% training and 30% test. Similarly, the microscopic images with negative FCA may be distributed into 70% training and 30% test. In a similar manner, to incorporate patient data, the embryo score for each image may be concatenated with patient data. The concatenated data may be distributed into 70% training data and 30% test data.

[0120] Because the technique is designed to be compatible with any image capture device, the training data for training the CNN may need to include a combination of images from different image capture devices. Different image capture devices may have different optics and different resolutions, so the training data may need to account for such differences. Augmenting the training data

[0121] One way to account for differences in optics and resolution may be to augment the training data. Each image in the training data may be augmented on the fly. For example, one or more transformations may be applied to each image. Some non-limiting examples of random transformations include randomly flipping images up and down, randomly flipping images left and right, randomly scaling images (e.g., scaling images by about 5% from the original image size), randomly rotating images from -90 degrees to 90 degrees, randomly varying the contrast, brightness, and / or saturation of images, combinations thereof, and / or the like. FIG. 11 illustrates an exemplary variation of an example of an augmented image following application of a random transformation to one or more images of an embryo. In this way, by augmenting the training data, the CNN may be able to account for at least some variation in optics and resolution while analyzing the images. Equilibrium incidence of outcomes

[0122] Despite expanding the training data, it is possible that the CNN may still introduce bias when scoring images based on the occurrence of outcomes per clinic. For example, if the training data from one clinic has a significantly higher percentage of positive outcomes compared to all other clinics, the CNN may learn to apply a positive bias on all images from that clinic. This may lead to suboptimal analysis of embryos, as embryos from clinics with higher positive outcomes in the training data may generate false positives. Similarly, if the training data has images with a micropipette to hold the embryo, the CNN may learn to apply a positive or negative bias on all images with a micropipette.

[0123] Thus, to solve this problem, the training data may be resampled so that all clinics and / or facilities have the same ratio of positive to negative images at each epoch of training. Similarly, the training data may be resampled so that images with a micropipette have the same ratio of positive to negative images as images without a micropipette at each epoch of training. A CNN trained in this way may be able to balance the occurrence rates of outcomes and may be able to score images with better accuracy (e.g., without introducing bias).

[0124] Thus, by implementing a combination of expanding training data and balancing outcome occurrence rates, the technology can be adapted to any type of image capture device and can fit into any existing workflow for any clinic and / or facility. Illustrative Training Examples

[0125] In some variations, the training dataset may include images of implanted embryos with pregnancy outcomes (e.g., 1,000, 2,000, 3,000, 4,000, 5,000, 10,000 or more images of implanted embryos with pregnancy outcomes from seven different clinics). In some variations, Python and the open source framework PyTorch may be used to train the CNN. In some variations, training may be performed for, for example, 50 epochs. A final model may be selected from the epoch with the highest accuracy. In some variations, a series of models may be trained using hyperparameter search to find optimal values ​​for parameters such as learning rate and batch size. An ensemble of two or more models trained with different data sampling, hyperparameters, and / or architectures may be deployed to perform a final prediction (e.g., assessment of embryo viability).

[0126] Training U-Net for image cropping and image segmentationIn some variations, the training data for the U-Net described above may include manual foreground labeling on hundreds of raw embryo images, augmented image training data with random flipping and / or rotation to create thousands of images and masks, and images and masks that are padded to squares and then resized to 112x112, etc.

[0127] Training an autoencoder for quality control In some variations, the training data for the autoencoder described above may include thousands of images from two clinics. The data may include a combination of images of frozen embryos and images of fresh embryos. As discussed above, the collected image data (e.g., collected data of images of frozen embryos and images of fresh embryos) may be split in any suitable ratio to form training data and test data, such as 70% training data and 30% test data. In some variations, the training data may also include embryo cropped images (e.g., images cropped by a U-Net model) resized to a suitable size, such as 128x128.

[0128] Training a fully connected neural network to incorporate patient data In some variations, the training data for the fully connected neural network described above may include embryo scores per image concatenated with patient data. This data may be split in any suitable ratio to form training and testing data, such as 70% training and 30% testing.

[0129] In some variations, the training dataset may include images of transferred embryos (or other suitable number) with pregnancy outcomes from seven different clinics. FIG. 12 illustrates an overview of the characteristics within this training dataset. For each image of an embryo with an associated transfer outcome, information regarding patient age, day of transfer, donor status, and / or the like may be collected. In some variations, important parameters that may introduce potential bias in the CNN may be tracked. These parameters may include patient age, ethnicity, day of transfer (day 5, 6, or 7), donor versus non-donor oocytes, fresh transfer versus frozen transfer, and Gardner grade. The plot in FIG. 12 shows the distribution of the embryo dataset for each outcome. Exemplary performance data

[0130] To evaluate the performance of the present technology (e.g., the performance of the CNN), the main performance metric may be the area under the curve (AUC). The AUC may be derived from the receiver operating characteristic (ROC) curve. The AUC may be defined as the ability of a model to rank cases in a binary classification problem. In this example, the AUC may measure the extent to which the CNN described herein can accurately score embryos with positive outcomes (e.g., positive FCA) better than embryos with negative outcomes (e.g., negative FCA).

[0131] To create a reference standard, Gardner grades were collected from IVF clinics. Alphanumeric Gardner grades (e.g., 3AA or 5AB) were mapped to numerical scores (1 - 43). The mapping was performed using an ordinal technique assuming that for blastocyst expansion 1 < 2 < 4 < 5 < 6 and for both inner cell mass quality and trophectoderm quality C < B < A. Thus, the ranking of the grading may be as follows.

Chemical formula

Chemical formula

[0132] Figure 13 illustrates the receiver operating characteristic curve for fresh embryo transfer using the techniques described herein compared to the Gardner grading system. The results for the test data with N=334 show that the techniques described herein score embryos according to their likelihood of reaching a clinical pregnancy, with an AUC of 0.702 using images only and 0.734 using a combination of images and patient data. The AUC for the manual Gardner grading system is 0.585. This represents an absolute improvement of 15% by using the techniques described herein compared to standard care. Non-invasive aneuploidy prediction

[0133] In addition to scoring embryos and ranking images based on embryo viability scores, the technology may also perform non-invasive aneuploidy prediction and post-thaw viability assessment. Preimplantation embryo chromosomal aneuploidy testing (PGT-A) may involve performing a biopsy of the trophectoderm, which is sequenced to determine whether the embryo has the correct number of chromosomes. Embryos with abnormal numbers of chromosomes are aneuploid embryos, while embryos with normal numbers of chromosomes are euploid embryos. Rejecting aneuploid embryos may eliminate embryos that may lead to unsuccessful pregnancy outcomes. In other words, by performing PGT-A, aneuploid embryos may be excluded from implantation. However, existing methods of performing PGT-A prediction are invasive. Indeed, concerns about embryo safety exist to date. More recently, the PGT-A field has begun to transition to non-invasive cell-free deoxyribonucleic acid (DNA) testing. However, cell-free DNA testing has not yet been widely adopted. Thus, there is an unmet need to non-invasively predict the ploidy status of embryos with greater accuracy.

[0134] In addition, even if invasive PGT-A test and / or cell-free DNA test become widespread, it may still be possible that more than one euploid embryo is available for transfer.Not all euploid embryos can lead to normal outcome.Therefore, even in PGT-A cycle, euploid embryos may need to be graded to prioritize the order for transfer.In addition, grading embryos in PGT-A cycle (e.g., morphological grading) may provide additional information about the likelihood of aneuploidy and the possibility of leading to pregnancy.

[0135] Thus, the techniques described herein may non-invasively predict the ploidy status of an embryo. Embryo image capture

[0136] Plug-and-play software described herein, such as application 106, may be used to capture one or more images of the embryo. The application may transmit the images in real time to a controller, such as controller 108 of FIG. 1. The method and / or system for capturing images of an embryo to predict ploidy status may be the same as the method and / or system for capturing images of an embryo to generate a viability score (as described above). Ploidy status prediction

[0137] The controller may implement one or more deep CNNs to predict ploidy status from images. In some variations, these CNNs may be different from the CNNs described above (i.e., the CNNs for assessing embryo viability). For example, these CNNs may be specifically trained and modeled to predict ploidy status from images of embryos. That is, instead of building a CNN to predict fetal heartbeat, a CNN may be modeled to identify morphological features in images that may be associated with aneuploidy.

[0138] Alternatively, in addition to generating a score for the embryo, the CNN described above may be trained to predict the ploidy status of the embryo. For example, images captured via plug-and-play software such as application 106 of FIG. 1 may be analyzed and classified in real time by the CNN to generate the ploidy status of the embryo. FIG. 14 illustrates an example variation of an image of an aneuploid embryo. The CNN may be trained to identify morphological features within the image that may be associated with aneuploidy.

[0139] As described herein, a series of CNNs may be implemented to analyze and classify images. In one example, an input image may be cropped and segmented by a CNN, such as a U-Net model (such as U-Net501 in FIG. 5). The U-Net model may be trained to segment the image of the embryo and crop to the embryo boundary. A CNN for performing quality control, such as the autoencoder of FIG. 6, may be trained to identify outliers using a learned latent space clustering method. The autoencoder may verify whether the image contains an aneuploid embryo and, if so, whether the aneuploid embryo is correctly labeled. A CNN for image classification, such as the resnet-18 of FIG. 8, may be trained to classify images to generate a binary output representing ploidy status. For example, the resnet-18 may generate an output "1" indicating that the embryo is not an aneuploid embryo and may generate an output "0" indicating that the embryo is an aneuploid embryo. In some variations, the CNN trained to predict ploidy status may incorporate patient data to improve the accuracy of the prediction.

[0140] In some variations, a CNN trained to identify ploidy status may incorporate patient data to improve the accuracy of predictions, for example, patient age may be highly correlated with the ploidy status of an embryo. Viewing the output

[0141] The output may be displayed in a manner similar to displaying the output regarding the viability of the embryo as described above. For example, the plug-and-play software described herein, such as application 106, may display the ploidy status for the embryo. Training the CNN

[0142] Data may be collected from various sources, including a consortium of clinical partners, databases with microscopic images of embryos, electronic medical records, and / or the like. The collected data may include microscopic images of embryos, along with electronic medical record data, which may contain pregnancy outcomes for transferred embryos, preimplantation embryo chromosomal aneuploidy testing (PGT-A) results for embryos that may have been biopsied and tested, and patient data. After collecting the data, the microscopic images in the collected data may be distributed into two groups based on their ploidy status. For example, the microscopic images in the collected data may be distributed into euploid and aneuploid embryos.

[0143] In some variations, the training data may include a combination of pregnancy outcome and ploidy status. For example, all euploid embryos with a negative pregnancy outcome may be placed in a negative outcome group. All aneuploid embryos may be placed in a negative outcome group. All euploid embryos with a positive pregnancy outcome may be placed in a positive outcome group. In this manner, by combining training data to indicate both pregnancy outcome and ploidy status, the techniques described herein may be seamlessly integrated to accurately predict both the ploidy status of an embryo and the viability of the embryo based on the ploidy status.

[0144] In some variations, the training dataset may include over 2,000 images of transferred embryos with pregnancy outcomes from seven different clinics. FIG. 15 illustrates an overview of the characteristics within this training dataset. For each embryo image with associated ploidy status, information on patient age, date of transfer, and / or the like may be collected. In some variations, important parameters that may introduce potential bias in the CNN may be tracked. These parameters may include patient age, date of transfer (day 5, 6, or 7). The plot in FIG. 15 shows the distribution of the embryo dataset by ploidy status. Post-thaw viability assessment

[0145] In some variations, the techniques disclosed herein may be used to perform post-thaw viability assessment in addition or as an alternative. For example, as explained above, the techniques disclosed herein may assess the viability of embryos at the blastocyst stage. However, instead of transferring embryos at the blastocyst stage, embryos deemed viable may undergo cryopreservation. Cryopreservation before transferring embryos may be an all-freeze cycle. This may in turn minimize the risk of overstimulation and allow hormone levels to be reset prior to embryo transfer. Embryos may be transferred after cryopreservation and thawing. Post-thaw viability assessment may detect embryos that may have lost at least a portion of their viability after freezing and thawing. That is, some embryos that may be deemed viable at the blastocyst stage may have reduced viability (e.g., have a lower level of viability or are not viable) following the freezing and thawing process. Post-thaw viability assessment may identify such embryos.

[0146] Thus, the techniques described herein may perform post-thaw viability analysis to identify embryos that do not survive the freeze-thaw process. However, it should be understood that such post-thaw analysis may be performed in combination with the analysis prior to freezing or in a stand-alone manner. For example, in some variations, embryos may be imaged and scored and / or ranked at multiple time points, including prior to freezing (e.g., at the blastocyst stage) and after thawing. Alternatively, in some variations, embryos may be imaged and scored and / or ranked only after thawing. 3. Image capture of post-thaw embryos

[0147] Plug-and-play software described herein, such as application 106, may be used to capture one or more images of the post-thawed embryo. The application may transmit these images in real time to a controller, such as controller 108 of FIG. 1. The method and / or system for capturing images of an embryo to predict post-thaw viability may be the same as the method and / or system for capturing images of an embryo to generate a viability score (as described above) at the blastocyst stage. Predicting post-thaw viability

[0148] The controller may implement one or more deep CNNs to predict post-thaw viability from images. In some variations, these CNNs may be different from the CNNs described above (i.e., CNNs for assessing embryo viability). For example, these CNNs may be specifically trained and modeled to predict post-thaw viability from images. More specifically, various architectures, hyper-parameter optimization, and ensemble techniques may be implemented to train and model CNNs that can predict post-thaw viability from images. The CNNs may be trained to generate a probability score that may indicate whether an embryo survives the freeze-thaw process. If the probability score is below a threshold, the post-thaw embryo may be considered non-viable.

[0149] Alternatively, in addition to generating a score for the embryo, the CNN described above may be trained to predict post-thaw viability. As described herein, a series of CNNs may be implemented to analyze and classify images. In one example, an input image may be cropped and segmented by a CNN, such as a U-Net model (such as U-Net501 in FIG. 5). The U-Net model may be trained to segment the image of the post-thaw embryo and crop to the boundary of the post-thaw embryo. A CNN for performing quality control, such as the autoencoder of FIG. 6, may be trained to identify outliers using a learned latent space clustering method. The autoencoder may verify whether the image contains a post-thaw embryo and, if so, whether the post-thaw embryo is correctly labeled. A CNN for image classification, such as resnet-18 in FIG. 8, may be trained to classify the image to generate a binary output representing post-thaw viability. For example, following a freeze-thaw process, resnet-18 may generate an output "1" indicating that the post-thaw embryo is viable, or an output "0" indicating that the post-thaw embryo has reduced viability (e.g., a lower level of viability, not viable, etc.). In some variations, the CNN trained for post-thaw viability may incorporate patient data to improve prediction accuracy. Viewing the output

[0150] The output may be displayed as binary "1" and "0" and / or "yes" and "no." In other words, instead of displaying a probability and / or score for any embryo, the post-thaw viability assessment may simply indicate whether the embryo survived the freeze-thaw process. This allows the clinician to decide whether to thaw another embryo or whether the analyzed post-thawed embryo should be transferred. Training the CNN

[0151] Data may be collected from a variety of sources, including a consortium of clinical partners, databases with microscopic images of embryos, electronic medical records, and / or the like. The collected data may include microscopic images of the thawed embryos, along with electronic medical record data, which may contain pregnancy outcomes and patient data regarding the transferred thawed embryos. Exemplary Performance Data

[0152] FIG. 16 illustrates post-thaw viability assessment results from a single center. Lighter colored dots shown under 1702a represent negative outcomes. Darker colored dots shown under 1702b represent positive outcomes. Line 1704 represents a threshold. Embryos below line 1704 are embryos that did not survive the freeze-thaw process. As can be seen in FIG. 16, there are no false positives. Thus, the present technology can use post-thaw images to identify embryos that did not survive the freeze-thaw process. Additional Examples of Exemplary Performance Data

[0153] A retrospective study was conducted using data collected from 11 different IVF clinics across the United States. Images of blastocyst stage embryos and associated metadata were collected for IVF cycles initiated between 2015 and 2020. Each clinic captured a single image of the embryo using existing hardware, such as an inverted microscope, stereo zoom microscope, and time-lapse incubation system. Images of blastocyst stage embryos were captured on days 5, 6, or 7 prior to transfer, biopsy, or cryopreservation. Approximately 5,900 blastocysts from single embryo fresh, frozen, and frozen euploid transfers were matched for clinical pregnancy outcomes as determined by fetal heartbeat (FHB) at 6–8 weeks. Embryos from frozen transfers were selected for warming according to standard practice at each clinic. An additional 2,600 blastocysts were matched for aneuploid (abnormal) PGT-A results.

[0154] The training data included microscopic images of embryos. The images were aggregated together and then sorted into training, validation, and test datasets. Five clinical centers contributed between 600 and 2,000 images, each with known fetal heartbeat outcomes. These images were stratified by clinic where they were acquired, cycle type (e.g., PGT or non-PGT cycles), and outcome. These images were also randomly distributed into groups for validation (e.g., 3-fold cross-validation). Another five clinics contributed fewer than 250 images, each with known fetal heartbeat outcomes. All of these images were included within the training. One clinic with 1,000 images captured by the time-lapse system was reserved as the test dataset. Embryos were sorted into a positive class if they yielded a positive fetal heartbeat and into a negative class if they did not yield a positive fetal heartbeat. To reduce potential bias in training against only transferred embryos, non-transferred embryos diagnosed as aneuploid were added to the negative class.

[0155] As an example, the CNN described herein was trained using the training data described above. Figures 17A and 17B illustrate receiver operating characteristic (ROC) curves for an example embryo transfer using the CNN described herein compared to a manual Gardner grading system. As described above, PGT-A may involve performing a biopsy and determining whether the embryo has the correct number of chromosomes (euploid embryo). A non-PGT cycle may include an embryo transfer without performing PGT-A. Figure 17A illustrates the ROC curve for an embryo transfer without performing PGT-A. As seen in Figure 17A, the CNN described herein scores embryos (non-PGT cycle embryos) according to their likelihood of achieving a clinical pregnancy, with an AUC of 0.669 using only images (trace 1804 in Figure 17A) and 0.675 using a combination of images and patient data (trace 1806 in Figure 17A). The AUC of the manual Gardner grading system is 0.560 (trace 1802 in FIG. 17A ). Thus, scoring embryos using the techniques described herein represents a significant improvement compared to scoring embryos using the manual Gardner grading system.

[0156] FIG. 17B illustrates the ROC curve for an exemplary embryo transfer after performing PGT-A. PGT-A may eliminate aneuploid embryos with abnormal numbers of chromosomes. Aneuploid embryos may lead to failed pregnancies. As seen in the example of FIG. 17B, the CNN described herein scores embryos (euploid embryos) according to their likelihood of reaching a clinical pregnancy, with an AUC of 0.608 using only images (trace 1804 of FIG. 17B) and 0.634 using a combination of images and patient data (trace 1806 of FIG. 17B). The AUC of the manual Gardner grading system is 0.496 (trace 1802 of FIG. 17B). Thus, similar to FIG. 17A, FIG. 17B shows that scoring embryos using the techniques described herein illustrates a significant improvement compared to scoring embryos using the manual Gardner grading system.

[0157] To further illustrate the performance of the techniques described herein, the embryos were divided into three different subgroups. The three different subgroups were top ranked embryos with scores close to 0.9, middle ranked embryos with scores close to 0.5, and bottom ranked embryos with scores close to 0.1. The embryo images for each of these subgroups were visually inspected. FIG. 18A illustrates an embryo image that is top ranked (e.g., high score) by the techniques described herein (e.g., CNN). As seen in FIG. 18A, the top ranked embryo depicts an almost fully expanded blastocyst with a tightly packed and focused inner cell mass, which is quite consistent with a highly viable embryo. The trophectoderm is also symmetrical and focused with a boulder or wave patterning. FIG. 18B illustrates an embryo image that is bottom ranked by the techniques described herein (e.g., CNN). The techniques described herein rank these embryos as the lowest ranked embryos when the inner cell mass is not visible in the images, which is quite consistent with the less viable embryos.

[0158] In some variations, an attribution algorithm, including integrated gradients and occlusion maps, was used to determine whether the techniques described herein focused on relevant features. The integrated gradients were used to determine which pixels of the image were attributable to the predictions of the techniques described herein, while the occlusion maps were used to show that the techniques described herein are sensitive to local structures (e.g., blastocyst structures). FIG. 19A illustrates an embryo image scored high by the techniques described herein. As seen in FIG. 19A, a fully expanded blastocyst structure with a tightly packed inner cell mass and symmetrical trophectoderm showed positive attribution and sensitivity (e.g., classified as a positive outcome image). FIG. 19B illustrates an embryo image scored low by the techniques described herein. As seen in FIG. 19B, a fragmented blastocyst structure with cell clumps showed negative attribution and sensitivity (e.g., classified as a negative outcome image).

[0159] The scores assigned by the techniques described herein were compared to pregnancy rates (e.g., a calibration curve). Figure 20 illustrates that the scores assigned by the techniques described herein relate to observed pregnancy rates. Figure 20 depicts the monotonic increase in pregnancy rates for all embryos (implanted and aneuploid), all transferred embryos, and non-PGT transfers. Bias Removal

[0160] As discussed above, images from different image capture devices have different optics and different resolutions (e.g., unique optical signatures), so when a CNN is trained with images from different image capture devices, it is possible that bias is introduced with respect to images from a specific image capture device compared to some other image capture device. Differences in image capture devices between different clinics may lead to biased training datasets. For example, Figures 21A-21D illustrate a control experiment to depict the bias introduced by the unique optical signatures of images from two different image capture devices in two different clinics. Figure 21A illustrates an image of an embryo captured by an image capture device at facility A (first clinic). Figure 21B illustrates an image of an embryo captured by a different image capture device at facility B (second clinic). Figure 21C illustrates a biased dataset due to different optical signatures from facilities A and B. As seen in Figure 21C, a biased dataset may result in a CNN described herein classifying embryos from site B as positive fetal heartbeat with a lower score than embryos from site A that are classified as negative fetal heartbeat. To counter this, the CNN may be trained by balancing the occurrence rates of the outcomes.

[0161] FIG. 22 illustrates a table illustrating the balancing of training data based on the occurrence of outcomes for different clinics (e.g., site A, site B, site C, site D). The training data may be resampled such that all clinics have the same ratio of positive to negative images to balance the occurrence of outcomes per clinic. FIG. 21D illustrates an unbiased dataset balanced based on the occurrence of outcomes. As seen in FIG. 21D, the unbiased dataset results in the CNN described herein classifying embryos from site B as positive fetal heartbeats with a higher score and negative fetal heartbeats with a lower score, similar to the classification of embryos from site A.

[0162] As discussed above, another source of bias may be the presence of an embryo-holding micropipette in an image. Figures 23A and 23B illustrate a control experiment to depict the bias introduced by the presence of a micropipette in an image. Figure 23A illustrates that a CNN trained with a biased dataset may focus almost exclusively on the micropipette, rather than on the embryo. To counter this, the training data was resampled such that images with a micropipette had the same ratio of positive to negative images as images without a micropipette. Figure 23B illustrates that training data resampled to balance the occurrence rates of outcomes resulted in the CNN also focusing on embryos in images with a micropipette. The foregoing description has used specific nomenclature for purposes of explanation to provide a thorough understanding of the invention. However, it will be apparent to one skilled in the art that specific details are not required to practice the invention. Thus, the foregoing description of specific embodiments of the invention is presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the invention to the precise forms disclosed, and obviously, many modifications and variations are possible in light of the above teachings. The embodiments were chosen and described in order to explain the principles of the invention and its practical application, thereby enabling those skilled in the art to utilize the invention and its various embodiments with various modifications as suited to the particular use envisioned. It is intended that the following claims and their equivalents define the scope of the invention.

Claims

1. 1. A computer-implemented method for predicting embryo viability, the method comprising: receiving a single image via a real-time communications link with an image capture device; cropping the single image to a boundary of the embryo via a first convolutional neural network; generating a viability score for the embryo by classifying the cropped single image via at least a second convolutional neural network; A method comprising:

2. The method of claim 1 , wherein the single image is not part of a time series of images.

3. The method of claim 1 , wherein generating the viability score for the embryo is performed in response to determining that the single image depicts an embryo.

4. 10. The method of claim 1, further comprising: in response to determining that the single image does not depict an embryo, providing an alert to a user of the image capture device.

5. 10. The method of claim 1, further comprising determining the probability that the embryo is a single blastocyst.

6. The method of claim 1 , wherein the real-time communication link is provided by an application executing on a computing device that is communicatively coupled to the image capture device.

7. The method of claim 6 , wherein the application causes a display on the computing device to display a capture button.

8. 8. The method of claim 7, wherein in response to a user selecting the capture button, the image capture device captures a first single image of the embryo.

9. 2. The method of claim 1, wherein the viability score represents the likelihood of the embryo achieving a clinical pregnancy.

10. 10. The method of claim 1, wherein the viability score represents the likelihood of the embryo achieving live birth.

11. 10. The method of claim 1, wherein the likelihood of the embryo achieving clinical pregnancy is associated with an outcome of fetal cardiac activity.

12. The method of claim 1 , wherein the viability score is based at least in part on data associated with a patient.

13. The method of claim 12 , wherein the data includes at least one of age, body mass index, date of image capture, and donor status.

14. The method of claim 1 , further comprising storing the viability score in a database.

15. The method of claim 1 , further comprising communicating the viability score to at least one of a patient and a clinician.

16. 10. The method of claim 1, further comprising predicting whether the embryo is euploid or aneuploid via a fourth convolutional neural network.

17. 17. The method of claim 16, wherein predicting whether the embryo is euploid or aneuploid depends at least in part on data associated with a subject.

18. 20. The method of claim 17, wherein the data is at least one of age and date of biopsy.

19. generating a ploidy outcome based on whether the embryo is euploid or aneuploid; updating at least the fourth convolutional neural network based at least in part on the ploidy outcome and the data; and 20. The method of claim 17, further comprising:

20. 10. The method of claim 1, wherein the embryo is to undergo at least one of biopsy and freezing, the method further comprising receiving the single image of the embryo prior to biopsy or freezing, and determining viability of the embryo prior to at least one of biopsy and freezing.

21. 10. The method of claim 1, wherein the embryo has been frozen and thawed, the method further comprising receiving the single image of the embryo after thawing and determining viability of the embryo after thawing via the second convolutional neural network.

22. 22. The method of claim 21 , wherein determining viability of the embryo after thawing comprises classifying the single image into either a first class indicative of the embryo being viable after thawing or a second class indicative of the embryo not being viable after thawing.

23. 2. The method of claim 1, further comprising: receiving a plurality of single images, each single image depicting an individual embryo of a plurality of embryos; generating a viability score for each embryo by classifying each single image via the second convolutional neural network; and ranking the plurality of embryos based on the viability scores for the plurality of embryos.

24. 1. A computer-implemented method for characterizing a plurality of embryos, the method comprising: receiving a plurality of single images, each single image depicting a different individual embryo of the plurality of embryos; generating a viability score for each embryo by classifying each single image via at least one convolutional neural network; ranking the plurality of embryos based on the viability scores for the plurality of embryos; A method comprising:

25. The method of claim 24 , wherein the single image is not part of a time series of images.

26. 25. The method of claim 24, wherein the viability score represents the likelihood of the embryo achieving a clinical pregnancy.

27. 27. The method of claim 26, wherein the likelihood of the embryo achieving clinical pregnancy is associated with an outcome of fetal cardiac activity.

28. 25. The method of claim 24, further comprising predicting whether the embryo is euploid or aneuploid via the at least one convolutional neural network.

29. 1. A computer-implemented method for predicting embryo viability, the method comprising: receiving a single image of the embryo captured with an image capture device; generating a viability score for each embryo by classifying each single image via at least one convolutional neural network; Including, The method, wherein the at least one convolutional neural network is trained based on training data comprising a plurality of single images of an embryo captured using a plurality of image capture devices.

30. 1. A computer-implemented method for predicting embryo viability, the method comprising: receiving a single image of the embryo; generating a viability score for each embryo by classifying each single image via at least one convolutional neural network; Including, The method of claim 1, wherein the at least one convolutional neural network is trained based at least in part on training data comprising a plurality of augmented images of a plurality of embryos.

Images