Establishment method and device of embryo evaluation model, equipment and storage medium

Abstract

The invention provides an embryo evaluation model establishment method and device, equipment and a storage medium, and relates to the technical field of embryo quality evaluation. The method comprises the following steps: carrying out image frame insertion processing on a multi-focus original plane image of a blastocyst obtained in a time difference incubator to obtain a multi-focus target image corresponding to the blastocyst; performing blastocyst three-dimensional reconstruction according to the multi-focus target image to generate a blastocyst three-dimensional model; extracting multiple items of three-dimensional feature information according to the blastocyst three-dimensional model; and performing feature screening according to the three-dimensional feature information, determining target three-dimensional feature information, and constructing an embryo evaluation model based on the target three-dimensional feature information. According to the method, three-dimensional reconstruction is carried out based on the multi-focus target image, and the obtained blastocyst three-dimensional model can express more feature details; therefore, the multi-item three-dimensional feature information extracted based on the blastocyst three-dimensional model is more accurate, the development condition of the blastocyst can be better expressed, and the constructed embryo evaluation model has better prediction capability and can more accurately predict the development potential of the embryo.

Description

Technical Field

[0001] The present application relates to the technical field of embryo quality assessment, and in particular to a method, device, equipment and storage medium for establishing an embryo assessment model. Background Art

[0002] Embryo assessment models can be used to assess embryo quality and predict high-quality embryos with developmental potential. By mining the relevant characteristics of embryos, a data foundation can be laid for the establishment of embryo assessment models.

[0003] In the prior art, the embryo is usually reconstructed by extracting the two-dimensional features of the embryo, and then a plurality of embryo features for building an embryo evaluation model are extracted based on the reconstruction results.

[0004] However, the embryo model reconstructed by the above method is poor, resulting in poor accuracy of the extracted embryo features, which in turn leads to poor predictive ability of the established embryo assessment model. Summary of the invention

[0005] The purpose of the present application is to provide a method, device, equipment and storage medium for establishing an embryo assessment model to address the deficiencies in the above-mentioned prior art, so as to improve the predictive ability of the constructed embryo assessment model.

[0006] To achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0007] In a first aspect, the present application provides a method for establishing an embryo assessment model, comprising:

[0008] Performing image interpolation processing based on the multi-focus original plane image of the blastocyst obtained in the time difference incubator to obtain a multi-focus target image corresponding to the blastocyst;

[0009] Performing three-dimensional reconstruction of the blastocyst according to the multi-focus target image to generate a three-dimensional model of the blastocyst; the three-dimensional model of the blastocyst includes: a blastocyst surface model, an inner cell mass surface model, and a trophoblast cell surface model;

[0010] Extracting multiple pieces of three-dimensional feature information according to the three-dimensional model of the blastocyst, wherein the three-dimensional feature information includes: three-dimensional features of the blastocyst, three-dimensional features of the inner cell mass, and three-dimensional features of the trophoblast cells;

[0011] Feature screening is performed based on each item of three-dimensional feature information to determine the target three-dimensional feature information, and an embryo evaluation model is constructed based on the target three-dimensional feature information.

[0012] Optionally, performing three-dimensional reconstruction of the blastocyst according to the multi-focus target image to generate a three-dimensional model of the blastocyst includes:

[0013] According to the image regions of interest in the multifocal target image, the multifocal target image is divided to obtain a multifocal inner cell mass image and a multifocal trophoblast cell image;

[0014] According to each focal inner cell cluster image, inner cell cluster segmentation is performed to obtain a multi-focal inner cell cluster mask image;

[0015] According to each focal trophoblast cell image, trophoblast cell segmentation is performed to obtain a multi-focal trophoblast cell mask image;

[0016] According to each focal inner cell mass mask image, an inner cell mass point cloud model is constructed, and according to the inner cell mass point cloud model, an inner cell mass surface model is converted;

[0017] A trophoblast cell point cloud model is constructed according to each focal trophoblast cell mask image, and a trophoblast cell surface model is converted according to the trophoblast cell point cloud model.

[0018] Optionally, performing inner cell cluster segmentation according to each focal inner cell cluster image to obtain a multi-focal inner cell cluster mask image includes:

[0019] Each in-focus inner cell cluster image is input into a pre-trained inner cell cluster segmentation model, and the inner cell cluster segmentation model is used to segment the inner cell cluster mask in each in-focus inner cell cluster image to obtain a multi-focus inner cell cluster mask image.

[0020] Optionally, the training process of the inner cell mass segmentation model includes:

[0021] Collecting an initial training sample data set, the initial training sample data set comprising: multi-focal plane sample blastocyst images corresponding to a plurality of blastocysts, the multi-focal plane sample blastocyst images corresponding to each blastocyst comprising: an original multi-focal plane sample blastocyst image and an extended multi-focal plane sample blastocyst image obtained after image interpolation; and the inner cell mass mask in each focal plane sample blastocyst image has a mark;

[0022] Performing image amplification processing on the sample blastocyst images in the initial training sample data set to generate a target training sample data set;

[0023] The target training sample data set is used to train the inner cell mass segmentation model.

[0024] Optionally, performing image amplification processing on the sample blastocyst images in the initial training sample data set to generate a target training sample data set includes:

[0025] Perform at least one amplification process on the multi-focal plane sample blastocyst images corresponding to each blastocyst in the initial training sample data set to obtain an amplified blastocyst image corresponding to each blastocyst; the amplification process includes: image rotation, contrast enhancement, and brightness enhancement;

[0026] The target training sample data set is generated according to the multi-focal plane sample blastocyst images corresponding to each blastocyst and the amplified blastocyst images corresponding to each blastocyst.

[0027] Optionally, constructing an inner cell mass point cloud model according to each in-focus inner cell mass mask image comprises:

[0028] Taking the center point of the cell cluster mask image in the center focus as the coordinate origin, stacking and arranging the cell cluster mask images in each focus according to the depth information of each focal plane, and obtaining the arranged cell cluster mask images in each focus;

[0029] According to the coordinate origin, according to the preset rotation angle interval, the mask contours of each in-focus cell cluster mask image are traversed to obtain a mask contour coordinate set corresponding to each rotation angle; wherein the mask contour coordinate set corresponding to the rotation angle includes: coordinate information of mask contour points on each in-focus cell cluster mask image arranged in sequence according to the focal plane depth under the rotation angle;

[0030] Curve fitting is performed according to the mask contour coordinate set corresponding to each rotation angle to construct the inner cell mass point cloud model.

[0031] Optionally, constructing a trophoblast cell point cloud model according to each focal trophoblast cell mask image comprises:

[0032] stacking the focal trophoblast cell mask images according to the focal plane depth information of the focal trophoblast cell mask images to obtain a stacked trophoblast cell mask image;

[0033] The trophoblast cell point cloud model is constructed according to the stacked trophoblast cell mask images.

[0034] Optionally, converting the inner cell mass point cloud model to obtain an inner cell mass surface model comprises:

[0035] Normal estimation and surface reconstruction are performed on the inner cell mass point cloud model to obtain the inner cell mass surface model.

[0036] Optionally, extracting at least multiple pieces of three-dimensional feature information based on the blastocyst three-dimensional model includes:

[0037] Extracting a plurality of basic three-dimensional feature information according to the three-dimensional model of the blastocyst, wherein the basic three-dimensional feature information includes: the volume of the blastocyst, the surface area of ​​the blastocyst, the diameter of the blastocyst, the volume of the cavity, the number of trophoblast cells, the surface area of ​​the inner cell mass, and the volume of the inner cell mass;

[0038] Performing feature mining based on the multiple pieces of basic three-dimensional feature information to obtain multiple pieces of extended three-dimensional feature information;

[0039] The multiple pieces of three-dimensional feature information are obtained according to the multiple pieces of basic three-dimensional feature information and the multiple pieces of extended three-dimensional feature information.

[0040] Optionally, performing feature mining according to the multiple pieces of basic three-dimensional feature information to obtain multiple pieces of extended three-dimensional feature information includes:

[0041] Determining a ratio of the surface area to the volume of the blastocyst according to the volume of the blastocyst and the surface area of ​​the blastocyst;

[0042] Determining the density of the trophoblast cells according to the number of the trophoblast cells and the surface area of ​​the blastocyst;

[0043] determining a ratio of the surface area to the volume of the inner cell mass according to the surface area of ​​the inner cell mass and the volume of the inner cell mass;

[0044] The ratio of the surface area to the volume of the blastocyst, the density of the trophoblast cells, and the ratio of the surface area to the volume of the inner cell mass are determined as the multiple pieces of extended three-dimensional feature information.

[0045] Optionally, the performing feature screening according to each item of three-dimensional feature information to determine the target three-dimensional feature information includes:

[0046] According to the information of each three-dimensional feature, the variance analysis method or chi-square verification method is used to determine the relationship between each three-dimensional feature and the clinical outcome of the embryo;

[0047] According to the relationship between each three-dimensional feature and the clinical outcome of the embryo, the target three-dimensional feature information is screened from the three-dimensional feature information.

[0048] Optionally, constructing an embryo assessment model based on the target three-dimensional feature information includes:

[0049] One or more items of the target three-dimensional feature information are added as new training parameters to the original training parameters of the original embryo assessment model to construct a new embryo assessment model.

[0050] In a second aspect, an embodiment of the present application further provides a device for establishing an embryo assessment model, comprising: a processing module, a reconstruction module, an extraction module, and a model building module;

[0051] The processing module is used to perform image interpolation processing based on the multi-focus original plane image of the blastocyst obtained in the time difference incubator to obtain a multi-focus target image corresponding to the blastocyst;

[0052] The reconstruction module is used to perform three-dimensional reconstruction of the blastocyst according to the multi-focus target image to generate a three-dimensional model of the blastocyst; the three-dimensional model of the blastocyst includes: a blastocyst surface model, an inner cell mass surface model and a trophoblast cell surface model;

[0053] The extraction module is used to extract multiple pieces of three-dimensional feature information according to the blastocyst three-dimensional model, wherein the three-dimensional feature information includes: three-dimensional features of the blastocyst, three-dimensional features of the inner cell mass, and three-dimensional features of the trophoblast cells;

[0054] The model building module is used to perform feature screening according to various three-dimensional feature information, determine target three-dimensional feature information, and build an embryo evaluation model based on the target three-dimensional feature information.

[0055] Optionally, the reconstruction module is specifically used to divide the multifocal target image according to the image regions of interest in the multifocal target image to obtain a multifocal inner cell mass image and a multifocal trophoblast cell image;

[0056] According to each focal inner cell cluster image, inner cell cluster segmentation is performed to obtain a multi-focal inner cell cluster mask image;

[0057] According to each focal trophoblast cell image, trophoblast cell segmentation is performed to obtain a multi-focal trophoblast cell mask image;

[0058] According to each focal inner cell mass mask image, an inner cell mass point cloud model is constructed, and according to the inner cell mass point cloud model, an inner cell mass surface model is converted;

[0059] A trophoblast cell point cloud model is constructed according to each focal trophoblast cell mask image, and a trophoblast cell surface model is converted according to the trophoblast cell point cloud model.

[0060] Each in-focus inner cell cluster image is input into a pre-trained inner cell cluster segmentation model, and the inner cell cluster segmentation model is used to segment the inner cell cluster mask in each in-focus inner cell cluster image to obtain a multi-focus inner cell cluster mask image.

[0061] Optionally, a training module is also included;

[0062] The training module is used to collect an initial training sample data set, wherein the initial training sample data set includes: multi-focal plane sample blastocyst images corresponding to a plurality of blastocysts, and the multi-focal plane sample blastocyst images corresponding to each blastocyst include: an original multi-focal plane sample blastocyst image and an extended multi-focal plane sample blastocyst image obtained after image interpolation; and the inner cell mass mask in each focal plane sample blastocyst image has a mark;

[0063] Performing image amplification processing on the sample blastocyst images in the initial training sample data set to generate a target training sample data set;

[0064] The target training sample data set is used to train the inner cell mass segmentation model.

[0065] Optionally, the training module is specifically used to perform at least one amplification process on the multi-focal plane sample blastocyst images corresponding to each blastocyst in the initial training sample data set to obtain an amplified blastocyst image corresponding to each blastocyst; the amplification process includes: image rotation, contrast enhancement, and brightness enhancement;

[0066] The target training sample data set is generated according to the multi-focal plane sample blastocyst images corresponding to each blastocyst and the amplified blastocyst images corresponding to each blastocyst.

[0067] Optionally, the reconstruction module is specifically used to stack and arrange the cell cluster mask images in each focus according to the depth information of each focal plane, using the center point of the cell cluster mask image in the center focus as the coordinate origin, to obtain the arranged cell cluster mask images in each focus;

[0068] According to the coordinate origin, according to the preset rotation angle interval, the mask contours of each in-focus cell cluster mask image are traversed to obtain a mask contour coordinate set corresponding to each rotation angle; wherein the mask contour coordinate set corresponding to the rotation angle includes: coordinate information of mask contour points on each in-focus cell cluster mask image arranged in sequence according to the focal plane depth under the rotation angle;

[0069] Curve fitting is performed according to the mask contour coordinate set corresponding to each rotation angle to construct the inner cell mass point cloud model.

[0070] Optionally, the reconstruction module is specifically configured to stack each of the focused trophoblast cell mask images according to the focal plane depth information of each of the focused trophoblast cell mask images to obtain a stacked trophoblast cell mask image;

[0071] The trophoblast cell point cloud model is constructed according to the stacked trophoblast cell mask images.

[0072] Optionally, the reconstruction module is specifically used to perform normal estimation and surface reconstruction on the inner cell mass point cloud model to obtain the inner cell mass surface model.

[0073] Optionally, the extraction module is specifically used to extract multiple basic three-dimensional feature information according to the blastocyst three-dimensional model, wherein the basic three-dimensional feature information includes: the volume of the blastocyst, the surface area of ​​the blastocyst, the diameter of the blastocyst, the volume of the cavity, the number of trophoblast cells, the surface area of ​​the inner cell mass, and the volume of the inner cell mass;

[0074] Performing feature mining based on the multiple pieces of basic three-dimensional feature information to obtain multiple pieces of extended three-dimensional feature information;

[0075] The multiple pieces of three-dimensional feature information are obtained according to the multiple pieces of basic three-dimensional feature information and the multiple pieces of extended three-dimensional feature information.

[0076] Optionally, the extraction module is specifically used to determine the ratio of the surface area to the volume of the blastocyst according to the volume of the blastocyst and the surface area of ​​the blastocyst;

[0077] Determining the density of the trophoblast cells according to the number of the trophoblast cells and the surface area of ​​the blastocyst;

[0078] determining a ratio of the surface area to the volume of the inner cell mass according to the surface area of ​​the inner cell mass and the volume of the inner cell mass;

[0079] The ratio of the surface area to the volume of the blastocyst, the density of the trophoblast cells, and the ratio of the surface area to the volume of the inner cell mass are determined as the multiple pieces of extended three-dimensional feature information.

[0080] Optionally, the model building module is specifically used to determine the relationship between each three-dimensional feature and the clinical outcome of the embryo by using a variance analysis method or a chi-square verification method according to each three-dimensional feature information;

[0081] According to the relationship between each three-dimensional feature and the clinical outcome of the embryo, the target three-dimensional feature information is screened from the three-dimensional feature information.

[0082] Optionally, the model building module is specifically used to add one or more items of the target three-dimensional feature information as new training parameters to original training parameters of the original embryo assessment model to construct a new embryo assessment model.

[0083] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to implement the method for establishing an embryo assessment model as provided in the first aspect.

[0084] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for establishing an embryo assessment model as provided in the first aspect is executed.

[0085] The beneficial effects of this application are:

[0086] The present application provides a method, device, equipment and storage medium for establishing an embryo evaluation model, including: performing image interpolation processing based on the multi-focus original plane image of the blastocyst obtained under the time difference incubator to obtain a multi-focus target image corresponding to the blastocyst; performing three-dimensional reconstruction of the blastocyst according to the multi-focus target image to generate a three-dimensional model of the blastocyst; extracting multiple three-dimensional feature information according to the three-dimensional model of the blastocyst; performing feature screening according to each three-dimensional feature information, determining the target three-dimensional feature information, and constructing an embryo evaluation model based on the target three-dimensional feature information. The method performs image interpolation processing based on the collected multi-focus original plane image to obtain a multi-focus target image, expands the image details, performs three-dimensional reconstruction based on the multi-focus target image, and the obtained three-dimensional model of the blastocyst can express more feature details; thereby, the multiple three-dimensional feature information extracted based on the blastocyst three-dimensional model is more accurate and can better express the development of the blastocyst. The embryo evaluation model constructed based on the screened target three-dimensional feature information has better prediction ability and can more accurately predict the development potential of the embryo.

[0087] In addition, by training the segmentation model, the masks of the inner cell mass image and the trophoblast cell image can be accurately segmented, so that a three-dimensional model of the blastocyst can be constructed based on the segmentation results, thereby improving the construction efficiency and accuracy of the three-dimensional model. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0089] Figure 1 A schematic diagram of a process for establishing an embryo assessment model provided in an embodiment of the present application;

[0090] Figure 2 A schematic diagram of a process for establishing another embryo assessment model provided in an embodiment of the present application;

[0091] Figure 3 A schematic diagram of a process for establishing another embryo assessment model provided in an embodiment of the present application;

[0092] Figure 4A schematic diagram of a process for establishing another embryo assessment model provided in an embodiment of the present application;

[0093] Figure 5 A schematic diagram of a process for establishing another embryo assessment model provided in an embodiment of the present application;

[0094] Figure 6 A schematic diagram of an inner cell mass mask image stacking effect provided in an embodiment of the present application;

[0095] Figure 7 A schematic diagram of an inner cell mass point cloud model fitting provided in an embodiment of the present application;

[0096] Figure 8 A schematic diagram of a process for establishing another embryo assessment model provided in an embodiment of the present application;

[0097] Fig. 9 A schematic diagram of the fitting of the trophoblast cell point cloud model provided in an embodiment of the present application;

[0098] Fig.10 A schematic diagram of a curved surface model provided in an embodiment of the present application;

[0099] Fig.11 A schematic diagram of a process for establishing another embryo assessment model provided in an embodiment of the present application;

[0100] Fig.12 A schematic diagram of a process for establishing another embryo assessment model provided in an embodiment of the present application;

[0101] Fig.13 A schematic diagram of a performance curve of a clinical pregnancy decision tree model provided in an embodiment of the present application;

[0102] Fig.14 A schematic diagram of a device for establishing an embryo assessment model provided in an embodiment of the present application;

[0103] Fig.15 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0104] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0105] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0106] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0107] With the rapid development of computer vision and assisted reproductive technology, the application of artificial intelligence in the field of assisted reproduction is becoming increasingly in-depth. How to simulate the spatial structure of the blastocyst through three-dimensional reconstruction technology without damaging the embryo, thereby revealing information that cannot be observed from a two-dimensional perspective, has always been one of the focuses of embryology experts.

[0108] Since its introduction into the field of embryo culture, time-difference culture technology has brought about tremendous changes in assisted reproductive technology, subverted the traditional working mode of embryologists, and greatly improved work efficiency. With the help of time-difference culture technology, not only can more embryo characteristics be obtained, but it also facilitates the formation of a variety of embryo scoring systems. In addition, the mature application of time-difference culture technology in the field of assisted reproduction has laid a solid foundation for non-invasive three-dimensional reconstruction of embryos. Through the embryo three-dimensional reconstruction technology based on time-difference culture imaging data, the three-dimensional spatial structure of the blastocyst can be accurately simulated, which provides a new way to deeply explore the characteristics of the blastocyst period and lays a data foundation for building a clinical pregnancy prediction model.

[0109] Figure 1The following is a flow chart of a method for establishing an embryo assessment model provided in an embodiment of the present application; the execution subject of the method may be a computer device. Figure 1 As shown, the method may include:

[0110] S101, performing image interpolation processing based on the multi-focus original plane image of the blastocyst obtained in the time difference incubator to obtain a multi-focus target image corresponding to the blastocyst.

[0111] Optionally, a multi-focus original plane image of the blastocyst under the premise of time difference culture can be selected, and the multi-focus original plane image can be 11 or more focal points. Generally, the focal plane interval of the acquired multi-focus original plane image may be large, and there is a loss of image details between two adjacent focal images. Therefore, the multi-focus original plane image can be expanded by image interpolation processing, and a multi-focus new plane image can be simulated and generated based on the image features of the multi-focus original plane image.

[0112] In some embodiments, a pre-trained image interpolation model can be used for image interpolation processing, and the multi-focus original plane image is used as the input data of the image interpolation model. The image interpolation model learns the image features of each focal original plane image, and based on the image features of any two adjacent focal original plane images, a multi-focus new plane image between the two adjacent focal original plane images can be simulated and generated; after the interpolation processing, the generated multi-focus new plane image can play a role in transition and information connection of the multi-focus original plane image, which is beneficial to the subsequent contour fitting and surface reconstruction. Thus, the multi-focus original plane image and the multi-focus new plane image generated by the interpolation constitute the multi-focus target image corresponding to the blastocyst.

[0113] S102, performing three-dimensional reconstruction of the blastocyst according to the multi-focus target image to generate a three-dimensional model of the blastocyst.

[0114] The blastocyst three-dimensional model includes: a blastocyst surface model, an inner cell mass surface model and a trophoblast cell surface model.

[0115] Optionally, spatial three-dimensional modeling is performed based on the multi-focus target image to generate a three-dimensional model of the blastocyst.

[0116] In some embodiments, the blastocyst can be taken as a whole to reconstruct a blastocyst surface model; in other embodiments, in order to facilitate the feature extraction and study of the inner cell mass and trophoblast cells, the inner cell mass and trophoblast cells in the blastocyst can be reconstructed separately to obtain the inner cell mass surface model and the trophoblast cell surface model respectively.

[0117] S103. Extract multiple pieces of three-dimensional feature information based on the blastocyst three-dimensional model.

[0118] The three-dimensional feature information includes: the three-dimensional features of the blastocyst, the three-dimensional features of the inner cell mass, and the three-dimensional features of the trophoblast cells.

[0119] Optionally, based on the reconstructed three-dimensional model of the blastocyst, multiple three-dimensional feature information of the blastocyst, inner cell mass and trophoblast cells can be extracted respectively, and operations such as data cleaning and data mining can be performed.

[0120] It is worth mentioning that compared with two-dimensional features, the three-dimensional feature information extracted above can express more detailed information about the embryo, so that clinical outcome analysis of embryo development based on the three-dimensional features can better predict the developmental potential of the blastocyst.

[0121] S104, performing feature screening according to each item of three-dimensional feature information, determining target three-dimensional feature information, and constructing an embryo evaluation model based on the target three-dimensional feature information.

[0122] Optionally, some of the extracted three-dimensional feature information may not have a particularly large impact on the clinical outcome. Then, after statistical analysis of the data, the three-dimensional feature information can be screened to screen out multiple target three-dimensional feature information that are more important for the clinical outcome, thereby building an embryo assessment model based on the target three-dimensional feature information. The embryo assessment model can be used to evaluate the development of any embryo.

[0123] In summary, the method for establishing an embryo assessment model provided in this embodiment includes: performing image interpolation processing based on the multi-focus original plane image of the blastocyst obtained under the time difference incubator to obtain a multi-focus target image corresponding to the blastocyst; performing three-dimensional reconstruction of the blastocyst according to the multi-focus target image to generate a three-dimensional model of the blastocyst; extracting multiple three-dimensional feature information according to the three-dimensional model of the blastocyst; performing feature screening according to each three-dimensional feature information, determining the target three-dimensional feature information, and constructing an embryo assessment model based on the target three-dimensional feature information. This method performs image interpolation processing based on the collected multi-focus original plane image to obtain a multi-focus target image, expands the image details, and performs three-dimensional reconstruction based on the multi-focus target image. The obtained three-dimensional model of the blastocyst can express more feature details; thereby, the multiple three-dimensional feature information extracted based on the blastocyst three-dimensional model is more accurate and can better express the development of the blastocyst. The embryo assessment model constructed based on the screened target three-dimensional feature information has better predictive ability and can more accurately predict the developmental potential of the embryo.

[0124] Figure 2 A schematic flow chart of another method for establishing an embryo evaluation model provided in an embodiment of the present application; Optionally, in step S102, performing three-dimensional reconstruction of the blastocyst according to the multi-focus target image to generate a three-dimensional model of the blastocyst may include:

[0125] S201. Divide the multi-focus target image according to the image regions of interest in the multi-focus target image to obtain a multi-focus inner cell mass image and a multi-focus trophoblast cell image.

[0126] Next, the image regions of interest in each focal target image may be extracted. In this embodiment, the inner cell mass and trophoblast cells may be respectively taken as regions of interest for image segmentation to extract the inner cell mass image and the trophoblast cell image from each focal target image.

[0127] Of course, the blastocyst may also be taken as a region of interest, and the blastocyst image may be extracted from each focus target image.

[0128] S202 , performing inner cell cluster segmentation according to each focal inner cell cluster image to obtain a multi-focal inner cell cluster mask image.

[0129] Since Hoffman imaging is used in this solution when extracting multi-focal original plane images from the time-difference incubator, artifacts may appear, and the boundaries of the new focal plane images obtained after interpolation may be unclear. Therefore, image enhancement processing can be performed on the extracted cell cluster images in each focus to reduce artifacts caused by Hoffman imaging and unclear boundaries of the interpolation-generated images.

[0130] Based on each in-focus inner cell cluster image after enhancement processing, inner cell cluster segmentation processing may be performed to segment each in-focus inner cell cluster image to obtain an inner cell cluster mask image.

[0131] S203 , performing trophoblast cell segmentation according to each focal trophoblast cell image to obtain a multi-focal trophoblast cell mask image.

[0132] Similarly, for each focal trophoblast cell image, image enhancement processing may be performed first, and then trophoblast cell segmentation processing may be performed to obtain a multi-focal trophoblast cell mask image.

[0133] S204 , constructing an inner cell mass point cloud model according to each in-focus inner cell mass mask image, and converting the inner cell mass point cloud model to obtain an inner cell mass surface model.

[0134] Optionally, based on the mask images of each in-focus inner cell mass obtained by segmentation, boundary fitting can be performed to construct an inner cell mass point cloud model, and by performing normal estimation and surface reconstruction on the inner cell mass point cloud model, the inner cell mass point cloud model can be converted into grid geometry structure data to obtain an inner cell mass surface model.

[0135] S205 . Construct a trophoblast cell point cloud model based on each focal trophoblast cell mask image, and convert the trophoblast cell point cloud model into a trophoblast cell surface model.

[0136] Referring to the processing method of the inner cell mass, a trophoblast cell surface model can also be constructed.

[0137] Optionally, in step S202, inner cell cluster segmentation is performed according to each in-focus inner cell cluster image to obtain a multi-focus inner cell cluster mask image, including: inputting each in-focus inner cell cluster image into a pre-trained inner cell cluster segmentation model, and using the inner cell cluster segmentation model to segment the inner cell cluster mask in each in-focus inner cell cluster image to obtain a multi-focus inner cell cluster mask image.

[0138] In this embodiment, a pre-trained inner cell cluster segmentation model may be used to perform segmentation processing on each in-focus inner cell cluster image, and an inner cell cluster mask image may be obtained from each in-focus inner cell cluster image.

[0139] Similarly, a pre-trained trophoblast cell segmentation model may be used to perform segmentation processing on each focal trophoblast cell image, and a trophoblast cell mask image may be obtained from each focal trophoblast cell image.

[0140] Since the characteristics of the inner cell mass and trophoblast cells are different, it is necessary to train the inner cell mass segmentation model and the trophoblast cell segmentation model separately to achieve accurate segmentation of the inner cell mass and trophoblast cells.

[0141] Figure 3 A schematic flow chart of another method for establishing an embryo assessment model provided in an embodiment of the present application; optionally, the training process of the inner cell mass segmentation model may include:

[0142] S301. Collect an initial training sample data set, which includes: multi-focal plane sample blastocyst images corresponding to multiple blastocysts, and the multi-focal plane sample blastocyst images corresponding to each blastocyst include: an original multi-focal plane sample blastocyst image and an extended multi-focal plane sample blastocyst image obtained after image interpolation; and the inner cell mass mask in each focal plane sample blastocyst image is marked.

[0143] In this embodiment, the training process of the inner cell mass segmentation model is used as an illustration, and the training process of the trophoblast cell segmentation model is not described in detail. The training process of the inner cell mass segmentation model can be referred to for execution.

[0144] Optionally, the segmentation network used in this embodiment can be a Unet++ (an improved neural network structure based on Unet) network. Compared with the traditional U-Net semantic segmentation network, Unet++ combines the DenseNet (densely connected network) structure, and the dense jump connection improves the gradient fluidity. The Unet++ network is mainly composed of an encoder (Encoder) and a decoder (Decoder), but compared with the traditional Unet network, Unet++ adds more fusion modules to the decoder to achieve multi-level feature fusion. This structure enables the network to perform feature fusion at different levels, thereby better extracting the semantic information of the image.

[0145] Optionally, images of multiple blastocysts can be collected, wherein each blastocyst can collect multi-focal plane sample blastocyst images, for example: 275 blastocysts are selected, and 21 focal plane sample blastocyst images can be collected for each blastocyst, wherein 11 focal planes can be the collected original plane sample blastocyst images, and 10 focal planes can be the extended image generated after image interpolation processing is performed on the original plane sample blastocyst images.

[0146] Of course, in actual applications, the number of selected blastocysts and the number of sample blastocyst images to be collected for each blastocyst are not limited to the data in the above example.

[0147] Since the location of the inner cell mass is not fixed, all the collected focal plane sample blastocyst images can be screened to select multiple plane sample blastocyst images with clearly visible inner cell masses, and each screened plane sample blastocyst image can be labeled. Specifically, the inner cell mass in each plane sample blastocyst image can be labeled to be used as label information in model training. Based on the labeled plane sample blastocyst images, an initial training sample data set can be obtained.

[0148] S302: Perform image amplification processing on the sample blastocyst images in the initial training sample data set to generate a target training sample data set.

[0149] In some embodiments, based on each image in the initial training sample data set obtained above, image amplification processing can also be performed to generate more sample images to be added to the training sample data set, thereby improving the generalization ability of the trained segmentation model, reducing model overfitting, improving model robustness, saving data collection costs, and accelerating the model training process.

[0150] S303, using the target training sample data set to train an inner cell mass segmentation model.

[0151] Based on the target training sample data set obtained after image amplification processing, an inner cell mass segmentation model can be trained.

[0152] Figure 4 A schematic flow chart of another method for establishing an embryo assessment model provided in an embodiment of the present application; Optionally, in step S302, image amplification processing is performed on the sample blastocyst image in the initial training sample data set to generate a target training sample data set, which may include:

[0153] S401, performing at least one amplification process on the multi-focal plane sample blastocyst image corresponding to each blastocyst in the initial training sample data set to obtain an amplified blastocyst image corresponding to each blastocyst.

[0154] The amplification processing includes: image rotation, contrast enhancement, and brightness enhancement. Of course, the processing means of image amplification are not limited to the ones listed above, and can also be achieved by image scaling, image flipping, etc.

[0155] It is worth noting that the amplification process does not change the image content of the original image, but only changes the display effect of the image. Therefore, the inner cell mass marked in the amplified blastocyst image generated after the above-mentioned image amplification process will not change, and there is no need to perform additional marking on the amplified blastocyst image.

[0156] Image augmentation increases the number and diversity of training samples by generating variations of the original dataset. This helps the model learn invariances in the data (such as rotation, translation, lighting changes, etc. of objects), thereby improving the model's ability to predict unseen data, i.e., generalization.

[0157] When the amount of training data is small, the model tends to overfit the training data, resulting in poor performance on the test set. Increasing the number of training samples through data augmentation can effectively alleviate the overfitting problem, because the model needs to learn more change patterns rather than just remembering the specific samples in the training set.

[0158] By performing operations such as flipping, rotating, scaling, and adjusting brightness / contrast on images, the model can learn the representation of images under different conditions, thereby improving the robustness to common problems in practical applications such as noise, lighting changes, and perspective changes.

[0159] In some cases, collecting large, high-quality datasets can be time-consuming and costly. Data augmentation provides a cost-effective way to expand datasets without requiring additional data collection efforts.

[0160] In practical applications, image data is often affected by various factors (such as lighting, occlusion, perspective changes, etc.). Through image enhancement, the model can learn these changes in advance, so that it can work more robustly in practical applications.

[0161] S402 , generating a target training sample data set according to the multi-focal plane sample blastocyst images corresponding to each blastocyst and the amplified blastocyst images corresponding to each blastocyst.

[0162] Optionally, the generated amplified blastocyst image is added to the initial training sample data set, and then a target training sample data set can be generated.

[0163] Figure 5 A schematic flow chart of another method for establishing an embryo assessment model provided in an embodiment of the present application; Optionally, in step S204, constructing an inner cell mass point cloud model according to each focal inner cell mass mask image may include:

[0164] S501 , taking the center point of the cell cluster mask image in the center focus as the coordinate origin, stacking and arranging the cell cluster mask images in each focus according to the depth information of each focal plane, and obtaining the arranged cell cluster mask images in each focus.

[0165] In one achievable manner, due to the difference in detailed features between the inner cell mass and the trophoblast cells, the number of images required for reconstructing the inner cell mass is less than the number of images required for reconstructing the trophoblast cells, so the number of extracted inner cell mass mask images is relatively small relative to the trophoblast cell mask images. The point cloud model of the inner cell mass cannot be accurately fitted by stacking the mask images.

[0166] Therefore, the center point of the cell cluster mask image in the central focus (F0 focus) can be first used as the coordinate origin, and the cell cluster mask image in the central focus can be used as the central layer. The remaining cell cluster mask images in each focus can be stacked and arranged in sequence above and below the central focal plane according to the depth information of the focal plane to obtain the arranged cell cluster mask images in each focus.

[0167] The depth information of the focal plane may also refer to the vertical distance between the focal plane and the central focal plane.

[0168] S502 , according to the coordinate origin and the preset rotation angle interval, traverse the mask contours of the mask images of each in-focus cell cluster to obtain a set of mask contour coordinates corresponding to each rotation angle.

[0169] The mask contour coordinate set corresponding to the rotation angle includes: coordinate information of mask contour points on the mask images of each in-focus cell cluster arranged in sequence according to the focal plane depth under the rotation angle.

[0170] Assuming that the rotation angle interval is [0,360°], the coordinate origin can be used as the center, and each rotation angle in the rotation angle interval can be used as a traversal parameter to traverse the points on the contour of each in-focus cell cluster mask image to obtain the mask contour coordinate set corresponding to each rotation angle. Among them, the mask contour coordinate set corresponding to each rotation angle contains multiple coordinate points arranged in sequence according to the focal plane depth, and each coordinate point corresponds to a cell cluster mask image in the focal plane.

[0171] S503 , performing curve fitting according to the mask contour coordinate sets corresponding to each rotation angle, and constructing an inner cell mass point cloud model.

[0172] Optionally, curve fitting can be performed on the coordinate set according to the obtained mask contour coordinate set corresponding to each rotation angle, and fitting curves of all rotation angles are drawn with the coordinate origin as a reference point to obtain the inner cell mass point cloud model.

[0173] Figure 6 This is a schematic diagram of the inner cell mass mask image stacking effect provided in an embodiment of the present application. Figure 6 The effect diagram obtained after the inner cell mass mask image is stacked in the manner of step S501 is shown.

[0174] Figure 7 A schematic diagram of an inner cell mass point cloud model fitting provided in an embodiment of the present application is shown in FIG. Figure 7 a in FIG. 5 shows a schematic diagram of a mask contour coordinate set corresponding to a rotation angle obtained after traversing the stacked inner cell mass mask images in the manner of step S502. The coordinate set may include [M0, M1, M2 ..... M8]. In practical applications, since the number of extracted inner cell mass mask images may be greater, the number of coordinate points included in the coordinate set corresponding to a rotation angle may also be greater, and is not limited to the following. Figure 7 The 8 coordinate points shown in a. Figure 7 Panel b shows a schematic diagram of the constructed inner cell mass point cloud model.

[0175] Figure 8 A schematic flow chart of another method for establishing an embryo assessment model provided in an embodiment of the present application; Optionally, in step S205, constructing a trophoblast cell point cloud model according to each focal trophoblast cell mask image may include:

[0176] S801. Stacking the focused trophoblast cell mask images according to the focal plane depth information of the focused trophoblast cell mask images to obtain a stacked trophoblast cell mask image.

[0177] Since a large number of trophoblast cell mask images are extracted, the trophoblast cell point cloud model can be simulated by stacking and arranging them.

[0178] Optionally, in the same manner as stacking the inner cell mass mask image, the central focal plane trophoblast cell mask image is used as the central layer, and the remaining focal trophoblast cell mask images are arranged in sequence above and below the central layer according to the focal plane depth information to obtain a stacked trophoblast cell mask image.

[0179] S802. Construct a trophoblast cell point cloud model based on the stacked trophoblast cell mask images.

[0180] The stacked and arranged trophoblast cell mask images can be used as a trophoblast cell point cloud model.

[0181] Fig. 9 A schematic diagram of the trophoblast cell point cloud model fitting provided in the embodiment of the present application is shown in FIG. Fig. 9 Figure a shows the schematic diagram of the effect obtained after stacking only 11 focus trophoblast cell mask images. Obviously, when there are fewer mask images, the stacking effect is not ideal. Fig. 9 b in the figure shows a schematic diagram of the effect obtained by stacking a large number of trophoblast cell mask images extracted based on this scheme. Fig. 9 The stacking effect obtained in b can be used as a point cloud model of trophoblast cells.

[0182] Optionally, in step S204, converting the inner cell mass point cloud model to obtain the inner cell mass surface model may include: performing normal estimation and surface reconstruction on the inner cell mass point cloud model to obtain the inner cell mass surface model.

[0183] Since the inner cell mass point cloud model and the trophoblast cell point cloud model obtained by the above fitting are both sets composed of some points and are not the final required three-dimensional models, the point cloud models need to be processed to convert them into geometric structure data.

[0184] The conversion of the inner cell mass point cloud model and the conversion of the trophoblast cell point cloud model are processed in the same way. Taking the conversion of the inner cell mass point cloud model as an example, the following steps may be roughly included:

[0185] 1. Point cloud preprocessing

[0186] Before normal estimation and surface reconstruction, the point cloud usually needs to be preprocessed to remove noise, outliers, and perform necessary smoothing.

[0187] Noise Removal: Statistical filters such as radius or statistical outlier removal can be used to remove noise points.

[0188] Outlier removal: This is done by calculating the distance of each point to its nearest neighbor and removing those points whose distance is too large.

[0189] Smoothing: The point cloud can be smoothed using Moving Least Squares (MLS) or other smoothing algorithms.

[0190] 2. Normal Estimation

[0191] Normal estimation is an important step in surface reconstruction, since normal information is crucial for understanding the shape and orientation of the local surface.

[0192] Normal estimation based on PCA: For each point in the point cloud, select the k nearest neighboring points around it and construct a local covariance matrix. By calculating the eigenvalues ​​and eigenvectors of the matrix, the eigenvector corresponding to the minimum eigenvalue is the normal direction of the point.

[0193] Other methods: such as normal estimation based on Voronoi diagram, normal estimation based on integral image, etc.

[0194] 3. Surface reconstruction

[0195] Surface reconstruction is the process of converting point cloud data into a continuous surface representation.

[0196] Moving Least Squares (MLS): This is a popular surface reconstruction method that generates a smooth surface by fitting each point in the point cloud locally. MLS estimates not only the position of the point but also its normal and curvature.

[0197] Delaunay triangulation: Convert the point cloud into a triangular mesh. This method first triangulates the point cloud using an algorithm (such as Qhull or CGAL library), and then may also require mesh optimization (such as Laplacian smoothing) to improve the mesh quality.

[0198] Implicit surface reconstruction: such as radial basis function (RBF), Poisson surface reconstruction, etc. These methods reconstruct the surface by solving an implicit function (such as the signed distance function), which is negative inside the surface, positive outside the surface, and zero on the surface.

[0199] According to the above method, the inner cell mass point cloud model can be converted into the inner cell mass surface model and the trophoblast cell point cloud model can be converted into the trophoblast cell surface model.

[0200] Fig.10 This is a schematic diagram of a curved surface model provided in an embodiment of the present application. Fig.10 a in the figure shows the original F0 focal plane image; Fig.10 b in the figure shows the reconstructed surface model of the inner cell mass; Fig.10 Figure c shows the reconstructed trophoblast cell surface model.

[0201] Fig.11 A schematic flow chart of another method for establishing an embryo evaluation model provided in an embodiment of the present application; Optionally, in step S103, extracting at least multiple pieces of three-dimensional feature information based on the blastocyst three-dimensional model may include:

[0202] S1101. Extract multiple basic three-dimensional feature information based on the three-dimensional model of the blastocyst, the basic three-dimensional feature information including: the volume of the blastocyst, the surface area of ​​the blastocyst, the diameter of the blastocyst, the volume of the cavity, the number of trophoblast cells, the surface area of ​​the inner cell mass, and the volume of the inner cell mass.

[0203] In some embodiments, based on the reconstructed three-dimensional model of the blastocyst, at least multiple basic three-dimensional feature information such as the volume of the blastocyst, the surface area of ​​the blastocyst, the diameter of the blastocyst, the volume of the cavity, the number of trophoblast cells, the surface area of ​​the inner cell mass, and the volume of the inner cell mass can be extracted.

[0204] S1102: Perform feature mining based on multiple pieces of basic three-dimensional feature information to obtain multiple pieces of extended three-dimensional feature information.

[0205] By mining the features of various basic three-dimensional feature information, multiple pieces of extended three-dimensional feature information can be generated.

[0206] There are many methods for feature mining, such as feature discretization, high-order feature mining, aggregated features, PCA (Principal Component Analysis) for feature dimensionality reduction, etc. This solution combines actual application scenarios and can use high-order feature mining methods to mine multiple extended three-dimensional feature information from multiple basic three-dimensional feature information.

[0207] a) Feature discretization: Use quartiles or decision trees to find feature thresholds and discretize continuous features.

[0208] b) High-order feature mining: Consider the interaction between features, generate high-order features, or perform polynomial transformation on the original features.

[0209] c) Aggregate features: Aggregate the original features, such as summation, averaging, etc.

[0210] d) PCA for feature dimensionality reduction: Using the PCA algorithm to reduce the dimensionality of features can reduce the dimension of the data set while retaining the important information of the original data as much as possible.

[0211] S1103 . Obtain multiple pieces of three-dimensional feature information according to multiple pieces of basic three-dimensional feature information and multiple pieces of extended three-dimensional feature information.

[0212] Then, multiple pieces of three-dimensional feature information may be constituted by multiple pieces of basic three-dimensional feature information and multiple pieces of extended three-dimensional feature information.

[0213] Optionally, in step S1102, feature mining is performed based on multiple basic three-dimensional feature information to obtain multiple extended three-dimensional feature information, which may include: determining the ratio of the surface area to the volume of the blastocyst based on the volume of the blastocyst and the surface area of ​​the blastocyst; determining the density of the trophoblast cells based on the number of trophoblast cells and the surface area of ​​the blastocyst; determining the ratio of the surface area to the volume of the inner cell mass based on the surface area of ​​the inner cell mass and the volume of the inner cell mass; and determining the ratio of the surface area to the volume of the blastocyst, the density of the trophoblast cells, and the ratio of the surface area to the volume of the inner cell mass as multiple extended three-dimensional feature information.

[0214] In some embodiments, the surface area to volume ratio of the blastocyst can be calculated based on the volume of the blastocyst and the surface area of ​​the blastocyst; the trophoblast cell density can be calculated based on the number of trophoblast cells and the surface area of ​​the blastocyst using the formula number of trophoblast cells / surface area of ​​the blastocyst*1000; the surface area to volume ratio of the inner cell mass can be calculated based on the surface area of ​​the inner cell mass and the volume of the inner cell mass.

[0215] It should be noted that after extracting multiple basic three-dimensional feature information, data cleaning can be performed first to clean the features from the perspectives of data distribution and feature definition, so as to reduce the misleading information that may be caused by errors in the three-dimensional reconstruction of the blastocyst.

[0216] Based on the multiple three-dimensional feature information obtained after feature mining, feature correlation analysis can also be performed to reduce the tediousness of the entire analysis process. Feature correlation analysis can be performed from both linear and nonlinear perspectives, and scatter plots of multiple basic three-dimensional feature information and multiple extended three-dimensional feature information can be drawn to observe the linear and nonlinear relationships between features.

[0217] Fig.12 A flowchart of another method for establishing an embryo assessment model provided in an embodiment of the present application; Optionally, in step S104, feature screening is performed according to various three-dimensional feature information to determine target three-dimensional feature information, which may include:

[0218] S1201. Based on the three-dimensional feature information, use variance analysis or chi-square verification method to determine the relationship between each three-dimensional feature and the clinical outcome of the embryo.

[0219] In some embodiments, for continuous data, Mannwhitney or variance analysis methods can be used for data screening, while for categorical data, chi-square test methods can be used for data screening, so as to analyze the correlation between each three-dimensional feature data and clinical outcomes.

[0220] The Mann-Whitney U test, also known as the Wilcoxon rank sum test, is a nonparametric statistical test method. It is mainly used to compare whether there is a significant difference between the medians of two independent samples. This test method does not require any assumptions about the distribution of the data, so it is applicable to various types of data, including continuous data, non-normal distribution data, and rank data.

[0221] ANOVA is a statistical method used to compare the differences between multiple sample means. It is mainly used to analyze whether one or more independent variables have a significant effect on the dependent variable, and which factors have the most significant effect on the dependent variable. ANOVA requires that the data meet the assumptions of normal distribution and homogeneity of variance.

[0222] Chi-square test, also known as χ2 test or X2 test, is a widely used hypothesis testing method, mainly used to analyze categorical data. Its basic principle is to determine whether there is a significant relationship or independence between two or more categorical variables by comparing the difference between the actual observed value and the theoretical expected value.

[0223] The specific screening and analysis process can be understood by referring to the principles and implementation details of the above methods, which will not be described in detail here.

[0224] S1202. Filter target three-dimensional feature information from each three-dimensional feature information according to the relationship between each three-dimensional feature and the clinical outcome of the embryo.

[0225] Based on the above analysis of each three-dimensional feature information, the relationship between each three-dimensional feature and the clinical outcome of the embryo can be determined, so as to screen out each target three-dimensional feature information that has a greater impact on the clinical outcome from the three-dimensional feature information.

[0226] In some embodiments, feature analysis shows that: compared with embryos with failed pregnancy, embryos with successful pregnancy have larger blastocysts (larger blastocyst diameter, surface area, volume and cavity volume, p<0.001); at the same time, the number of trophoblast cells is more and the density is lower (p<0.001); in addition, the surface area to volume ratio of the blastocyst of embryos with successful pregnancy is smaller (p<0.001); the features are discretized using a decision tree algorithm, and the surface area of ​​the inner cell mass is greater than 11082um2, and the blastocyst has the highest clinical pregnancy rate; the surface area to volume ratio of the inner cell mass is between 0.186 and 0.196, and the blastocyst has the highest clinical pregnancy rate; the volume of the inner cell mass has no direct relationship with the pregnancy outcome.

[0227] Of course, in addition to analyzing the differences in clinical pregnancy outcomes of various three-dimensional feature information, in actual applications, the differences in abortion rates, live birth rates, etc. of various three-dimensional feature information can also be analyzed.

[0228] Optionally, in step S104, constructing an embryo assessment model based on the target three-dimensional feature information may include: adding one or more items of the target three-dimensional feature information as new training parameters to the original training parameters of the original embryo assessment model to construct a new embryo assessment model.

[0229] In one feasible manner, in order to further measure the performance of the screened target three-dimensional feature information in the clinical prediction model, this embodiment can add the screened target three-dimensional feature information as a new training parameter to the original training parameter for training a new embryo scoring model.

[0230] Since the patient's age has a crucial impact on the clinical outcomes, the above-mentioned target three-dimensional feature information and patient information (age) are used to establish a decision tree model: In order to verify the predictive performance of the model, the data set is divided into a training set (2018-2020) and a validation set (2021) according to time, and GridSearch is used to establish a prediction model for clinical pregnancy outcomes. The AUC of the model on the training set is 0.759, and the AUC on the test set is 0.708. AUC refers to the area under the ROC curve, an indicator for measuring model performance in machine learning. The value range of AUC is usually between 0.5 and 1. The closer to 1, the better the classifier performance, and 0.5 means that the classifier performance is equivalent to random guessing.

[0231] Table 1 shows the differences in clinical pregnancy outcomes among various three-dimensional feature information:

[0232] Table 1

[0233]

[0234] Fig.13 Schematic diagram of ROC curve of a clinical pregnancy model provided in the embodiment of the present application. Fig.13 As shown in the figure, the model has a slight overfitting, but the overfitting is within an acceptable range. Curve 1 indicates that the ACU on the training set is 0.759; Curve 2 indicates that the ACU on the test set is 0.708.

[0235] In this embodiment, only the construction of a clinical pregnancy prediction model is taken as an example. In actual applications, the extracted target three-dimensional feature information can also be used to construct a miscarriage prediction model or a live birth rate prediction model. Whether it is a clinical pregnancy prediction model, a miscarriage prediction model or a live birth rate prediction model, it can be called an embryo assessment model, which is used to evaluate the development status of the embryo.

[0236] In summary, the method for establishing an embryo assessment model provided in this embodiment includes: performing image interpolation processing based on the multi-focus original plane image of the blastocyst obtained under the time difference incubator to obtain a multi-focus target image corresponding to the blastocyst; performing three-dimensional reconstruction of the blastocyst according to the multi-focus target image to generate a three-dimensional model of the blastocyst; extracting multiple three-dimensional feature information according to the three-dimensional model of the blastocyst; performing feature screening according to each three-dimensional feature information, determining the target three-dimensional feature information, and constructing an embryo assessment model based on the target three-dimensional feature information. This method performs image interpolation processing based on the collected multi-focus original plane image to obtain a multi-focus target image, expands the image details, and performs three-dimensional reconstruction based on the multi-focus target image. The obtained three-dimensional model of the blastocyst can express more feature details; thereby, the multiple three-dimensional feature information extracted based on the blastocyst three-dimensional model is more accurate and can better express the development of the blastocyst. The embryo assessment model constructed based on the screened target three-dimensional feature information has better predictive ability and can more accurately predict the developmental potential of the embryo.

[0237] In addition, by training the segmentation model, the masks of the inner cell mass image and the trophoblast cell image can be accurately segmented, so that a three-dimensional model of the blastocyst can be constructed based on the segmentation results, thereby improving the construction efficiency and accuracy of the three-dimensional model.

[0238] The following describes the apparatus, device, storage medium, etc. used to execute the method for establishing the embryo assessment model provided in the present application. The specific implementation process and technical effects are described above and will not be repeated below.

[0239] Fig.14 A schematic diagram of an apparatus for establishing an embryo assessment model provided in an embodiment of the present application, the functions implemented by the apparatus correspond to the steps performed by the above method. The apparatus can be understood as the above-mentioned computer device or server, or the processor of the server, or can also be understood as a component independent of the above-mentioned server or processor that implements the functions of the present application under the control of the server, such as Fig.14 As shown, the apparatus may include: a processing module 140, a reconstruction module 141, an extraction module 142, and a model building module 143;

[0240] A processing module 140 is used to perform image interpolation processing based on the multi-focus original plane image of the blastocyst obtained in the time difference incubator to obtain a multi-focus target image corresponding to the blastocyst;

[0241] The reconstruction module 141 is used to perform three-dimensional reconstruction of the blastocyst according to the multi-focus target image to generate a three-dimensional model of the blastocyst; the three-dimensional model of the blastocyst includes: a blastocyst surface model, an inner cell mass surface model and a trophoblast cell surface model;

[0242] An extraction module 142 is used to extract multiple pieces of three-dimensional feature information based on the three-dimensional model of the blastocyst, where the three-dimensional feature information includes: three-dimensional features of the blastocyst, three-dimensional features of the inner cell mass, and three-dimensional features of the trophoblast cells;

[0243] The model building module 143 is used to perform feature screening according to various three-dimensional feature information, determine target three-dimensional feature information, and build an embryo evaluation model based on the target three-dimensional feature information.

[0244] Optionally, the reconstruction module 141 is specifically configured to divide the multifocal target image according to the image regions of interest in the multifocal target image to obtain a multifocal inner cell mass image and a multifocal trophoblast cell image;

[0245] According to each focal inner cell cluster image, inner cell cluster segmentation is performed to obtain a multi-focal inner cell cluster mask image;

[0246] According to each focal trophoblast cell image, trophoblast cell segmentation is performed to obtain a multi-focal trophoblast cell mask image;

[0247] According to the mask images of each focal inner cell mass, a point cloud model of the inner cell mass is constructed, and according to the point cloud model of the inner cell mass, a surface model of the inner cell mass is converted;

[0248] A trophoblast cell point cloud model is constructed according to each focal trophoblast cell mask image, and a trophoblast cell surface model is converted according to the trophoblast cell point cloud model.

[0249] Each focal inner cell cluster image is input into a pre-trained inner cell cluster segmentation model, and the inner cell cluster segmentation model segments the inner cell cluster mask in each focal inner cell cluster image to obtain a multi-focal inner cell cluster mask image.

[0250] Optionally, a training module is also included;

[0251] A training module is used to collect an initial training sample data set, wherein the initial training sample data set includes: multi-focal plane sample blastocyst images corresponding to a plurality of blastocysts, wherein the multi-focal plane sample blastocyst images corresponding to each blastocyst include: an original multi-focal plane sample blastocyst image and an extended multi-focal plane sample blastocyst image obtained after image interpolation; and the inner cell mass mask in each focal plane sample blastocyst image has a mark;

[0252] Performing image amplification processing on the sample blastocyst images in the initial training sample data set to generate a target training sample data set;

[0253] The target training sample dataset is used to train the inner cell mass segmentation model.

[0254] Optionally, the training module is specifically used to perform at least one amplification process on the multi-focal plane sample blastocyst images corresponding to each blastocyst in the initial training sample data set to obtain an amplified blastocyst image corresponding to each blastocyst; the amplification process includes: image rotation, contrast enhancement, and brightness enhancement;

[0255] A target training sample data set is generated according to the multi-focal plane sample blastocyst images corresponding to each blastocyst and the amplified blastocyst images corresponding to each blastocyst.

[0256] Optionally, the reconstruction module 141 is specifically configured to stack and arrange the cell cluster mask images in each focus according to the depth information of each focal plane, using the center point of the cell cluster mask image in the center focus as the coordinate origin, to obtain the arranged cell cluster mask images in each focus;

[0257] According to the coordinate origin and the preset rotation angle interval, the mask contours of the mask images of the in-focus cell clusters are traversed to obtain a set of mask contour coordinates corresponding to each rotation angle; wherein the set of mask contour coordinates corresponding to the rotation angle includes: coordinate information of mask contour points on the mask images of the in-focus cell clusters arranged in sequence according to the focal plane depth under the rotation angle;

[0258] According to the mask contour coordinate set corresponding to each rotation angle, curve fitting is performed to construct the inner cell mass point cloud model.

[0259] Optionally, the reconstruction module 141 is specifically configured to stack each of the focused trophoblast cell mask images according to the focal plane depth information of each of the focused trophoblast cell mask images to obtain a stacked trophoblast cell mask image;

[0260] According to the stacked trophoblast cell mask images, a trophoblast cell point cloud model is constructed.

[0261] Optionally, the reconstruction module 141 is specifically used to perform normal estimation and surface reconstruction on the inner cell mass point cloud model to obtain the inner cell mass surface model.

[0262] Optionally, the extraction module 142 is specifically used to extract multiple basic three-dimensional feature information according to the three-dimensional model of the blastocyst, the basic three-dimensional feature information including: the volume of the blastocyst, the surface area of ​​the blastocyst, the diameter of the blastocyst, the volume of the cavity, the number of trophoblast cells, the surface area of ​​the inner cell mass, and the volume of the inner cell mass;

[0263] Based on multiple basic three-dimensional feature information, feature mining is performed to obtain multiple extended three-dimensional feature information;

[0264] Multiple pieces of three-dimensional feature information are obtained according to multiple pieces of basic three-dimensional feature information and multiple pieces of extended three-dimensional feature information.

[0265] Optionally, the extraction module 142 is specifically configured to determine a ratio of the surface area to the volume of the blastocyst according to the volume of the blastocyst and the surface area of ​​the blastocyst;

[0266] Determine the density of trophoblast cells based on the number of trophoblast cells and the surface area of ​​the blastocyst;

[0267] Determine the ratio of the surface area to the volume of the inner cell mass based on the surface area of ​​the inner cell mass and the volume of the inner cell mass;

[0268] The ratio of the surface area to the volume of the blastocyst, the density of the trophoblast cells, and the ratio of the surface area to the volume of the inner cell mass are determined as multiple pieces of extended three-dimensional feature information.

[0269] Optionally, the model building module 143 is specifically used to determine the relationship between each three-dimensional feature and the embryo clinical outcome by using a variance analysis method or a chi-square verification method according to each three-dimensional feature information;

[0270] According to the relationship between each three-dimensional feature and the clinical outcome of the embryo, the target three-dimensional feature information is screened from the three-dimensional feature information.

[0271] Optionally, the model building module 143 is specifically used to add one or more items of the target three-dimensional feature information as new training parameters to the original training parameters of the original embryo assessment model to construct a new embryo assessment model.

[0272] The above-mentioned device is used to execute the method provided by the aforementioned embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0273] The above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more digital singnal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processors that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0274] The above modules can be connected or communicated with each other via a wired connection or a wireless connection. The wired connection may include a metal cable, an optical cable, a hybrid cable, etc., or any combination thereof. The wireless connection may include a connection in the form of a LAN, a WAN, Bluetooth, a ZigBee, or NFC, or any combination thereof. Two or more modules may be combined into a single module, and any one module may be divided into two or more units. Those skilled in the art will clearly understand that for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application.

[0275] Fig.15 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, which may be a computing device with data processing capabilities.

[0276] The device may include: a processor 801 and a storage medium 802 .

[0277] The storage medium 802 is used to store programs, and the processor 801 calls the programs stored in the storage medium 802 to execute the above method embodiment. The specific implementation method and technical effect are similar and will not be repeated here.

[0278] Among them, the storage medium 802 stores program code, and when the program code is executed by the processor 801, the processor 801 executes the various steps in the method for establishing an embryo assessment model according to various exemplary embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0279] Processor 801 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor (DigitalSignal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as a hardware processor to be executed, or a combination of hardware and software modules in the processor can be executed.

[0280] Storage medium 802, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules. Storage medium may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type storage medium, random access storage medium (Random Access Memory, RAM), static random access storage medium (Static Random Access Memory, SRAM), programmable read-only storage medium (Programmable Read Only Memory, PROM), read-only storage medium (Read Only Memory, ROM), electrically erasable programmable read-only storage medium (Electrically Erasable Programmable Read-Only Memory, EEPROM), magnetic storage medium, disk, optical disk, etc. Storage medium is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The storage medium 802 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0281] Optionally, the present application also provides a program product, such as a computer-readable storage medium, including a program, which is used to execute the above method embodiment when executed by a processor.

[0282] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0283] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0284] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0285] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only storage medium (English: Read-Only Memory, referred to as: ROM), random access storage medium (English: Random Access Memory, referred to as: RAM), disk or optical disk and other media that can store program code.

Claims

1. A method for establishing an embryo assessment model, characterized in that: include: Performing image interpolation processing based on the multi-focus original plane image of the blastocyst obtained in the time difference incubator to obtain a multi-focus target image corresponding to the blastocyst; Performing three-dimensional reconstruction of the blastocyst according to the multi-focus target image to generate a three-dimensional model of the blastocyst; The blastocyst three-dimensional model includes: a blastocyst curved surface model, an inner cell mass curved surface model and a trophoblast cell curved surface model; Extracting multiple pieces of three-dimensional feature information according to the three-dimensional model of the blastocyst, wherein the three-dimensional feature information includes: three-dimensional features of the blastocyst, three-dimensional features of the inner cell mass, and three-dimensional features of the trophoblast cells; Feature screening is performed based on each item of three-dimensional feature information to determine the target three-dimensional feature information, and an embryo evaluation model is constructed based on the target three-dimensional feature information.

2. The method according to claim 1, characterized in that The step of performing three-dimensional reconstruction of the blastocyst according to the multi-focus target image to generate a three-dimensional model of the blastocyst includes: According to the image regions of interest in the multifocal target image, the multifocal target image is divided to obtain a multifocal inner cell mass image and a multifocal trophoblast cell image; According to each focal inner cell cluster image, inner cell cluster segmentation is performed to obtain a multi-focal inner cell cluster mask image; According to each focal trophoblast cell image, trophoblast cell segmentation is performed to obtain a multi-focal trophoblast cell mask image; According to each focal inner cell mass mask image, an inner cell mass point cloud model is constructed, and according to the inner cell mass point cloud model, an inner cell mass surface model is converted; A trophoblast cell point cloud model is constructed according to each focal trophoblast cell mask image, and a trophoblast cell surface model is converted according to the trophoblast cell point cloud model.

3. The method according to claim 2, characterized in that The inner cell cluster segmentation is performed according to each focal inner cell cluster image to obtain a multi-focal inner cell cluster mask image, including: Each in-focus inner cell cluster image is input into a pre-trained inner cell cluster segmentation model, and the inner cell cluster segmentation model is used to segment the inner cell cluster mask in each in-focus inner cell cluster image to obtain a multi-focus inner cell cluster mask image.

4. The method according to claim 3, characterized in that The training process of the inner cell mass segmentation model includes: Collecting an initial training sample data set, the initial training sample data set comprising: multi-focal plane sample blastocyst images corresponding to a plurality of blastocysts, the multi-focal plane sample blastocyst images corresponding to each blastocyst comprising: an original multi-focal plane sample blastocyst image and an extended multi-focal plane sample blastocyst image obtained after image interpolation; and the inner cell mass mask in each focal plane sample blastocyst image has a mark; Performing image amplification processing on the sample blastocyst images in the initial training sample data set to generate a target training sample data set; The target training sample data set is used to train the inner cell mass segmentation model.

5. The method according to claim 4, characterized in that The step of performing image amplification processing on the sample blastocyst image in the initial training sample data set to generate a target training sample data set includes: Perform at least one amplification process on the multi-focal plane sample blastocyst images corresponding to each blastocyst in the initial training sample data set to obtain an amplified blastocyst image corresponding to each blastocyst; the amplification process includes: image rotation, contrast enhancement, and brightness enhancement; The target training sample data set is generated according to the multi-focal plane sample blastocyst images corresponding to each blastocyst and the amplified blastocyst images corresponding to each blastocyst.

6. The method according to claim 2, characterized in that The method of constructing an inner cell cluster point cloud model according to each focal inner cell cluster mask image comprises: Taking the center point of the cell cluster mask image in the center focus as the coordinate origin, stacking and arranging the cell cluster mask images in each focus according to the depth information of each focal plane, and obtaining the arranged cell cluster mask images in each focus; According to the coordinate origin, according to the preset rotation angle interval, the mask contours of each in-focus cell cluster mask image are traversed to obtain a mask contour coordinate set corresponding to each rotation angle; wherein the mask contour coordinate set corresponding to the rotation angle includes: coordinate information of mask contour points on each in-focus cell cluster mask image arranged in sequence according to the focal plane depth under the rotation angle; Curve fitting is performed according to the mask contour coordinate set corresponding to each rotation angle to construct the inner cell mass point cloud model.

7. The method according to claim 2, characterized in that The step of constructing a trophoblast cell point cloud model based on each focal trophoblast cell mask image comprises: stacking the focal trophoblast cell mask images according to the focal plane depth information of the focal trophoblast cell mask images to obtain a stacked trophoblast cell mask image; The trophoblast cell point cloud model is constructed according to the stacked trophoblast cell mask images.

8. The method according to claim 2, characterized in that: The step of converting the inner cell mass point cloud model to obtain an inner cell mass surface model comprises: Normal estimation and surface reconstruction are performed on the inner cell mass point cloud model to obtain the inner cell mass surface model.

9. The method according to claim 1, characterized in that: The step of extracting at least multiple pieces of three-dimensional feature information based on the blastocyst three-dimensional model includes: Extracting a plurality of basic three-dimensional feature information according to the three-dimensional model of the blastocyst, wherein the basic three-dimensional feature information includes: the volume of the blastocyst, the surface area of ​​the blastocyst, the diameter of the blastocyst, the volume of the cavity, the number of trophoblast cells, the surface area of ​​the inner cell mass, and the volume of the inner cell mass; Performing feature mining based on the multiple pieces of basic three-dimensional feature information to obtain multiple pieces of extended three-dimensional feature information; The multiple pieces of three-dimensional feature information are obtained according to the multiple pieces of basic three-dimensional feature information and the multiple pieces of extended three-dimensional feature information.

10. The method according to claim 9, characterized in that The method of performing feature mining based on the plurality of basic three-dimensional feature information to obtain a plurality of extended three-dimensional feature information includes: Determining a ratio of the surface area to the volume of the blastocyst according to the volume of the blastocyst and the surface area of ​​the blastocyst; Determining the density of the trophoblast cells according to the number of the trophoblast cells and the surface area of ​​the blastocyst; determining a ratio of the surface area to the volume of the inner cell mass according to the surface area of ​​the inner cell mass and the volume of the inner cell mass; The ratio of the surface area to the volume of the blastocyst, the density of the trophoblast cells, and the ratio of the surface area to the volume of the inner cell mass are determined as the multiple pieces of extended three-dimensional feature information.

11. The method according to claim 1, characterized in that: The step of screening the features according to the three-dimensional feature information to determine the target three-dimensional feature information includes: According to the information of each three-dimensional feature, the variance analysis method or chi-square verification method is used to determine the relationship between each three-dimensional feature and the clinical outcome of the embryo; According to the relationship between each three-dimensional feature and the clinical outcome of the embryo, the target three-dimensional feature information is screened from the three-dimensional feature information.

12. The method according to claim 1, characterized in that The step of constructing an embryo assessment model based on the target three-dimensional feature information includes: One or more items of the target three-dimensional feature information are added as new training parameters to the original training parameters of the original embryo assessment model to construct a new embryo assessment model.

13. A device for establishing an embryo evaluation model, characterized in that: include: a processing module, a reconstruction module, an extraction module, and a model building module; The processing module is used to perform image interpolation processing based on the multi-focus original plane image of the blastocyst obtained in the time difference incubator to obtain a multi-focus target image corresponding to the blastocyst; The reconstruction module is used to perform three-dimensional reconstruction of the blastocyst according to the multi-focus target image to generate a three-dimensional model of the blastocyst; The blastocyst three-dimensional model includes: a blastocyst curved surface model, an inner cell mass curved surface model and a trophoblast cell curved surface model; The extraction module is used to extract multiple pieces of three-dimensional feature information according to the blastocyst three-dimensional model, wherein the three-dimensional feature information includes: three-dimensional features of the blastocyst, three-dimensional features of the inner cell mass, and three-dimensional features of the trophoblast cells; The model building module is used to perform feature screening according to various three-dimensional feature information, determine target three-dimensional feature information, and build an embryo evaluation model based on the target three-dimensional feature information.

14. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to implement the method for establishing an embryo assessment model as described in any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the method for establishing an embryo assessment model as described in any one of claims 1 to 12 is implemented.

Images