Three-dimensional fitting reconstruction method and device of blastula, electronic equipment and storage medium

Abstract

The application provides a blastocyst three-dimensional fitting reconstruction method and device, electronic equipment and storage medium. The method comprises the following steps: acquiring a plurality of focal plane images of a blastocyst; identifying a trophoblast cell mask and an inner cell mass mask in the blastocyst based on the plurality of focal plane images; constructing a trophoblast cell grid model and an inner cell mass grid model according to the trophoblast cell mask and the inner cell mass mask; constructing a depth map of the blastocyst according to the plurality of focal plane images; performing image fusion on the plurality of focal plane images to obtain a texture image; performing plane mapping according to the texture image and the depth map to generate an internal three-dimensional point cloud model of the blastocyst; constructing a three-dimensional geometric model of the blastocyst according to the trophoblast cell grid model, the inner cell mass grid model and the internal three-dimensional point cloud model; and mapping the texture image to the surface of the three-dimensional geometric model to generate a blastocyst three-dimensional model. The reconstruction based on the method does not affect the development of the embryo, and is low in cost and safe and reliable.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a three-dimensional fitting reconstruction method and device for blastocyst, electronic equipment and storage medium. BACKGROUND

[0002] With the development of embryology, embryologists are more expected to observe embryos from a higher dimension and hope to obtain a complete, morphological feature information rich and spatial feature model for embryo evaluation and research.

[0003] At present, the sequence fluorescence section data is obtained by using fluorescence staining and confocal microscopic imaging to reconstruct the three-dimensional model of the embryo. However, such a method needs to stain the embryo and irradiate high-energy spectrum, which will cause irreversible damage to the embryo. SUMMARY

[0004] Therefore, the embodiments of the present application provide a three-dimensional fitting reconstruction method and device for blastocyst, electronic equipment and storage medium to solve the problem of irreversible damage to the embryo when reconstructing the three-dimensional model of the embryo.

[0005] In a first aspect, the embodiments of the present application provide a three-dimensional fitting reconstruction method for blastocyst, comprising:

[0006] obtaining a plurality of focal plane images of the blastocyst;

[0007] based on the plurality of focal plane images, identifying a mask of trophoblast cells and a mask of inner cell mass in the blastocyst;

[0008] constructing a trophoblast cell grid model and an inner cell mass grid model according to the mask of trophoblast cells and the mask of inner cell mass, respectively;

[0009] constructing a depth map of the blastocyst according to the plurality of focal plane images;

[0010] performing image fusion on the plurality of focal plane images to obtain a texture image;

[0011] performing plane mapping according to the texture image and the depth map to generate an internal three-dimensional point cloud model of the blastocyst;

[0012] constructing a three-dimensional geometric model of the blastocyst according to the trophoblast cell grid model, the inner cell mass grid model and the internal three-dimensional point cloud model;

[0013] mapping the texture image to the surface of the three-dimensional geometric model to generate a three-dimensional model of the blastocyst.

[0014] In an optional implementation, the constructing the depth map of the blastula according to the plurality of focal plane images comprises:

[0015] obtaining the sharpness of each pixel point in the plurality of focal plane images;

[0016] determining a target focal plane with the maximum sharpness of the each pixel point from the plurality of focal planes corresponding to the plurality of focal plane images, to obtain the depth information of the target focal plane;

[0017] constructing the depth map according to the depth information of the target focal plane.

[0018] In an optional implementation, the obtaining the plurality of focal plane images of the blastula comprises:

[0019] obtaining a plurality of first focal plane images of the blastula;

[0020] performing interpolation processing on the plurality of first focal plane images to obtain a plurality of second focal plane images of the blastula, wherein the plurality of focal plane images comprise the plurality of first focal plane images and the plurality of second focal plane images.

[0021] In an optional implementation, before the constructing the trophoblast cell grid model and the inner cell mass grid model according to the mask of the trophoblast cells and the mask of the inner cell mass respectively, the method further comprises:

[0022] segmenting the plurality of focal plane images to obtain the blastula in the plurality of focal plane images;

[0023] if there is a target focal plane image in the plurality of focal plane images, moving the blastula in the target focal plane image to the center of a preset field of view and performing boundary padding to regenerate the target focal plane image, wherein the blastula in the target focal plane image is not in the center of the preset field of view.

[0024] In an optional implementation, the identifying the mask of the trophoblast cells and the mask of the inner cell mass in the blastula based on the plurality of focal plane images comprises:

[0025] marking a trophoblast cell region on each of the plurality of focal plane images according to a preset marking pattern corresponding to the plurality of focal plane images;

[0026] identifying the trophoblast cell region to obtain the mask of the trophoblast cells;

[0027] identifying an inner cell mass region inside the trophoblast cell region to obtain the mask of the inner cell mass.

[0028] In an optional implementation, the method further comprises:

[0029] determine a preset focal plane in the plurality of focal planes corresponding to the plurality of focal plane images as an optimal focal plane of the trophoblast cells;

[0030] determine the optimal focal plane of the inner cell mass from the plurality of focal planes according to the area of the mask of the inner cell mass in the plurality of focal plane images;

[0031] determine the three-dimensional position of the trophoblast cell grid model according to the optimal focal plane of the trophoblast cells, and determine the three-dimensional position of the inner cell mass grid model according to the optimal focal plane of the inner cell mass;

[0032] The constructing the three-dimensional geometric model of the blastula according to the trophoblast cell grid model, the inner cell mass grid model and the internal three-dimensional point cloud model comprises:

[0033] performing position matching of the trophoblast cell grid model and the internal three-dimensional point cloud model according to the three-dimensional position of the trophoblast cell grid model, and performing position matching of the inner cell mass grid model and the internal three-dimensional point cloud model according to the three-dimensional position of the inner cell mass grid model, to obtain the three-dimensional geometric model.

[0034] In an optional implementation, the determining the optimal focal plane of the inner cell mass from the plurality of focal planes according to the area of the mask of the inner cell mass in the plurality of focal plane images comprises:

[0035] determining a target focal plane image with the largest area of the mask of the inner cell mass from the plurality of focal plane images according to the area of the mask of the inner cell mass in the plurality of focal plane images;

[0036] segmenting the plurality of focal plane images by taking the mask of the inner cell mass in the target focal plane image as a segmentation mask to obtain a plurality of segmented focal plane images;

[0037] determining the optimal focal plane of the inner cell mass from the plurality of focal planes according to the definition of the plurality of segmented focal plane images.

[0038] In a second aspect, the embodiments of the present application further provide a three-dimensional fitting reconstruction device of a blastula, comprising:

[0039] an acquisition module configured to acquire a plurality of focal plane images of a blastula;

[0040] an identification module configured to identify a mask of trophoblast cells and a mask of an inner cell mass in the blastula based on the plurality of focal plane images;

[0041] constructing a trophoblast cell grid model and a inner cell mass grid model according to the mask of the trophoblast cell and the mask of the inner cell mass respectively;

[0042] The constructing module is further configured to construct a depth map of the blastula according to the plurality of focal plane images.

[0043] The fusing module is configured to fuse the plurality of focal plane images to obtain a texture image.

[0044] The mapping module is configured to perform plane mapping according to the texture image and the depth map to generate an internal three-dimensional point cloud model of the blastula.

[0045] The constructing module is further configured to construct a three-dimensional geometric model of the blastula according to the trophoblast cell grid model, the inner cell mass grid model and the internal three-dimensional point cloud model.

[0046] The mapping module is further configured to map the texture image to a surface of the three-dimensional geometric model to generate a three-dimensional model of the blastula.

[0047] In an optional implementation, the constructing module is specifically configured to:

[0048] obtain the sharpness of each pixel point in the plurality of focal plane images;

[0049] determine a target focal plane with the maximum sharpness of the each pixel point from a plurality of focal planes corresponding to the plurality of focal plane images to obtain depth information of the target focal plane;

[0050] construct the depth map according to the depth information of the target focal plane.

[0051] In an optional implementation, the obtaining module is specifically configured to:

[0052] obtain a plurality of first focal plane images of the blastula;

[0053] perform frame interpolation processing on the plurality of first focal plane images to obtain a plurality of second focal plane images of the blastula, and the plurality of focal plane images include the plurality of first focal plane images and the plurality of second focal plane images.

[0054] In an optional implementation, the apparatus further includes:

[0055] The segmenting module is configured to segment the plurality of focal plane images to obtain the blastula in the plurality of focal plane images.

[0056] The moving module is configured to move the blastula in a target focal plane image to a preset central field of view and perform boundary padding to regenerate the target focal plane image if the target focal plane image exists in the plurality of focal plane images, and the blastula in the target focal plane image is not in the preset central field of view.

[0057] In an optional implementation, the identifying module is specifically configured to:

[0058] According to preset mark patterns corresponding to the plurality of focal plane images, mark a trophoblast cell region on each of the plurality of focal plane images;

[0059] Identify the trophoblast cell region to obtain a mask of the trophoblast cell;

[0060] Identify an inner cell mass region located in the trophoblast cell region to obtain a mask of the inner cell mass.

[0061] In an optional implementation, the device further includes:

[0062] The determining module is configured to determine a preset focal plane in a plurality of focal planes corresponding to the plurality of focal plane images as an optimal focal plane of the trophoblast cell;

[0063] The determining module is further configured to determine an optimal focal plane of the inner cell mass from the plurality of focal planes according to an area of the mask of the inner cell mass in the plurality of focal plane images;

[0064] The determining module is further configured to determine a three-dimensional position of the trophoblast cell grid model according to the optimal focal plane of the trophoblast cell, and determine a three-dimensional position of the inner cell mass grid model according to the optimal focal plane of the inner cell mass;

[0065] In an optional implementation, the constructing module is specifically configured to:

[0066] According to the three-dimensional position of the trophoblast cell grid model, perform position matching on the trophoblast cell grid model and the internal three-dimensional point cloud model, and according to the three-dimensional position of the inner cell mass grid model, perform position matching on the inner cell mass grid model and the internal three-dimensional point cloud model to obtain the three-dimensional geometric model.

[0067] In an optional implementation, the determining module is specifically configured to:

[0068] According to an area of the mask of the inner cell mass in the plurality of focal plane images, determine a target focal plane image with a maximum area of the mask of the inner cell mass from the plurality of focal plane images;

[0069] The mask of the inner cell mass in the target focal plane image is taken as a segmentation mask, and the plurality of focal plane images are segmented to obtain a plurality of segmented focal plane images;

[0070] The best focal plane of the inner cell mass is determined from the plurality of focal planes according to the definition of the plurality of segmented focal plane images.

[0071] In a third aspect, an electronic device is provided, which includes a processor, a memory, and a bus. The memory stores machine readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory through the bus. The processor executes the machine readable instructions to perform the method of any one of the first aspect.

[0072] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program. When the computer program is run by a processor, the method of any one of the first aspect is performed.

[0073] The application provides a blastocyst three-dimensional fitting reconstruction method, device, electronic device and storage medium. The method includes: acquiring a plurality of focal plane images of a blastocyst, identifying a mask of trophoblast cells and a mask of an inner cell mass in the blastocyst based on the plurality of focal plane images, constructing a trophoblast cell grid model and an inner cell mass grid model according to the mask of the trophoblast cells and the mask of the inner cell mass respectively, constructing a depth map of the blastocyst according to the plurality of focal plane images, performing image fusion on the plurality of focal plane images to obtain a texture image, performing plane mapping according to the texture image and the depth map to generate an internal three-dimensional point cloud model of the blastocyst, constructing a three-dimensional geometric model of the blastocyst according to the trophoblast cell grid model, the inner cell mass grid model and the internal three-dimensional point cloud model, and mapping the texture image to the surface of the three-dimensional geometric model to generate a three-dimensional model of the blastocyst. The scheme constructs a three-dimensional model of the blastocyst by using focal plane images of the blastocyst, does not need additional embryo operation and imaging hardware structure, does not affect the development of the embryo, and is low in cost and safe, reliable, convenient and effective. BRIEF DESCRIPTION OF DRAWINGS

[0074] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed in the embodiments. It should be understood that the following drawings only show some embodiments of the application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0075] Figure 1 Flowchart of the blastocyst three-dimensional fitting reconstruction method provided by the embodiments of the application Figure One ;

[0076] Figure 2 A schematic diagram of a light sheet model providing multiple focal planes for an embodiment of the present application;

[0077] Figure 3 A schematic diagram of a construction process of a trophoblast grid model and an inner cell mass grid model provided for an embodiment of the present application;

[0078] Figure 4 A schematic diagram of a texture image provided for an embodiment of the present application;

[0079] Figure 5 A schematic diagram of a texture mapping provided for an embodiment of the present application;

[0080] Figure 6 A schematic diagram of a process of a three-dimensional fitting reconstruction method of a blastocyst provided for an embodiment of the present application Figure Two ;

[0081] Figure 7 A schematic diagram of a process of a three-dimensional fitting reconstruction method of a blastocyst provided for an embodiment of the present application Figure Three ;

[0082] Figure 8 A schematic diagram of a process of a three-dimensional fitting reconstruction method of a blastocyst provided for an embodiment of the present application Figure Four ;

[0083] Figure 9 A schematic diagram of a focal plane image provided for an embodiment of the present application;

[0084] Figure 10 A schematic diagram of a process of a three-dimensional fitting reconstruction method of a blastocyst provided for an embodiment of the present application Figure Five ;

[0085] Figure 11 A schematic diagram of an auxiliary circle mark provided for an embodiment of the present application;

[0086] Figure 12 A schematic diagram of a process of a three-dimensional fitting reconstruction method of a blastocyst provided for an embodiment of the present application Figure Six ;

[0087] Figure 13 A schematic diagram of a structure of a three-dimensional fitting reconstruction device of a blastocyst provided for an embodiment of the present application;

[0088] Figure 14 A schematic diagram of a structure of an electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0089] To make the purposes, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person skilled in the art without creative work fall within the scope of protection of the present application.

[0090] First, the professional terms involved in the present application are explained:

[0091] Timelapse: time-lapse.

[0092] HMC: Hoffman Modulation Contrast.

[0093] TE: Trophectoderm.

[0094] ICM: Inner cell mass.

[0095] With the rapid development and maturation of artificial insemination (ART) and artificial intelligence (AI) technologies, AI-based three-dimensional imaging technology in the medical field makes non-invasive three-dimensional fitting reconstruction of embryos possible. At present, most embryologists prefer to use time-lapse (Timelapse) to obtain multi-focal plane data for morphological and dynamic evaluation of embryos. Such a method promotes the research and evaluation of pre-implantation embryo development by embryologists. However, compared with two-dimensional image observation, embryologists expect to observe embryos from a higher dimension and hope to obtain a complete, morphologically rich, and spatially characteristic model on the basis of existing imaging technology for evaluation and research of embryos.

[0096] In the current research, pre-implantation embryo three-dimensional model acquisition is mainly through fluorescent staining and confocal microscope imaging to obtain Z-stack (Z-axis-based stack image set) sequence fluorescent slice data for three-dimensional model reconstruction. However, such a method requires staining of embryos and irradiation of high-energy light spectrum, which can cause irreversible damage to embryos.

[0097] In addition, the current evaluation of blastocyst is mainly based on static two-dimensional images, or based on expert consensus (Gardener), or based on AI model, and the evaluation standard is based on the morphological characteristics of trophoblast cells and inner cell mass and the abstract high-dimensional features extracted by AI to make judgments. The spatial information of the vertical direction of the blastocyst is ignored in the evaluation process, and the influence of the interaction between the trophoblast cells and the inner cell mass on the development of the embryo is ignored in the evaluation standard.

[0098] To solve the above problems, the present application is based on the fitting reconstruction of the three-dimensional model of the blastocyst under the time difference culture system, and a plurality of focal plane images are obtained by a Hoffman modulation contrast microscope imaging system. This technical means is safe and reliable for embryo imaging. Compared with the coagulation microscope or other microscope imaging, the method of obtaining data under the time difference system is more simple and safe. In the popularization process of time difference, the reproductive center also accumulates a large number of embryo multi-focal plane images for three-dimensional model reconstruction, which makes the method of three-dimensional fitting reconstruction under the time difference culture system more valuable. That is, the present application uses the focal plane images of the blastocyst to construct a three-dimensional model of the blastocyst, without the need for additional embryo operation and imaging hardware structure, which will not affect the development of the embryo. It is low in cost, safe and reliable, convenient and effective, and by combining the trophoblast cells and inner cell mass of the blastocyst to reconstruct the geometric spatial structure of the blastocyst, the three-dimensional morphological structure characteristics of the blastocyst can be further analyzed.

[0099] It is worth noting that the fitting reconstruction of the non-invasive living blastocyst three-dimensional model by the plurality of focal plane embryo images obtained under the time difference culture system can calculate the morphology and spatial characteristics of the blastocyst through the three-dimensional model of the blastocyst, and analyze the correlation between the corresponding characteristics and the embryo transfer outcome, which can be used to explore the characteristics that have an impact on pregnancy and live birth during the development of the blastocyst.

[0100] Figure 1 The flowchart of the three-dimensional fitting reconstruction method of the blastocyst provided in the embodiment of the present application Figure One The execution subject of the embodiment can be an electronic device, such as a terminal device.

[0101] As shown in Figure 1 , the method can include:

[0102] S101, obtaining a plurality of focal plane images of a blastocyst.

[0103] The blastocyst is a blastocyst to be three-dimensionally fitted and reconstructed, for example, it can be a 3-stage blastocyst or a 4-stage blastocyst.

[0104] A Hoffman modulation contrast microscope is used to collect a plurality of focal plane images of the blastocyst.

[0105] It is worth mentioning that multiple focal plane images of the embryo can be obtained by the time difference incubator, and due to the particularity of the Hoffman modulation contrast microscope imaging, the focal plane images have a relief effect (the cell edges have strong contrast), wherein the multiple focal planes can be approximately considered as "light slices" for the horizontal cutting of the sphere, multiple focal planes are taken at equal intervals around the Z axis, and the number of focal planes is not limited to 11 focal planes, Figure 2 A schematic diagram of a light slice model providing multiple focal planes for the embodiment of the present application is shown as Figure 2 The multiple focal planes are respectively denoted as F75, F60, F45, F30, F15, F0, F-15, F-30, F-45, F-60, F-75, each focal plane is a plane determined by the horizontal cutting of the sphere, and as an example, the collection interval between each two focal planes in the embodiment of the present application can be 15 μm.

[0106] S102, based on the multiple focal plane images, identifying the mask of the trophoblast cells and the mask of the inner cell mass in the blastocyst.

[0107] The multiple focal plane images are respectively identified to identify the mask of the trophoblast cells and the mask of the inner cell mass in the blastocyst in each focal plane image, wherein the trophoblast cells are used for developing into the placenta, and the inner cell mass is used for developing into the embryonic individual.

[0108] In some embodiments, a pre-trained semantic segmentation model of the trophoblast cells is used to perform semantic segmentation on each focal plane image to obtain the mask of the trophoblast cells in the blastocyst in each focal plane image, and a pre-trained semantic segmentation model of the inner cell mass is used to perform semantic segmentation on each focal plane image to obtain the mask of the inner cell mass in the blastocyst in each focal plane image, wherein the semantic segmentation model of the trophoblast cells and the semantic segmentation model of the inner cell mass are respectively obtained by training and learning on a plurality of focal plane image samples manually annotated, and the semantic segmentation method has the following schemes: FCN, UNet, Unet++, DeepLabV3, and the preferred method is U2Net.

[0109] S103, constructing a trophoblast cell grid model and an inner cell mass grid model according to the mask of the trophoblast cells and the mask of the inner cell mass, respectively.

[0110] According to the multiple masks of the trophoblast cells corresponding to the multiple focal plane images, the geometric structure of the trophoblast cells is fitted to construct a trophoblast cell grid model, and according to the multiple masks of the inner cell mass corresponding to the multiple focal plane images, the geometric structure of the inner cell mass is fitted to construct an inner cell mass grid model, that is, the three-dimensional model of the trophoblast cells and the three-dimensional model of the inner cell mass are obtained by the "light slice stacking" method.

[0111] In some embodiments, the contours of the multiple trophoblast cell masks are extracted respectively, and the coordinate points of the contours corresponding to the multiple trophoblast cell masks are geometrically fitted in azimuth angle order to obtain a point cloud fitting model of the trophoblast cells, and the contours of the multiple inner cell mass cell masks are extracted respectively, and the coordinate points of the contours corresponding to the multiple inner cell mass masks are geometrically fitted in azimuth angle order to obtain a point cloud fitting model of the inner cell mass, and then the point cloud fitting model of the trophoblast cells and the point cloud fitting model of the inner cell mass are converted into a grid model according to a normal calculation and surface reconstruction algorithm, wherein the azimuth angle order can be understood as the order from 0° to 360°.

[0112] Figure 3 The construction process of the trophoblast grid model and the inner cell mass grid model provided by the embodiments of the present application is shown in Figure 3 As shown, for each focal plane image, the mask of the trophoblast cells and the mask of the inner cell mass in the blastula are identified, and the contours of the trophoblast cell mask and the inner cell mass mask are extracted respectively, and the coordinate points of the contours of the multiple trophoblast cell masks and the coordinate points of the contours of the multiple inner cell mass masks are geometrically fitted to generate the trophoblast grid model and the inner cell mass grid model.

[0113] S104, constructing a depth map of the blastula according to the multiple focal plane images.

[0114] The depth estimation method is used to estimate the depth information of the blastula according to the multiple focal plane images to construct a depth map of the blastula, wherein the depth map of the blastula is used to indicate the depth information of the blastula, and the depth estimation method is a geometric-based method to estimate a three-dimensional structure from a series of two-dimensional image sequences.

[0115] The depth estimation method can be a traditional focus ranging algorithm (Deep Flow Field Fusion, DFF), which obtains image data with different degrees of defocus by adjusting different Z-axis positions in the same field of view, and estimates the depth distance of the image by the clarity of the different layer images, or the depth estimation method can be an end-to-end depth prediction method, which estimates the depth by a single image. The depth estimation method has the following schemes: Defocus-Net, DepthCNN, DFV, and the preferred one is DepthCNN.

[0116] In some embodiments, the plurality of focal plane images are divided into two groups of image sequences, the plurality of focal planes are marked as Fn~F-n, the focal planes of the two groups of image sequences are Fn~F0 and F0~F-n respectively, it is worth noting that the number of focal plane images in the image sequence is greater than or equal to 10, a depth estimation method is used to construct an upper layer depth map of the blastula according to one group of image sequences, and a depth estimation method is used to construct a lower layer depth map of the blastula according to the other group of image sequences.

[0117] S105, image fusion is performed on the plurality of focal plane images to obtain a texture image.

[0118] An image fusion network is used to perform image fusion on the plurality of focal plane images to obtain a texture image of the blastula, wherein the image fusion method has the following schemes: Densefuse-net, FusionNet, and U2FusionNet, and the preferred one is U2FusionNet.

[0119] In some embodiments, image fusion is performed on one group of image sequences to obtain an upper layer texture image (Up_TextMap) of the blastula, and image fusion is performed on the other group of image sequences to obtain a lower layer texture image (Down_Textmap) of the blastula.

[0120] Figure 4 A schematic diagram of the texture image provided by the embodiments of the present application is shown in Figure 4 , image fusion is performed on the plurality of focal plane images to obtain a texture image.

[0121] S106, plane mapping is performed according to the texture image and the depth map to generate an internal three-dimensional point cloud model of the blastula.

[0122] A texture mapping algorithm is used to perform plane mapping of the texture image to the depth map of the blastula to generate an internal three-dimensional point cloud model of the blastula, wherein the texture image is obtained by fusing the plurality of focal plane images, the texture image is used to reflect the surface texture of the internal of the blastula, the texture image is a plane image, and the depth map is used to reflect the depth information of the blastula, so that the internal three-dimensional point cloud model of the blastula can be generated by performing plane mapping of the texture image to the depth map of the blastula.

[0123] In some embodiments, plane mapping is performed on the upper layer depth map and the upper layer texture image to generate an upper layer internal three-dimensional model of the blastula, and plane mapping is performed on the lower layer depth map and the lower layer texture image to generate a lower layer internal three-dimensional point cloud model of the blastula.

[0124] S107, a three-dimensional geometric model of the blastula is constructed according to the trophoblast cell grid model, the inner cell mass grid model, and the internal three-dimensional point cloud model.

[0125] Point cloud position matching is performed on the trophoblast cell mesh model, the inner cell mass mesh model, and the internal 3D point cloud model to set the trophoblast cell mesh model and the inner cell mass mesh model within the internal 3D point cloud model, thereby generating the 3D geometric model of the blastocyst.

[0126] S108. Map the texture image onto the surface of the three-dimensional geometric model to generate a three-dimensional model of the blastocyst.

[0127] Since HMC is a bright-field imaging method, the surface of the blastocyst is mainly composed of the zona pellucida and TE cells, both of which are cell membrane structures and are transparent. Therefore, the information presented on the surface of the blastocyst includes the texture information inside the blastocyst.

[0128] Therefore, what is seen from the surface of the blastocyst is the true texture of the blastocyst. Thus, the texture image of the blastocyst obtained by image fusion can be used as the outer surface texture image of the blastocyst. The texture mapping method is used to map the texture image onto the surface of the three-dimensional geometric model of the blastocyst to generate the three-dimensional model of the blastocyst.

[0129] In some embodiments, an upper texture image is mapped onto the upper surface of the 3D geometric model of the blastocyst, and a lower texture image is mapped onto the lower surface of the 3D geometric model of the blastocyst to generate a 3D model of the blastocyst.

[0130] Taking the hemispherical texture mapping method as an example, Figure 5 A schematic diagram of texture mapping provided in an embodiment of this application, as shown below. Figure 5 As shown, the 3D geometric model of the blastocyst is a sphere. The plane containing the texture image is used as the projection plane, which is parallel to the equatorial plane of the hemisphere and tangent to the hemisphere at the positive pole A. The projection center is placed at the negative pole B of the sphere. The correspondence between the coordinates of the texture image plane and the spherical coordinates is determined by projection. The arcs on the texture image plane are mapped to parallels of latitude on the hemisphere, and the radial lines on the texture image plane are mapped to meridians of longitude on the hemisphere. The mapping formula is as follows:

[0131]

[0132] in, θ is the pitch angle, yaw angle is the yaw angle, and uv is the texture coordinate of a pixel in the texture image.

[0133] In this embodiment, a three-dimensional model of the blastocyst is reconstructed using multiple focal plane images, which solves the difficulty of three-dimensional live cell imaging of the blastocyst before transplantation. This provides embryologists with a new perspective for exploring and studying the developmental process of the embryo. Embryologists can more safely and three-dimensionally observe and evaluate the development of the blastocyst, and it also creates conditions for evaluating the three-dimensional morphological characteristics of the embryo. Embryologists can calculate new three-dimensional features through the three-dimensional model of the blastocyst to obtain more details and processes of embryo development, and use these features for embryo selection. This is of great significance for retrospective analysis of embryos.

[0134] Figure 6 A flowchart illustrating the three-dimensional fitting and reconstruction method for blastocysts provided in this application embodiment. Figure Two ,like Figure 6 As shown, in an optional implementation, step S104, constructing a depth map of the blastocyst based on multiple focal plane images, may include:

[0135] S201. Obtain the sharpness of each pixel in multiple focal plane images.

[0136] Sharpness is extracted from each pixel in the focal plane image to obtain the sharpness of each pixel in the focal plane image. The number of pixels in multiple focal plane images is the same. The multiple focal plane images are denoted as f0 to fn, and the sharpness at the pixel (x,y) is y0-yn.

[0137] S202. From the multiple focal planes corresponding to multiple focal plane images, determine the target focal plane with the highest sharpness for each pixel, and obtain the depth information of the target focal plane.

[0138] For each pixel, from multiple focal planes corresponding to multiple focal plane images, the target focal plane with the highest sharpness for each pixel is determined. Each focal plane corresponds to a depth information value; the depth information of the target focal plane, i.e., the maximum sharpness y, is then obtained. max The depth information of a pixel is the depth information of the focal plane it occupies. The depth information of the focal plane can be understood as... Figure 2 In the three-dimensional spatial coordinate system established with the center of the F0 focal plane as the origin, the Z value of the focal plane on the Z axis.

[0139] S203. Construct a depth map based on the depth information of the target focal plane.

[0140] Based on the depth information of the target focal plane of all pixels, interpolation fitting is performed to construct the depth map of the blastocyst. In other words, by obtaining the sharp focus position corresponding to each pixel, a depth map with very sharp focus position is constructed. The depth information of the blastocyst is then recovered through focus analysis, and the depth information is interpolated to recover a more accurate depth information of the blastocyst.

[0141] In some embodiments, the depth map can also be mean filtered to smooth the depth map to avoid too large difference of pixels in the depth map. The method of mean filtering is mainly neighborhood averaging method. For a pixel (x, y) to be processed, a template composed of several neighboring pixels of the pixel (x, y) is selected. The average value of the depth values of all pixels in the template is calculated. The average value is assigned to the current pixel (x, y) as the gray value g(x, y) of the processed image at the pixel point:

[0142] g(x, y) = 1 / m∑f(x, y)

[0143] wherein m represents the number of pixels in the template including the current pixel point. The filtered depth map is selected for gray value. The selected gray value is the position of clear focus in the multiple focal plane images. Thus, a depth map with clear depth part at each position is generated. The range of the gray value is 0-255.

[0144] Figure 7 Flowchart of the three-dimensional fitting reconstruction method of the blastula provided in the embodiments of the present application Figure Three As shown in FIG. 1, in an optional embodiment, step S101, a plurality of focal plane images of a blastula are acquired, including: Figure 7

[0145] S301, a plurality of first focal plane images of the blastula are acquired.

[0146] S302, the plurality of first focal plane images are processed by frame interpolation to obtain a plurality of second focal plane images of the blastula.

[0147] The plurality of first focal plane images of the blastula are collected by using a Hough modulation contrast microscope. The number of the plurality of first focal plane images can be 11, i.e. 11 focal images. Then the plurality of first focal plane images are processed by frame interpolation to obtain a plurality of second focal plane images of the blastula. The plurality of focal plane images include the plurality of first focal plane images and the plurality of second focal plane images. The number of the plurality of focal plane images can be more than 20 focal images.

[0148] ​It is worth noting that in the time-lapse culture system, the number of focal plane images that can be collected is limited, and fewer focal planes mean fewer optical sections, and the number of optical sections determines the error between the reconstructed trophoblast grid model and inner cell mass grid model and the real blastula. If limited to the first focal plane image, more spatial information will be lost. In order to solve the problem of insufficient focal planes, an adversarial generative network can be used to perform inter-layer interpolation on multiple first focal plane images to generate a new focal plane between two adjacent first focal planes, and a second focal plane image between two adjacent first focal plane images.

[0149] In this embodiment, each layer of the interpolated image needs to have the spatial and texture information of the adjacent two focal plane images, so an adversarial generative network can be used to perform inter-layer interpolation on multiple first focal plane images to generate multiple second focal plane images, thereby ensuring that more focal plane images are generated by image interpolation without increasing the number of photographs and exposure time of the blastula.

[0150] Figure 8 Flowchart of the three-dimensional fitting reconstruction method of the blastula provided in the embodiments of the present application Figure Four As shown in Figure 8 In an optional embodiment, before constructing the trophoblast grid model and the inner cell mass grid model according to the trophoblast mask and the inner cell mass mask, the method can further include:

[0151] S401, segmenting multiple focal plane images to obtain blastulas in the multiple focal plane images.

[0152] Segmenting each focal plane image to obtain the blastula in each focal plane image, that is, preprocessing the focal plane image to determine the region of interest (ROI) of the blastula therefrom.

[0153] S402, if there is a target focal plane image in the multiple focal plane images, moving the blastula in the target focal plane image to the center of the preset field of view and performing boundary padding to regenerate the target focal plane image.

[0154] Determine whether there is a target focal plane image in the multiple focal plane images, the blastula in the target focal plane image is not in the center of the preset field of view, if there is a target focal plane image, move the blastula in the target focal plane image to the center of the preset field of view and perform boundary padding to regenerate the target focal plane image, wherein the center of the preset field of view can be understood as the center of the focal plane image, and the boundary padding can be understood as color padding of the area outside the blastula with a preset color such as black, wherein the target focal plane image can be, for example, the F0 focal plane image.

[0155] In some embodiments, the other focal plane images other than the target focal plane image in the plurality of focal plane images can also be boundary padded to regenerate the other focal plane images.

[0156] It is worth noting that the resolution of the regenerated target focal plane image is consistent with the resolution of the original target focal plane image, and the purpose of boundary padding is to set the spatial coordinate system reference point of the reconstructed three-dimensional model of the blastula as the center point of the blastula, thereby improving the reconstruction accuracy of the three-dimensional model.

[0157] Figure 9 The schematic diagram of the focal plane image provided for the embodiments of the present application is shown in Figure 9 The blastula is moved to the center of the preset field of view and the area outside the blastula is filled with black.

[0158] Figure 10 The flowchart of the three-dimensional fitting reconstruction method of the blastula provided for the embodiments of the present application is shown in Figure Five As shown in Figure 10 In an optional embodiment, step S102, based on the plurality of focal plane images, identifying the mask of the trophoblast cells and the mask of the inner cell mass in the blastula can include:

[0159] S501, according to the preset mark pattern corresponding to the plurality of focal plane images, marking the trophoblast cell region on the plurality of focal plane images respectively.

[0160] Wherein, the preset mark pattern can be an auxiliary circle, each focal plane corresponds to an auxiliary circle, different auxiliary circles are different in size, according to the preset mark pattern corresponding to the plurality of focal plane images, the trophoblast cell region is marked on the plurality of focal plane images respectively, the trophoblast cell region is the region marked by the preset mark pattern, such as the region inside the auxiliary circle.

[0161] S502, identifying the trophoblast cell region to obtain the mask of the trophoblast cells.

[0162] S503, identifying the inner cell mass region inside the trophoblast cell region to obtain the mask of the inner cell mass.

[0163] Identifying the marked trophoblast cell region to identify the mask of the trophoblast cells in the trophoblast cell region, and identifying the inner cell mass region inside the trophoblast cell region to obtain the mask of the inner cell mass.

[0164] In this embodiment, due to the particularity of HMC bright field imaging, in order to eliminate the interference of the adjacent layer artifact image, the focal plane image is marked by graph, combined with the structural characteristics of the blastomere ball, and a plurality of focal plane images are approximately considered as the horizontal cutting of the optical section to the spherical surface. The identification accuracy of the trophoblast cell mask and the inner cell mass mask is improved. Among them, the focal plane artifact overlap phenomenon in the range of F15-F75 is particularly obvious.

[0165] Referring to Figure 2 , the boundary line of each layer of optical section (i.e. the auxiliary circle in the figure) is used as the auxiliary marking range of the trophoblast cells, that is, the trophoblast cells are in the range of the auxiliary circle, Figure 11 , the auxiliary circle in the auxiliary circle is the trophoblast cell area, that is, the auxiliary circle outside is the interference of the artifact image brought by the adjacent focal plane. Figure 11

[0166] Figure 12 The flowchart of the three-dimensional fitting reconstruction method of the blastomere provided in the embodiment of the application is shown in Figure Six , in an optional implementation, the method can further include: Figure 12 S601, determine a preset focal plane in a plurality of focal planes corresponding to a plurality of focal plane images as the best focal plane of the trophoblast cells.

[0167] Among them, one focal plane image corresponds to one focal plane, and the preset focal plane can be F0 focal plane. The F0 focal plane is determined as the best focal plane of the trophoblast cells, that is, the mask of the trophoblast cells is the clearest in the preset focal plane.

[0168] S602, according to the area of the inner cell mass mask in the plurality of focal plane images, determine the best focal plane of the inner cell mass from the plurality of focal planes.

[0169] The area of the inner cell mass mask in the plurality of focal plane images is obtained, and according to the area of the inner cell mass mask, the best focal plane of the inner cell mass mask is determined from the plurality of focal planes.

[0170] In an optional implementation, step S602, according to the area of the inner cell mass mask in the plurality of focal plane images, determine the best focal plane of the inner cell mass from the plurality of focal planes, including:

[0171] According to the area of the inner cell mass mask in the plurality of focal plane images, determine the target focal plane image with the largest area of the inner cell mass mask from the plurality of focal plane images.

[0172]

[0173] ​​The mask of the inner cell mass in the target focal plane image is taken as a segmentation mask, and the plurality of focal plane images are segmented to obtain a plurality of segmented focal plane images.

[0174] The best focal plane of the inner cell mass is determined from the plurality of focal planes according to the definition of the plurality of segmented focal plane images.

[0175] The target focal plane image in which the area of the inner cell mass is the largest is determined from the plurality of focal planes according to the area of the mask of the inner cell mass in the plurality of focal plane images, and then the area of the inner cell mass in the target focal plane image is taken as a segmentation mask to segment the plurality of focal plane images to obtain a segmented focal plane image corresponding to each focal plane image, and then the best focal plane of the inner cell mass is determined from the plurality of focal planes according to the definition of the plurality of segmented focal plane images, wherein the best focal plane is a focal plane corresponding to a focal plane image with the largest definition in the plurality of segmented focal plane images.

[0176] In this embodiment, the mask of the inner cell mass in the target focal plane image is taken as a segmentation mask to obtain a plurality of segmented focal plane images, so that the best focal plane of the inner cell mass can be determined according to the definition of the plurality of segmented focal plane images, and the best focal plane can be determined based on image definition in the case of the same area, and the accuracy is high.

[0177] It should be noted that the best focal plane of the trophoblast cell can be understood as the equatorial plane of the mask of the trophoblast cell, and the best focal plane of the inner cell mass can be understood as the equatorial plane of the mask of the inner cell mass.

[0178] S603, according to the best focal plane of the trophoblast cell, the three-dimensional position of the trophoblast cell grid model is determined, and according to the best focal plane of the inner cell mass, the three-dimensional position of the inner cell mass grid model is determined.

[0179] A three-dimensional space coordinate system is established with the center of the best focal plane of the trophoblast cell as the origin, the plane position of the trophoblast cell grid model is obtained, that is, the X value and the Y value, and the Z axis position of the best focal plane of the trophoblast cell is taken as the Z value of the center point of the trophoblast cell grid model, so as to obtain the three-dimensional position of the trophoblast cell grid model. The three-dimensional space coordinate system established is similar to Figure 2 , except that the coordinate origin is different.

[0180] A three-dimensional space coordinate system is established with the center of the best focal plane of the trophoblast cell as the origin, the plane position of the trophoblast cell grid model is obtained, that is, the X value and the Y value, and the Z axis position of the best focal plane of the trophoblast cell is taken as the Z value of the center point of the trophoblast cell grid model, so as to obtain the three-dimensional position of the trophoblast cell grid model. The three-dimensional space coordinate system established is similar to Figure 2 , except that the coordinate origin is different.

[0181] Step S107, based on the trophoblast cell mesh model, the inner cell mass mesh model, and the internal 3D point cloud model, construct a 3D geometric model of the blastocyst, which may include:

[0182] S604. Based on the three-dimensional position of the trophoblast cell mesh model, perform position matching between the trophoblast cell mesh model and the internal three-dimensional point cloud model, and based on the three-dimensional position of the inner cell mass mesh model, perform position matching between the inner cell mass mesh model and the internal three-dimensional point cloud model to obtain a three-dimensional geometric model.

[0183] Based on the 3D position of the trophoblast cell mesh model, position matching is performed between the trophoblast cell mesh model and the internal 3D point cloud model to set the trophoblast cell mesh model within the internal 3D point cloud model. Similarly, based on the 3D position of the inner cell mass mesh model, position matching is performed between the inner cell mass mesh model and the internal 3D point cloud model to set the inner cell mass mesh model within the internal 3D point cloud model, thereby obtaining a 3D geometric model.

[0184] In this embodiment, when reconstructing the three-dimensional model of the blastocyst, the x and y values ​​are determined by the optimal focal plane, and the z value is obtained by the z-axis of the optimal focal plane. This allows for the accurate spatial positions of the trophoblast cell mesh model and the inner cell mass mesh model, which are then matched to obtain the three-dimensional geometric model of the blastocyst.

[0185] Based on the same inventive concept, this application also provides a blastocyst three-dimensional fitting and reconstruction device corresponding to the blastocyst three-dimensional fitting and reconstruction method. Since the principle of the device in this application is similar to the blastocyst three-dimensional fitting and reconstruction method described above in this application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0186] Figure 13 This is a schematic diagram of the structure of a three-dimensional fitting and reconstruction device for blastocysts provided in an embodiment of this application. This device can be integrated into an electronic device.

[0187] like Figure 13 As shown, the device may include:

[0188] The acquisition module 701 is used to acquire multiple focal plane images of the blastocyst;

[0189] The recognition module 702 is used to identify the mask of trophoblast cells and the mask of inner cell mass in the blastocyst based on multiple focal plane images;

[0190] Module 703 is used to construct trophoblast cell mesh models and inner cell mass mesh models based on the masks of trophoblast cells and inner cell mass, respectively.

[0191] The constructing module 703 is further configured to construct a depth map of the blastula according to the plurality of focal plane images.

[0192] The fusing module 704 is configured to fuse the plurality of focal plane images to obtain a texture image.

[0193] The mapping module 705 is configured to perform plane mapping according to the texture image and the depth map to generate an internal three-dimensional point cloud model of the blastula.

[0194] The constructing module 703 is further configured to construct a three-dimensional geometric model of the blastula according to the trophoblast cell grid model, the inner cell mass grid model and the internal three-dimensional point cloud model.

[0195] The mapping module 705 is further configured to map the texture image to a surface of the three-dimensional geometric model to generate a three-dimensional model of the blastula.

[0196] In an optional implementation, the constructing module 703 is specifically configured to:

[0197] acquire the sharpness of each pixel point in the plurality of focal plane images;

[0198] determine a target focal plane with the maximum sharpness of each pixel point from a plurality of focal planes corresponding to the plurality of focal plane images, and obtain depth information of the target focal plane;

[0199] construct the depth map according to the depth information of the target focal plane.

[0200] In an optional implementation, the acquiring module 701 is specifically configured to:

[0201] acquire a plurality of first focal plane images of the blastula;

[0202] perform frame interpolation processing on the plurality of first focal plane images to obtain a plurality of second focal plane images of the blastula, and the plurality of focal plane images include the plurality of first focal plane images and the plurality of second focal plane images.

[0203] In an optional implementation, the apparatus further includes:

[0204] The segmenting module 706 is configured to segment the plurality of focal plane images to obtain the blastula in the plurality of focal plane images.

[0205] The moving module 707 is configured to, if there is a target focal plane image in the plurality of focal plane images, move the blastula in the target focal plane image to a central part of a preset field of view, perform boundary padding to regenerate the target focal plane image, and the blastula in the target focal plane image is not at the central part of the preset field of view.

[0206] In an optional implementation, the identifying module 702 is specifically configured to:

[0207] According to the preset mark pattern corresponding to the plurality of focal plane images, mark the trophoblast cell region on the plurality of focal plane images respectively;

[0208] Identify the trophoblast cell region to obtain a mask of the trophoblast cell;

[0209] Identify the inner cell mass region located in the trophoblast cell region to obtain a mask of the inner cell mass.

[0210] In an optional implementation, the device further includes:

[0211] The determining module 708 is configured to determine a preset focal plane in the plurality of focal planes corresponding to the plurality of focal plane images as a best focal plane of the trophoblast cell;

[0212] The determining module 708 is further configured to determine a best focal plane of the inner cell mass from the plurality of focal planes according to the area of the mask of the inner cell mass in the plurality of focal plane images;

[0213] The determining module 708 is further configured to determine a three-dimensional position of the trophoblast cell grid model according to the best focal plane of the trophoblast cell, and determine a three-dimensional position of the inner cell mass grid model according to the best focal plane of the inner cell mass;

[0214] In an optional implementation, the constructing module 703 is specifically configured to:

[0215] According to the three-dimensional position of the trophoblast cell grid model, perform position matching on the trophoblast cell grid model and the internal three-dimensional point cloud model, and according to the three-dimensional position of the inner cell mass grid model, perform position matching on the inner cell mass grid model and the internal three-dimensional point cloud model to obtain the three-dimensional geometric model.

[0216] In an optional implementation, the determining module 708 is specifically configured to:

[0217] According to the area of the mask of the inner cell mass in the plurality of focal plane images, determine a target focal plane with the largest area of the mask of the inner cell mass from the plurality of focal planes;

[0218] Take the mask of the inner cell mass in the target focal plane image as a segmentation mask to segment the plurality of focal plane images to obtain a plurality of segmented focal plane images;

[0219] According to the clarity of the plurality of segmented focal plane images, determine a best focal plane of the inner cell mass from the plurality of focal planes.

[0220] The description of the processing procedure of each module in the device and the interaction procedure between the modules can refer to the related description in the above method embodiments, which will not be described in detail here.

[0221] Figure 14This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 14 As shown, the device may include a processor 801, a memory 802, and a bus 803. The memory 802 stores machine-readable instructions that can be executed by the processor 801. When the electronic device is running, the processor 801 communicates with the memory 802 through the bus 803, and the processor 801 executes the machine-readable instructions to perform the above-described method.

[0222] This application also provides a computer-readable storage medium storing a computer program that is executed by a processor, wherein the processor performs the above-described method.

[0223] In this embodiment, the computer program, when run by the processor, can also execute other machine-readable instructions to perform other methods as described in the embodiments. For details on the specific execution steps and principles, please refer to the description of the embodiments, which will not be repeated here.

[0224] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0225] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0226] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0227] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0228] It should be noted that similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In addition, the terms "first", "second", "third" and the like are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0229] Finally, it should be noted that the above-described embodiments are only specific implementations of the present application, which are used to illustrate the technical solutions of the present application, but not to limit them. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can make modifications or easily think of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present application, or make equivalent replacements to some technical features. These modifications, changes or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application. They should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of three-dimensional fitting reconstruction of blastocysts, characterized by, The method comprises the following steps: obtaining a plurality of focal plane images of a blastocyst; based on the plurality of focal plane images, identifying a mask of trophoblast cells and a mask of inner cell mass in the blastocyst; constructing a trophoblast cell grid model and an inner cell mass grid model according to the mask of trophoblast cells and the mask of inner cell mass, respectively; constructing a depth map of the blastocyst according to the plurality of focal plane images; performing image fusion on the plurality of focal plane images to obtain a texture image; performing plane mapping according to the texture image and the depth map to generate an internal three-dimensional point cloud model of the blastocyst; constructing a three-dimensional geometric model of the blastocyst according to the trophoblast cell grid model, the inner cell mass grid model and the internal three-dimensional point cloud model; mapping the texture image onto the surface of the three-dimensional geometric model to generate a three-dimensional model of the blastocyst.

2. The method of claim 1, wherein, The method further comprises the following steps: obtaining the clarity of each pixel point in the plurality of focal plane images; determining the target focal plane with the maximum clarity of each pixel point from the plurality of focal planes corresponding to the plurality of focal plane images to obtain the depth information of the target focal plane; constructing the depth map according to the depth information of the target focal plane.

3. The method of claim 1, wherein, The method further comprises the following steps: obtaining a plurality of first focal plane images of the blastocyst; performing frame interpolation processing on the plurality of first focal plane images to obtain a plurality of second focal plane images of the blastocyst, wherein the plurality of focal plane images comprise the plurality of first focal plane images and the plurality of second focal plane images.

4. The method of claim 1, wherein, The method further comprises the following steps before constructing the trophoblast cell grid model and the inner cell mass grid model according to the mask of trophoblast cells and the mask of inner cell mass: segmenting the plurality of focal plane images to obtain the blastocyst in the plurality of focal plane images; if there is a target focal plane image in the plurality of focal plane images, moving the blastocyst in the target focal plane image to the center of the preset field of view and performing boundary padding to regenerate the target focal plane image, wherein the blastocyst in the target focal plane image is not in the center of the preset field of view.

5. The method according to any one of claims 1 to 4, characterized in that, The method further comprises the following steps: marking the trophoblast cell region on the plurality of focal plane images according to the preset marking pattern corresponding to the plurality of focal plane images; identifying the trophoblast cell region to obtain the mask of trophoblast cells; identifying the inner cell mass region inside the trophoblast cell region to obtain the mask of inner cell mass.

6. The method of claim 1, wherein, The method further comprises the following steps: determining a preset focal plane in the plurality of focal planes corresponding to the plurality of focal plane images as the best focal plane of the trophoblast cells; determining the best focal plane of the inner cell mass from the plurality of focal planes according to the area of the mask of inner cell mass in the plurality of focal plane images. determine a three-dimensional position of the trophoblast cell grid model according to the optimal focal plane of the trophoblast cells, and determine a three-dimensional position of the inner cell mass grid model according to the optimal focal plane of the inner cell mass; the constructing the three-dimensional geometric model of the blastula according to the trophoblast cell grid model, the inner cell mass grid model, and the internal three-dimensional point cloud model comprises: performing position matching on the trophoblast cell grid model and the internal three-dimensional point cloud model according to the three-dimensional position of the trophoblast cell grid model, and performing position matching on the inner cell mass grid model and the internal three-dimensional point cloud model according to the three-dimensional position of the inner cell mass grid model, to obtain the three-dimensional geometric model.

7. The method of claim 6, wherein, the determining the optimal focal plane of the inner cell mass from the multiple focal planes according to the area of the mask of the inner cell mass in the multiple focal plane images comprises: determining a target focal plane image in which the area of the mask of the inner cell mass is the largest from the multiple focal plane images according to the area of the mask of the inner cell mass in the multiple focal plane images; segmenting the multiple focal plane images by taking the mask of the inner cell mass in the target focal plane image as a segmentation mask to obtain multiple segmented focal plane images; determining the optimal focal plane of the inner cell mass from the multiple focal planes according to the definition of the multiple segmented focal plane images.

8. An apparatus for three-dimensional fitting reconstruction of blastulae, characterized by comprise: an acquisition module configured to acquire multiple focal plane images of a blastula; an identification module configured to identify a mask of trophoblast cells and a mask of an inner cell mass in the blastula based on the multiple focal plane images; a construction module configured to construct a trophoblast cell grid model and an inner cell mass grid model according to the mask of the trophoblast cells and the mask of the inner cell mass, respectively; the construction module is further configured to construct a depth map of the blastula according to the multiple focal plane images; a fusion module configured to perform image fusion on the multiple focal plane images to obtain a texture image; a mapping module configured to perform plane mapping according to the texture image and the depth map to generate an internal three-dimensional point cloud model of the blastula; the construction module is further configured to construct a three-dimensional geometric model of the blastula according to the trophoblast cell grid model, the inner cell mass grid model, and the internal three-dimensional point cloud model; the mapping module is further configured to map the texture image to a surface of the three-dimensional geometric model to generate a three-dimensional model of the blastula.

9. An electronic device, comprising: comprise: a processor, a memory, and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, the processor executes the machine readable instructions to execute the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, the computer readable storage medium stores a computer program, the computer program is executed by the processor to execute the method of any one of claims 1 to 7.

Images