Blastocyst three-dimensional fitting reconstruction method and device, electronic equipment and storage medium

Abstract

The invention provides a blastocyst three-dimensional fitting reconstruction method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a plurality of focal plane images of a blastocyst, recognizing a mask of a trophoblast cell and a mask of an inner cell cluster in the blastocyst based on the plurality of focal plane images, and according to the mask of the trophoblast cell and the mask of the inner cell cluster, carrying out the three-dimensional fitting reconstruction of the blastocyst. The method comprises the following steps: respectively constructing a trophoblast cell grid model and an inner cell mass grid model, constructing a depth map of a blastocyst according to a plurality of focal plane images, carrying out image fusion on the plurality of focal plane images to obtain a texture image, carrying out plane mapping according to the texture image and the depth map, and generating an inner three-dimensional point cloud model of the blastocyst, and constructing a three-dimensional geometric model of the blastocyst according to the trophoblast cell grid model, the inner cell mass grid model and the inner three-dimensional point cloud model, and mapping the texture image to the surface of the three-dimensional geometric model to generate a blastocyst three-dimensional model. Reconstruction based on the method does not affect embryonic development, and the method is low in cost, safe and reliable.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a three-dimensional fitting reconstruction method, device, electronic device and storage medium for a blastocyst. Background Art

[0002] With the development of embryology, embryologists are looking forward to observing embryos from a higher dimension and hope to obtain a complete model with rich morphological information and spatial characteristics to evaluate and study embryos.

[0003] At present, fluorescent staining and confocal microscopy are mainly used to obtain serial fluorescent slice data for reconstructing the three-dimensional model of the embryo. However, this method requires the embryo to be stained and irradiated with a high-energy spectrum, which will cause irreversible damage to the embryo. Summary of the invention

[0004] In view of this, an embodiment of the present application provides a three-dimensional fitting reconstruction method, device, electronic device and storage medium for a blastocyst to solve the problem of irreversible damage to the embryo when reconstructing the three-dimensional model of the embryo.

[0005] In a first aspect, an embodiment of the present application provides a three-dimensional fitting reconstruction method for a blastocyst, comprising:

[0006] Acquire multiple focal plane images of the blastocyst;

[0007] Based on the multiple focal plane images, identifying a mask of trophoblast cells and a mask of inner cell mass in the blastocyst;

[0008] According to the mask of the trophoblast cells and the mask of the inner cell mass, constructing a trophoblast cell grid model and an inner cell mass grid model respectively;

[0009] constructing a depth map of the blastocyst according to the multiple focal plane images;

[0010] Performing image fusion on the multiple focal plane images to obtain a texture image;

[0011] Performing plane mapping according to the texture image and the depth map to generate an internal three-dimensional point cloud model of the blastocyst;

[0012] Constructing a three-dimensional geometric model of the blastocyst according to the trophoblast cell grid model, the inner cell mass grid model and the internal three-dimensional point cloud model;

[0013] The texture image is mapped onto the surface of the three-dimensional geometric model to generate a three-dimensional model of the blastocyst.

[0014] In an optional embodiment, constructing the depth map of the blastocyst according to the multiple focal plane images includes:

[0015] Obtaining the clarity of each pixel in the multiple focal plane images;

[0016] Determine, from the multiple focal planes corresponding to the multiple focal plane images, a target focal plane having the maximum clarity of each pixel point, and obtain depth information of the target focal plane;

[0017] The depth map is constructed according to the depth information of the target focal plane.

[0018] In an optional embodiment, the acquiring of multiple focal plane images of the blastocyst comprises:

[0019] Acquiring a plurality of first focal plane images of the blastocyst;

[0020] The multiple first focal plane images are interpolated to obtain multiple second focal plane images of the blastocyst, wherein the multiple focal plane images include: the multiple first focal plane images and the multiple second focal plane images.

[0021] In an optional embodiment, before constructing the trophoblast cell grid model and the inner cell mass grid model respectively according to the trophoblast cell mask and the inner cell mass mask, the method further comprises:

[0022] Segmenting the multiple focal plane images to obtain the blastocyst in the multiple focal plane images;

[0023] If there is a target focal plane image among the multiple focal plane images, the blastocyst in the target focal plane image is moved to the center of a preset field of view, and the boundary is filled to regenerate the target focal plane image, and the blastocyst in the target focal plane image is not in the center of the preset field of view.

[0024] In an optional embodiment, the identifying the mask of the trophoblast cells and the mask of the inner cell mass in the blastocyst based on the multiple focal plane images comprises:

[0025] Marking trophoblast cell regions on the multiple focal plane images respectively according to preset marking patterns corresponding to the multiple focal plane images;

[0026] Identifying the trophoblast cell region to obtain a mask of the trophoblast cell;

[0027] An inner cell mass region located inside the trophoblast cell region is identified to obtain a mask of the inner cell mass.

[0028] In an optional embodiment, the method further comprises:

[0029] Determining a preset focal plane among a plurality of focal planes corresponding to the plurality of focal plane images as an optimal focal plane for the trophoblast cells;

[0030] determining an optimal focal plane of the inner cell mass from the plurality of focal planes according to the area of ​​the mask of the inner cell mass in the plurality of focal plane images;

[0031] Determining the three-dimensional position of the trophoblast cell grid model according to the best focal plane of the trophoblast cell, and determining the three-dimensional position of the inner cell mass grid model according to the best focal plane of the inner cell mass;

[0032] The constructing of the three-dimensional geometric model of the blastocyst according to the trophoblast cell grid model, the inner cell mass grid model and the internal three-dimensional point cloud model comprises:

[0033] According to the three-dimensional position of the trophoblast cell mesh model, the trophoblast cell mesh model and the internal three-dimensional point cloud model are positionally matched, and according to the three-dimensional position of the inner cell mass mesh model, the inner cell mass mesh model and the internal three-dimensional point cloud model are positionally matched to obtain the three-dimensional geometric model.

[0034] In an optional embodiment, determining the best focal plane of the inner cell mass from the multiple focal planes according to the area of ​​the mask of the inner cell mass in the multiple focal plane images comprises:

[0035] According to the areas of the masks of the inner cell mass in the multiple focal plane images, determining a target focal plane image having the largest area of ​​the mask of the inner cell mass from the multiple focal plane images;

[0036] Using the mask of the inner cell mass in the target focal plane image as a segmentation mask, segmenting the multiple focal plane images to obtain multiple segmented focal plane images;

[0037] According to the sharpness of the plurality of segmented focal plane images, the best focal plane of the inner cell mass is determined from the plurality of focal planes.

[0038] In a second aspect, the present application also provides a three-dimensional fitting reconstruction device for a blastocyst, comprising:

[0039] An acquisition module, used for acquiring multiple focal plane images of the blastocyst;

[0040] an identification module, configured to identify a mask of trophoblast cells and a mask of an inner cell mass in the blastocyst based on the multiple focal plane images;

[0041] A construction module, used to construct a trophoblast cell grid model and an inner cell mass grid model respectively according to the trophoblast cell mask and the inner cell mass mask;

[0042] The construction module is further used to construct a depth map of the blastocyst according to the multiple focal plane images;

[0043] A fusion module, used for performing image fusion on the multiple focal plane images to obtain a texture image;

[0044] A mapping module, used for performing plane mapping according to the texture image and the depth map to generate an internal three-dimensional point cloud model of the blastocyst;

[0045] The construction module is further used to construct a three-dimensional geometric model of the blastocyst according to the trophoblast cell grid model, the inner cell mass grid model and the internal three-dimensional point cloud model;

[0046] The mapping module is also used to map the texture image onto the surface of the three-dimensional geometric model to generate the three-dimensional model of the blastocyst.

[0047] In an optional implementation, the building block is specifically used to:

[0048] Obtaining the clarity of each pixel in the multiple focal plane images;

[0049] Determine, from the multiple focal planes corresponding to the multiple focal plane images, a target focal plane having the maximum clarity of each pixel point, and obtain depth information of the target focal plane;

[0050] The depth map is constructed according to the depth information of the target focal plane.

[0051] In an optional implementation manner, the acquisition module is specifically used to:

[0052] Acquiring a plurality of first focal plane images of the blastocyst;

[0053] The multiple first focal plane images are interpolated to obtain multiple second focal plane images of the blastocyst, wherein the multiple focal plane images include: the multiple first focal plane images and the multiple second focal plane images.

[0054] In an optional embodiment, the device further comprises:

[0055] A segmentation module, used for segmenting the multiple focal plane images to obtain the blastocyst in the multiple focal plane images;

[0056] A moving module is used to move the blastocyst in the target focal plane image to the center of a preset field of view if there is a target focal plane image among the multiple focal plane images, and to perform boundary filling to regenerate the target focal plane image, wherein the blastocyst in the target focal plane image is not in the center of the preset field of view.

[0057] In an optional implementation manner, the identification module is specifically used to:

[0058] Marking trophoblast cell regions on the multiple focal plane images respectively according to preset marking patterns corresponding to the multiple focal plane images;

[0059] Identifying the trophoblast cell region to obtain a mask of the trophoblast cell;

[0060] An inner cell mass region located inside the trophoblast cell region is identified to obtain a mask of the inner cell mass.

[0061] In an optional embodiment, the device further comprises:

[0062] a determination module, configured to determine a preset focal plane among a plurality of focal planes corresponding to the plurality of focal plane images as an optimal focal plane for the trophoblast cells;

[0063] The determination module is further configured to determine the best focal plane of the inner cell mass from the multiple focal planes according to the area of ​​the mask of the inner cell mass in the multiple focal plane images;

[0064] The determination module is further used to determine the three-dimensional position of the trophoblast cell grid model according to the best focal plane of the trophoblast cell, and to determine the three-dimensional position of the inner cell mass grid model according to the best focal plane of the inner cell mass;

[0065] In an optional implementation, the building block is specifically used to:

[0066] According to the three-dimensional position of the trophoblast cell mesh model, the trophoblast cell mesh model and the internal three-dimensional point cloud model are positionally matched, and according to the three-dimensional position of the inner cell mass mesh model, the inner cell mass mesh model and the internal three-dimensional point cloud model are positionally matched to obtain the three-dimensional geometric model.

[0067] In an optional implementation manner, the determining module is specifically configured to:

[0068] According to the areas of the masks of the inner cell mass in the multiple focal plane images, determining a target focal plane image having the largest area of ​​the mask of the inner cell mass from the multiple focal plane images;

[0069] Using the mask of the inner cell mass in the target focal plane image as a segmentation mask, segmenting the multiple focal plane images to obtain multiple segmented focal plane images;

[0070] According to the sharpness of the multiple segmented focal plane images, the optimal focal plane of the inner cell mass is determined from the multiple focal planes.

[0071] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to execute any one of the methods described in the first aspect.

[0072] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method described in any one of the first aspects is executed.

[0073] The present application provides a three-dimensional fitting reconstruction method, device, electronic device and storage medium of a blastocyst, the method comprising: obtaining multiple focal plane images of the blastocyst, identifying the mask of the trophoblast cells and the mask of the inner cell mass in the blastocyst based on the multiple focal plane images, constructing a trophoblast cell grid model and an inner cell mass grid model respectively according to the mask of the trophoblast cells and the mask of the inner cell mass, constructing a depth map of the blastocyst according to the multiple focal plane images, performing image fusion on the multiple focal plane images to obtain a texture image, performing plane mapping according to the texture image and the depth map to generate an internal three-dimensional point cloud model of the blastocyst, constructing a three-dimensional geometric model of the blastocyst according to the trophoblast cell grid model, the inner cell mass grid model and the internal three-dimensional point cloud model, mapping the texture image onto the surface of the three-dimensional geometric model, and generating a three-dimensional model of the blastocyst. This scheme uses the focal plane image of the blastocyst to construct a three-dimensional model of the blastocyst, does not require additional embryo operation and imaging hardware structure, does not affect embryonic development, is low in cost, safe, reliable, convenient and effective. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0075] Figure 1 Schematic diagram of the process of the three-dimensional fitting reconstruction method of the blastocyst provided in the embodiment of the present application Figure 1 ;

[0076] Figure 2 A schematic diagram of a light slice model of multiple focal planes is provided for an embodiment of the present application;

[0077] Figure 3 A schematic diagram of the construction process of the trophoblast grid model and the inner cell mass grid model provided in the embodiments of the present application;

[0078] Figure 4 A schematic diagram of a texture image provided in an embodiment of the present application;

[0079] Figure 5 A schematic diagram of texture mapping provided in an embodiment of the present application;

[0080] Figure 6 Schematic diagram of the process of the three-dimensional fitting reconstruction method of the blastocyst provided in the embodiment of the present application Figure 2 ;

[0081] Figure 7 Schematic diagram of the process of the three-dimensional fitting reconstruction method of the blastocyst provided in the embodiment of the present application Figure 3 ;

[0082] Figure 8 Schematic diagram of the process of the three-dimensional fitting reconstruction method of the blastocyst provided in the embodiment of the present application Figure 4 ;

[0083] Fig. 9 A schematic diagram of a focal plane image provided in an embodiment of the present application;

[0084] Fig.10 Schematic diagram of the process of the three-dimensional fitting reconstruction method of the blastocyst provided in the embodiment of the present application Figure 5 ;

[0085] Fig.11 A schematic diagram of auxiliary circle markings provided in an embodiment of the present application;

[0086] Fig.12 Schematic diagram of the process of the three-dimensional fitting reconstruction method of the blastocyst provided in the embodiment of the present application Figure 6 ;

[0087] Fig.13 A schematic diagram of the structure of a three-dimensional fitting reconstruction device for a blastocyst provided in an embodiment of the present application;

[0088] Fig.14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0089] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0090] First, the professional terms involved in this application are explained:

[0091] Timelapse: time difference.

[0092] HMC: Hoffman Modulation Contrast.

[0093] TE: Trophectoderm.

[0094] ICM: Inner cell mass.

[0095] With the rapid development and increasing maturity of artificial insemination (ART) and artificial intelligence (AI) technologies, medical-based AI three-dimensional imaging technology has made non-invasive three-dimensional fitting reconstruction of embryos possible. At present, most embryologists prefer to use multi-focal plane data obtained by time lapse to evaluate the morphology and dynamics of embryos. This method has promoted embryologists' research and evaluation of pre-implantation embryo development. However, compared with the observation of two-dimensional images, embryologists are more looking forward to observing embryos from a higher dimension and hope to obtain a complete model with rich morphological information and spatial characteristics based on existing imaging technology to evaluate and study embryos.

[0096] In current research, the three-dimensional model of the pre-implantation embryo is mainly obtained by fluorescent staining and confocal microscopy imaging to obtain the serial fluorescent slice data of the Z-stack (a stack image set based on the Z axis) for reconstruction of the three-dimensional model. However, this method requires the embryo to be stained and irradiated with a high-energy spectrum, which will cause irreversible damage to the embryo.

[0097] In addition, the current mainstream direction of blastocyst evaluation is through static two-dimensional images, or based on expert consensus (Gardener), or based on AI models, and the evaluation criteria are based on the morphological characteristics of trophoblast cells and inner cell mass and the abstract high-dimensional features extracted by AI. For the data source, the vertical spatial information of the blastocyst is ignored in the evaluation process. For the evaluation criteria, the impact of the interaction between trophoblast cells and inner cell mass on embryonic development is ignored.

[0098] In response to the above problems, this application is based on the fitting and reconstruction of the three-dimensional model of the blastocyst under the time difference culture system, and multiple focal plane images are obtained through the Hoffman modulation contrast microscope imaging system. This technical means is safe and reliable for embryo imaging. Compared with confocal microscope or other microscope imaging, the method of obtaining data under the time difference system is simpler and safer. In the process of popularization of time difference, reproductive centers have also accumulated a large number of embryos with multiple focal plane images for reconstruction of three-dimensional models, which makes the method of three-dimensional fitting and reconstruction under the time difference culture system more valuable for exploration. In other words, this scheme uses the focal plane image of the blastocyst to construct a three-dimensional model of the blastocyst, without the need for additional embryo operation and imaging hardware structure, and will not affect embryonic development. It is low-cost, safe, reliable, convenient and effective. Moreover, by combining the trophoblast cells and inner cell mass of the blastocyst to reconstruct the geometric spatial structure of the blastocyst, the three-dimensional morphological structural characteristics of the blastocyst can be further analyzed.

[0099] It is worth mentioning that by fitting and reconstructing the non-invasive living blastocyst three-dimensional model through multiple focal plane embryo images obtained under the time-difference culture system, the morphological and spatial characteristics of the blastocyst can be calculated through the three-dimensional model of the blastocyst, and the corresponding characteristics can be analyzed for correlation with the embryo transplantation outcomes, in order to explore the characteristics that affect pregnancy and live birth during the blastocyst development process.

[0100] Figure 1 Schematic diagram of the process of the three-dimensional fitting reconstruction method of the blastocyst provided in the embodiment of the present application Figure 1 The execution subject of this embodiment may be an electronic device, such as a terminal device.

[0101] like Figure 1 As shown, the method may include:

[0102] S101. Acquire multiple focal plane images of the blastocyst.

[0103] The blastocyst is a blastocyst to be subjected to three-dimensional fitting reconstruction, for example, it may be a stage 3 blastocyst or a stage 4 blastocyst.

[0104] Using Hoffman modulation contrast microscopy, multiple focal plane images of the blastocyst were acquired.

[0105] It is worth noting that multiple focal plane images of the embryo can be obtained through the time-difference incubator. Due to the particularity of Hoffman modulation contrast microscopy imaging, the focal plane image has a sense of relief (the cell edge has a strong contrast). Among them, multiple focal planes can be approximately regarded as "light slices" for the horizontal cutting of the sphere. Around the Z axis, multiple focal planes are taken at equal intervals, not limited to 11 focal planes. Figure 2 A schematic diagram of a light slice model of multiple focal planes is provided for the embodiment of the present application, such as Figure 2 As shown, the multiple focal planes are respectively recorded as F75, F60, F45, F30, F15, F0, F-15, F-30, F-45, F-60, and F-75. Each focal plane is a plane determined by a horizontal cut of the spherical surface. As an example, in an embodiment of the present application, the collection spacing between each two focal planes can be 15μm.

[0106] S102. Based on the multiple focal plane images, identify the mask of the trophoblast cells and the mask of the inner cell mass in the blastocyst.

[0107] The multiple focal plane images are respectively identified to identify the mask of the trophoblast cells and the mask of the inner cell mass in the blastocyst in each focal plane image, wherein the trophoblast cells are used to develop into the placenta, and the inner cell mass is used to develop into the embryo individual.

[0108] In some embodiments, a pre-trained semantic segmentation model of trophoblast cells is used to perform semantic segmentation on each focal plane image to obtain a mask of the trophoblast cells in the blastocyst in each focal plane image, and a pre-trained semantic segmentation model of inner cell mass is used to perform semantic segmentation on each focal plane image to obtain a mask of the inner cell mass in the blastocyst in each focal plane image, wherein the semantic segmentation model of trophoblast cells and the semantic segmentation model of inner cell mass are respectively obtained by training and learning multiple manually annotated focal plane image samples, wherein the semantic segmentation method includes the following schemes: FCN, UNet, Unet++, DeepLabV3, and the preferred method is U2Net.

[0109] S103. Constructing a trophoblast cell grid model and an inner cell mass grid model respectively according to the trophoblast cell mask and the inner cell mass mask.

[0110] According to the masks of multiple trophoblast cells corresponding to the multiple focal plane images, the geometric structure of the trophoblast cells is fitted to construct a trophoblast cell grid model. According to the masks of multiple inner cell masses corresponding to the multiple focal plane images, the geometric structure of the inner cell masses is fitted to construct an inner cell mass grid model. In other words, a three-dimensional model of the trophoblast cells and a three-dimensional model of the inner cell masses are obtained by "optical slice stacking".

[0111] In some embodiments, contour extraction is performed on masks of multiple trophoblast cells respectively, and surface geometry fitting is performed on the coordinate points of the contours corresponding to the masks of the multiple trophoblast cells in azimuth order to obtain a point cloud fitting model of the trophoblast cells; and contour extraction is performed on masks of multiple inner cell mass cells respectively, and surface geometry fitting is performed on the coordinate points of the contours corresponding to the masks of the multiple inner cell mass cells in azimuth order to obtain a point cloud fitting model of the inner cell mass; then, the point cloud fitting model of the trophoblast cells and the point cloud fitting model of the inner cell mass are converted into a grid model based on a normal calculation and a surface reconstruction algorithm, wherein the azimuth order can be understood as an order from 0° to 360°.

[0112] Figure 3 A schematic diagram of the construction process of the trophoblast grid model and the inner cell mass grid model provided in the embodiments of the present application is shown in FIG. Figure 3 As shown, for each focal plane image, the mask of the trophoblast cells and the mask of the inner cell mass in the blastocyst are identified, and the contours of the trophoblast cells and the mask of the inner cell mass are extracted respectively, and the coordinate points of the contours of the masks of multiple trophoblast cells and the coordinate points of the contours of the mask of the inner cell mass are surface fitted to generate a trophoblast mesh model and an inner cell mass mesh model.

[0113] S104. Construct a depth map of the blastocyst based on the multiple focal plane images.

[0114] A depth estimation method is used to estimate the depth information of the blastocyst based on multiple focal plane images to construct a depth map of the blastocyst, wherein the depth map of the blastocyst is used to indicate the depth information of the blastocyst. The depth estimation method is based on a geometric method to estimate the three-dimensional structure from a series of two-dimensional image sequences.

[0115] Among them, the depth estimation method can be based on the traditional focus ranging algorithm (Deep Flow Field Fusion, DFF), which is to obtain image data with different defocus degrees by adjusting different Z-axis positions within the same field of view, and estimate the depth distance of the image by the clarity of different layers of images. Alternatively, the depth estimation method can be an end-to-end depth prediction method, which performs depth estimation through a single image. The depth estimation method has the following schemes: Defocus-Net, DepthCNN, DFV, preferably DepthCNN.

[0116] In some embodiments, multiple focal plane images are divided into two groups of image sequences, and the multiple focal planes are recorded as Fn~Fn. The focal planes of the two groups of image sequences are Fn~F0 and F0~Fn respectively. It is worth noting that the number of focal plane images in the image sequence is greater than or equal to 10. A depth estimation method is used to construct an upper depth map of the blastocyst based on one group of image sequences, and a depth estimation method is used to construct a lower depth map of the blastocyst based on the other group of image sequences.

[0117] S105 , performing image fusion on multiple focal plane images to obtain a texture image.

[0118] An image fusion network is used to fuse multiple focal plane images to obtain a texture image of the blastocyst. The image fusion methods include the following schemes: Densefuse-net, FusionNet, and U2FusionNet: U2FusionNet is preferred.

[0119] In some embodiments, image fusion is performed on one set of image sequences to obtain an upper texture image (Up_TextMap) of the blastocyst, and image fusion is performed on another set of image sequences to obtain a lower texture image (Down_Textmap) of the blastocyst.

[0120] Figure 4 A schematic diagram of a texture image provided in an embodiment of the present application, such as Figure 4 As shown, a texture image is obtained by performing image fusion on multiple focal plane images.

[0121] S106, performing plane mapping according to the texture image and the depth map to generate an internal three-dimensional point cloud model of the blastocyst.

[0122] A texture mapping algorithm is used to map the texture image plane onto the depth map of the blastocyst to generate a three-dimensional point cloud model of the interior of the blastocyst, wherein multiple focal plane images are fused to obtain a texture image, and the texture image is used to reflect the surface texture of the interior of the blastocyst. The texture image is a plane image, and the depth map is used to reflect the depth information of the blastocyst. Therefore, the texture image plane is mapped onto the depth map of the blastocyst to generate a three-dimensional point cloud model of the interior of the blastocyst.

[0123] In some embodiments, the upper depth map and the upper texture image are planar mapped to generate an upper internal three-dimensional model of the blastocyst, and the lower depth map and the lower texture image are planar mapped to generate a lower internal three-dimensional point cloud model of the blastocyst.

[0124] S107. Construct a three-dimensional geometric model of the blastocyst based on the trophoblast cell grid model, the inner cell mass grid model, and the internal three-dimensional point cloud model.

[0125] The point cloud positions of the trophoblast cell grid model, the inner cell mass grid model and the internal three-dimensional point cloud model are matched to set the trophoblast cell grid model and the inner cell mass grid model in the internal three-dimensional point cloud model to generate a three-dimensional geometric model of the blastocyst.

[0126] S108. Map the texture image onto the surface of the three-dimensional geometric model to generate a three-dimensional model of the blastocyst.

[0127] Since HMC is a method based on bright field imaging, the surface of the blastocyst is mainly composed of the zona pellucida and TE cells, both of which are cell membrane structures and are light-transmitting. Therefore, the information presented on the surface of the blastocyst includes the texture information inside the blastocyst.

[0128] Based on this, what is seen from the surface of the blastocyst is the real texture of the blastocyst. Therefore, the texture image of the blastocyst obtained by image fusion can be used as the outer surface texture image of the blastocyst. The texture mapping method is used to map the texture image onto the surface of the three-dimensional geometric model of the blastocyst to generate a three-dimensional model of the blastocyst.

[0129] In some embodiments, an upper texture image is mapped onto an upper surface of a three-dimensional geometric model of a blastocyst, and a lower texture image is mapped onto a lower surface of the three-dimensional geometric model of the blastocyst to generate a three-dimensional model of the blastocyst.

[0130] Taking the hemispherical texture mapping method as an example, Figure 5 A schematic diagram of texture mapping provided in an embodiment of the present application, such as Figure 5 As shown in the figure, the three-dimensional geometric model of the blastocyst is a spherical model, and the plane where the texture image is located is used as the projection plane. At the same time, the plane is parallel to the equatorial plane of the hemispherical surface of the spherical model, and is tangent to the hemispherical surface at the positive pole A, and the projection center is placed on the negative pole B of the sphere. The corresponding relationship between the coordinates of the plane where the texture image is located and the spherical coordinates is determined by projection, and the arc on the plane where the texture image is located is mapped to the latitude on the hemispherical surface, and the radial line segments on the plane where the texture image is located are mapped to the longitude on the hemispherical surface. The mapping formula is:

[0131]

[0132] in, is the pitch angle, θ is the yaw angle, uv is the texture coordinate of the pixel point of the texture image,

[0133] In this embodiment, a three-dimensional model of the blastocyst is reconstructed through multiple focal plane images of the blastocyst, which solves the difficulty of three-dimensional living cell imaging of the blastocyst before transplantation, provides a new perspective for embryologists to explore and study the development process of the embryo, and allows embryologists to observe and evaluate the development of the blastocyst more safely and stereoscopically, and creates conditions for evaluating the embryo based on its three-dimensional morphological characteristics. Embryologists can calculate new three-dimensional features through the three-dimensional model of the blastocyst to obtain more details and processes of embryonic development, and use these features for the selection of embryo transplantation, which is of great significance for retrospective analysis of embryos.

[0134] Figure 6 Schematic diagram of the process of the three-dimensional fitting reconstruction method of the blastocyst provided in the embodiment of the present application Figure 2 ,like Figure 6 As shown, in an optional embodiment, step S104, constructing a depth map of the blastocyst according to the multiple focal plane images, may include:

[0135] S201, obtaining the clarity of each pixel in a plurality of focal plane images.

[0136] The clarity of each pixel in the focal plane image is extracted to obtain the clarity of each pixel in the focal plane image, wherein the number of pixels in multiple focal plane images is the same, and the multiple focal plane images are denoted as f0 to fn, and the clarity at the pixel point (x, y) is y0-yn.

[0137] S202 . Determine a target focal plane with the greatest definition of each pixel point from among multiple focal planes corresponding to the multiple focal plane images, and obtain depth information of the target focal plane.

[0138] For each pixel, the target focal plane with the maximum definition of each pixel is determined from the multiple focal planes corresponding to the multiple focal plane images, wherein one focal plane corresponds to one depth information, and the depth information of the target focal plane is obtained, that is, the maximum definition y max The depth information of the focal plane is the depth information of the pixel. The depth information of the focal plane can be understood as Figure 2 The Z value of the focal plane on the Z axis in the three-dimensional space coordinate system established with the center of the F0 focal plane as the origin.

[0139] S203: construct a depth map according to the depth information of the target focal plane.

[0140] According to the depth information of the target focal plane of all pixels, interpolation fitting is performed to construct the depth map of the blastocyst. That is to say, by obtaining the clear focus position corresponding to each pixel, a depth map with very clear depth of each depth of field is constructed, so as to restore the depth information of the blastocyst through focusing analysis, and interpolation fitting is performed on the depth information to restore more accurate depth information of the blastocyst.

[0141] In some embodiments, the depth map may also be subjected to mean filtering to smooth the depth map, thereby avoiding excessive differences in pixel points in the depth map. The mean filtering method is mainly a neighborhood averaging method. For a pixel point (x, y) to be processed, a template consisting of several neighboring pixel points is selected, and the average value of the depth values ​​of all pixels in the template is calculated. The average value is then assigned to the current pixel point (x, y) as the grayscale value g(x, y) of the processed image at the pixel point:

[0142] g(x,y)=1 / m∑f(x,y)

[0143] Among them, m represents the number of pixels in the template including the current pixel. The grayscale value of the filtered depth map is selected. The selected grayscale value is the focus position in multiple focal plane images, thereby generating a depth map with clear depth of field at each position, wherein the grayscale value ranges from 0 to 255.

[0144] Figure 7 Schematic diagram of the process of the three-dimensional fitting reconstruction method of the blastocyst provided in the embodiment of the present application Figure 3 ,like Figure 7 As shown, in an optional embodiment, step S101, obtaining multiple focal plane images of the blastocyst, includes:

[0145] S301, acquiring a plurality of first focal plane images of the blastocyst.

[0146] S302, performing interpolation processing on the multiple first focal plane images to obtain multiple second focal plane images of the blastocyst.

[0147] A Hoffman modulation contrast microscope is used to collect multiple first focal plane images of the blastocyst, wherein the number of the multiple first focal plane images can be 11, that is, 11 focal images, and then the multiple first focal plane images are interpolated to obtain multiple second focal plane images of the blastocyst, wherein the multiple focal plane images include: multiple first focal plane images and multiple second focal plane images, and the number of multiple focal plane images can exceed 20 focal points.

[0148] It is worth noting that in the time-difference culture system, the number of focal plane images that can be collected is limited. Fewer focal planes means that a smaller number of light slices are obtained. The number of light slices determines the error between the reconstructed trophoblast cell grid model and the inner cell mass grid model and the real blastocyst. If it is limited to the first focal plane image, more spatial information will be lost. In order to solve the problem of insufficient number of focal planes, a generative adversarial network can be used to interpolate multiple first focal plane images to generate a new focal plane between two adjacent first focal planes and generate a second focal plane image between two adjacent first focal plane images.

[0149] In this embodiment, each layer of interpolated images needs to have the spatial and texture information of two adjacent focal plane images. Therefore, a generative adversarial network can be used to interpolate multiple first focal plane images to generate multiple second focal plane images, thereby ensuring that more focal plane images can be generated through image interpolation without increasing the number of photographs and exposure time of the blastocyst.

[0150] Figure 8 Schematic diagram of the process of the three-dimensional fitting reconstruction method of the blastocyst provided in the embodiment of the present application Figure 4 ,like Figure 8 As shown, in an optional embodiment, before step S103, respectively constructing the trophoblast cell grid model and the inner cell mass grid model according to the trophoblast cell mask and the inner cell mass mask, the method may further include:

[0151] S401, segmenting a plurality of focal plane images to obtain blastocysts in the plurality of focal plane images.

[0152] Each focal plane image is segmented to obtain the blastocyst in each focal plane image, that is, the focal plane image is preprocessed to determine the region of interest (ROI) of the blastocyst.

[0153] S402: If there is a target focal plane image in the multiple focal plane images, move the blastocyst in the target focal plane image to the center of a preset field of view, and perform boundary filling to regenerate the target focal plane image.

[0154] Determine whether there is a target focal plane image among the multiple focal plane images, and the blastocyst in the target focal plane image is not in the center of the preset field of view. If there is a target focal plane image, move the blastocyst in the target focal plane image to the center of the preset field of view, and perform border filling to regenerate the target focal plane image, wherein the center of the preset field of view can be understood as the center of the focal plane image, and the border filling can be understood as filling the area outside the blastocyst with a preset color such as black, wherein the target focal plane image can be, for example, an F0 focal plane image.

[0155] In some embodiments, other focal plane images other than the target focal plane image in the multiple focal plane images may be boundary filled to regenerate other focal plane images.

[0156] It is worth noting that the resolution of the regenerated target focal plane image is consistent with that of the original target focal plane image. The purpose of border filling is to set the spatial coordinate system reference point of the reconstructed blastocyst three-dimensional model to the center point of the blastocyst, thereby improving the reconstruction accuracy of the three-dimensional model.

[0157] Fig. 9 A schematic diagram of a focal plane image provided in an embodiment of the present application, such as Fig. 9 As shown, the blastocyst is moved to the center of the preset field of view and the area outside the blastocyst is filled with black.

[0158] Fig.10 Schematic diagram of the process of the three-dimensional fitting reconstruction method of the blastocyst provided in the embodiment of the present application Figure 5 ,like Fig.10 As shown, in an optional embodiment, step S102, based on multiple focal plane images, identifying the mask of the trophoblast cells and the mask of the inner cell mass in the blastocyst may include:

[0159] S501 . Mark trophoblast cell regions on a plurality of focal plane images respectively according to preset marking patterns corresponding to the plurality of focal plane images.

[0160] Among them, the preset marking graphic can be an auxiliary circle, each focal plane corresponds to an auxiliary circle, and the sizes of different auxiliary circles are different. According to the preset marking graphics corresponding to the multiple focal plane images, the trophoblast cell area is marked on the multiple focal plane images respectively. The trophoblast cell area is the area marked by the preset marking graphic, such as the area within the auxiliary circle.

[0161] S502. Identify the trophoblast cell region to obtain a mask of the trophoblast cells.

[0162] S503, identifying the inner cell mass region located inside the trophoblast cell region to obtain a mask of the inner cell mass.

[0163] The marked trophoblast cell region is identified to identify the mask of the trophoblast cells in the trophoblast cell region, and the inner cell mass region located inside the trophoblast cell region is identified to obtain the mask of the inner cell mass.

[0164] In this embodiment, due to the particularity of HMC bright field imaging, in order to eliminate the interference of artifact images between adjacent layers, the focus plane images are graphically marked, and combined with the structural characteristics of the spherical shape of the blastocyst, multiple focus plane images are approximately regarded as horizontal cuts of the light section on the spherical surface. The recognition accuracy of the mask of the trophoblast cells and the mask of the inner cell mass is improved. Among them, the overlap of focus plane artifacts in the range of F15-F75 is particularly obvious.

[0165] See also Figure 2 The boundary line of each optical section (i.e., the auxiliary circle in the figure) is used as the auxiliary marking range of the trophoblast cells, that is, the trophoblast cells are within the range of the auxiliary circle. Fig.11 A schematic diagram of auxiliary circle markings provided in an embodiment of the present application, such as Fig.11 As shown, the area within the auxiliary circle is the trophoblast cell area, that is, the area outside the auxiliary circle is the interference of the artifact image brought by the adjacent focal plane.

[0166] Fig.12 Schematic diagram of the process of the three-dimensional fitting reconstruction method of the blastocyst provided in the embodiment of the present application Figure 6 ,like Fig.12 As shown, in an optional implementation, the method may further include:

[0167] S601, determining a preset focal plane among a plurality of focal planes corresponding to a plurality of focal plane images as an optimal focal plane for trophoblast cells.

[0168] Among them, one focal plane image corresponds to one focal plane, the preset focal plane may be the F0 focal plane, and the F0 focal plane is determined to be the best focal plane for the trophoblast cells, that is, the mask of the trophoblast cells is clearest at the preset focal plane.

[0169] S602: Determine the best focal plane of the inner cell mass from multiple focal planes according to the area of ​​the mask of the inner cell mass in multiple focal plane images.

[0170] The areas of the inner cell mass mask in the multiple focal plane images are acquired, and according to the areas of the inner cell mass mask, an optimal focal plane where the inner cell mass mask is clearest is determined from the multiple focal planes.

[0171] In an optional embodiment, step S602, determining the best focal plane of the inner cell mass from multiple focal planes according to the area of ​​the mask of the inner cell mass in multiple focal plane images, includes:

[0172] According to the areas of the masks of the inner cell mass in the multiple focal plane images, a target focal plane image having the largest area of ​​the mask of the inner cell mass is determined from the multiple focal plane images.

[0173] The mask of the inner cell mass in the target focal plane image is used as a segmentation mask to segment the multiple focal plane images to obtain multiple segmented focal plane images.

[0174] The best focal plane of the inner cell mass was determined from multiple focal planes based on the clarity of multiple segmented focal plane images.

[0175] According to the area of ​​the mask of the inner cell mass in the multiple focal plane images, a target focal plane with the largest mask area of ​​the inner cell mass is determined from the multiple focal planes, and then the area of ​​the inner cell mass in the target focal plane image is used as a segmentation mask to segment the multiple focal plane images to obtain segmented focal plane images corresponding to each focal plane image, and then according to the clarity of the multiple segmented focal plane images, the best focal plane of the inner cell mass is determined from the multiple focal planes, wherein the best focal plane is the focal plane corresponding to the focal plane image with the largest clarity among the multiple segmented focal plane images.

[0176] In this embodiment, the mask of the inner cell mass in the target focal plane image is used as a segmentation mask to obtain multiple segmented focal plane images, and then the optimal focal plane of the inner cell mass is determined according to the clarity of the multiple segmented focal plane images. In this way, the optimal focal plane can be determined based on the image clarity when the area is the same, with high accuracy.

[0177] It is worth noting that the best focal plane of the trophoblast cells can be understood as the equatorial plane of the mask of the trophoblast cells, and the best focal plane of the inner cell mass can be understood as the equatorial plane of the mask of the inner cell mass.

[0178] S603. Determine the three-dimensional position of the trophoblast cell grid model according to the best focal plane of the trophoblast cells, and determine the three-dimensional position of the inner cell mass grid model according to the best focal plane of the inner cell mass.

[0179] The center of the best focal plane of the trophoblast cells is used as the origin to establish a three-dimensional spatial coordinate system, obtain the plane position of the trophoblast cell grid model, that is, the X value and Y value, and set the Z axis position of the best focal plane of the trophoblast cells as the Z value of the center point of the trophoblast cell grid model, thereby obtaining the three-dimensional position of the trophoblast cell grid model. Figure 2 Similar, except that the coordinate origin is different.

[0180] The center of the best focal plane of the inner cell mass is used as the origin to establish a three-dimensional plane coordinate system, and the plane position of the inner cell mass grid model, i.e., the X value and Y value, is obtained. The Z-axis position of the best focal plane of the inner cell mass is used as the Z value of the center point of the inner cell mass grid model, thereby obtaining the three-dimensional position of the inner cell mass grid model. Figure 2 Similar, except that the coordinate origin is different.

[0181] Step S107, constructing a three-dimensional geometric model of the blastocyst according to the trophoblast cell grid model, the inner cell mass grid model and the internal three-dimensional point cloud model, may include:

[0182] S604. According to the three-dimensional position of the trophoblast cell mesh model, the trophoblast cell mesh model and the internal three-dimensional point cloud model are matched in position, and according to the three-dimensional position of the inner cell mass mesh model, the inner cell mass mesh model and the internal three-dimensional point cloud model are matched in position to obtain a three-dimensional geometric model.

[0183] According to the three-dimensional position of the trophoblast cell mesh model, the trophoblast cell mesh model and the internal three-dimensional point cloud model are positionally matched to set the trophoblast cell mesh model in the internal three-dimensional point cloud model; and according to the three-dimensional position of the inner cell mass mesh model, the inner cell mass mesh model and the internal three-dimensional point cloud model are positionally matched to set the inner cell mass mesh model in the internal three-dimensional point cloud model, thereby obtaining a three-dimensional geometric model.

[0184] In this embodiment, when reconstructing the three-dimensional model of the blastocyst, the x and y values ​​are determined by the optimal focal plane, and the z value is obtained through the z axis of the optimal focal plane, so that the accurate spatial positions of the trophoblast cell grid model and the inner cell mass grid model can be obtained, and then the positions are matched to obtain the three-dimensional geometric model of the blastocyst.

[0185] Based on the same inventive concept, an embodiment of the present application also provides a three-dimensional fitting reconstruction device for a blastocyst corresponding to the three-dimensional fitting reconstruction method for a blastocyst. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the three-dimensional fitting reconstruction method for the blastocyst mentioned above in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0186] Fig.13 This is a schematic structural diagram of a three-dimensional fitting reconstruction device for a blastocyst provided in an embodiment of the present application. The device can be integrated into an electronic device.

[0187] like Fig.13 As shown, the device may include:

[0188] An acquisition module 701 is used to acquire multiple focal plane images of the blastocyst;

[0189] An identification module 702 is used to identify the mask of the trophoblast cells and the mask of the inner cell mass in the blastocyst based on the multiple focal plane images;

[0190] A construction module 703 is used to construct a trophoblast cell grid model and an inner cell mass grid model respectively according to the trophoblast cell mask and the inner cell mass mask;

[0191] The construction module 703 is further used to construct a depth map of the blastocyst according to the multiple focal plane images;

[0192] A fusion module 704 is used to fuse multiple focal plane images to obtain a texture image;

[0193] A mapping module 705 is used to perform plane mapping according to the texture image and the depth map to generate an internal three-dimensional point cloud model of the blastocyst;

[0194] The construction module 703 is further used to construct a three-dimensional geometric model of the blastocyst based on the trophoblast cell grid model, the inner cell mass grid model and the internal three-dimensional point cloud model;

[0195] The mapping module 705 is further used to map the texture image onto the surface of the three-dimensional geometric model to generate a three-dimensional model of the blastocyst.

[0196] In an optional implementation, the construction module 703 is specifically configured to:

[0197] Obtaining the clarity of each pixel in multiple focal plane images;

[0198] Determine a target focal plane with the maximum clarity of each pixel from multiple focal planes corresponding to the multiple focal plane images, and obtain depth information of the target focal plane;

[0199] A depth map is constructed based on the depth information of the target focal plane.

[0200] In an optional implementation, the acquisition module 701 is specifically configured to:

[0201] acquiring a plurality of first focal plane images of the blastocyst;

[0202] The multiple first focal plane images are interpolated to obtain multiple second focal plane images of the blastocyst, where the multiple focal plane images include: multiple first focal plane images and multiple second focal plane images.

[0203] In an optional embodiment, the device further comprises:

[0204] A segmentation module 706 is used to segment the multiple focal plane images to obtain the blastocysts in the multiple focal plane images;

[0205] The moving module 707 is used to move the blastocyst in the target focal plane image to the center of the preset field of view if there is a target focal plane image in the multiple focal plane images, and to regenerate the target focal plane image by filling the borders. The blastocyst in the target focal plane image is not in the center of the preset field of view.

[0206] In an optional implementation, the identification module 702 is specifically configured to:

[0207] Marking trophoblast cell regions on the multiple focal plane images respectively according to preset marking patterns corresponding to the multiple focal plane images;

[0208] The trophoblast cell region is identified to obtain a mask of the trophoblast cells;

[0209] The inner cell mass region located inside the trophoblast cell region is identified to obtain a mask of the inner cell mass.

[0210] In an optional embodiment, the device further comprises:

[0211] A determination module 708, configured to determine that a preset focal plane among a plurality of focal planes corresponding to a plurality of focal plane images is an optimal focal plane for trophoblast cells;

[0212] The determination module 708 is further configured to determine the best focal plane of the inner cell mass from the multiple focal planes according to the area of ​​the mask of the inner cell mass in the multiple focal plane images;

[0213] The determination module 708 is further used to determine the three-dimensional position of the trophoblast cell grid model according to the best focal plane of the trophoblast cell, and to determine the three-dimensional position of the inner cell mass grid model according to the best focal plane of the inner cell mass;

[0214] In an optional implementation, the construction module 703 is specifically configured to:

[0215] According to the three-dimensional position of the trophoblast cell mesh model, the trophoblast cell mesh model and the internal three-dimensional point cloud model are matched in position, and according to the three-dimensional position of the inner cell mass mesh model, the inner cell mass mesh model and the internal three-dimensional point cloud model are matched in position to obtain a three-dimensional geometric model.

[0216] In an optional implementation, the determination module 708 is specifically configured to:

[0217] According to the areas of the masks of the inner cell mass in the multiple focal plane images, determining a target focal plane having the largest area of ​​the mask of the inner cell mass from the multiple focal planes;

[0218] Using the mask of the inner cell mass in the target focal plane image as a segmentation mask, segmenting the multiple focal plane images to obtain multiple segmented focal plane images;

[0219] The best focal plane of the inner cell mass was determined from multiple focal planes based on the clarity of multiple segmented focal plane images.

[0220] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0221] Fig.14A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Fig.14 As shown, the device may include: a processor 801, a memory 802 and a bus 803. The memory 802 stores machine-readable instructions executable by the processor 801. When the electronic device is running, the processor 801 communicates with the memory 802 through the bus 803, and the processor 801 executes the machine-readable instructions to perform the above method.

[0222] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The computer program is executed when a processor is running, and the processor executes the above method.

[0223] In the embodiment of the present application, the computer program can also execute other machine-readable instructions when run by the processor to execute other methods described in the embodiment. For the specific execution method steps and principles, please refer to the description of the embodiment, which will not be repeated here.

[0224] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0225] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0226] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0227] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0228] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0229] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A three-dimensional fitting reconstruction method for blastocysts, characterized in that: include: Acquire multiple focal plane images of the blastocyst; Based on the multiple focal plane images, identifying a mask of trophoblast cells and a mask of inner cell mass in the blastocyst; According to the mask of the trophoblast cells and the mask of the inner cell mass, constructing a trophoblast cell grid model and an inner cell mass grid model respectively; constructing a depth map of the blastocyst according to the multiple focal plane images; Performing image fusion on the multiple focal plane images to obtain a texture image; Performing plane mapping according to the texture image and the depth map to generate an internal three-dimensional point cloud model of the blastocyst; Constructing a three-dimensional geometric model of the blastocyst according to the trophoblast cell grid model, the inner cell mass grid model and the internal three-dimensional point cloud model; The texture image is mapped onto the surface of the three-dimensional geometric model to generate a three-dimensional model of the blastocyst.

2. The method according to claim 1, characterized in that: The step of constructing a depth map of the blastocyst according to the multiple focal plane images comprises: Obtaining the clarity of each pixel in the multiple focal plane images; Determine, from the multiple focal planes corresponding to the multiple focal plane images, a target focal plane having the maximum clarity of each pixel point, and obtain depth information of the target focal plane; The depth map is constructed according to the depth information of the target focal plane.

3. The method according to claim 1, characterized in that The step of obtaining multiple focal plane images of the blastocyst comprises: Acquiring a plurality of first focal plane images of the blastocyst; The multiple first focal plane images are interpolated to obtain multiple second focal plane images of the blastocyst, wherein the multiple focal plane images include: the multiple first focal plane images and the multiple second focal plane images.

4. The method according to claim 1, characterized in that Before respectively constructing the trophoblast cell grid model and the inner cell mass grid model according to the trophoblast cell mask and the inner cell mass mask, the method further comprises: Segmenting the multiple focal plane images to obtain the blastocyst in the multiple focal plane images; If there is a target focal plane image among the multiple focal plane images, the blastocyst in the target focal plane image is moved to the center of a preset field of view, and the boundary is filled to regenerate the target focal plane image, and the blastocyst in the target focal plane image is not in the center of the preset field of view.

5. The method according to any one of claims 1 to 4, characterized in that: The step of identifying the mask of the trophoblast cells and the mask of the inner cell mass in the blastocyst based on the multiple focal plane images comprises: Marking trophoblast cell regions on the multiple focal plane images respectively according to preset marking patterns corresponding to the multiple focal plane images; Identifying the trophoblast cell region to obtain a mask of the trophoblast cell; An inner cell mass region located inside the trophoblast cell region is identified to obtain a mask of the inner cell mass.

6. The method according to claim 1, characterized in that The method further comprises: Determining a preset focal plane among a plurality of focal planes corresponding to the plurality of focal plane images as an optimal focal plane for the trophoblast cells; determining an optimal focal plane of the inner cell mass from the plurality of focal planes according to the area of ​​the mask of the inner cell mass in the plurality of focal plane images; Determining the three-dimensional position of the trophoblast cell grid model according to the best focal plane of the trophoblast cell, and determining the three-dimensional position of the inner cell mass grid model according to the best focal plane of the inner cell mass; The constructing of the three-dimensional geometric model of the blastocyst according to the trophoblast cell grid model, the inner cell mass grid model and the internal three-dimensional point cloud model comprises: According to the three-dimensional position of the trophoblast cell mesh model, the trophoblast cell mesh model and the internal three-dimensional point cloud model are positionally matched, and according to the three-dimensional position of the inner cell mass mesh model, the inner cell mass mesh model and the internal three-dimensional point cloud model are positionally matched to obtain the three-dimensional geometric model.

7. The method according to claim 6, characterized in that Determining the best focal plane of the inner cell mass from the multiple focal planes according to the area of ​​the mask of the inner cell mass in the multiple focal plane images comprises: According to the areas of the masks of the inner cell mass in the multiple focal plane images, determining a target focal plane image having the largest area of ​​the mask of the inner cell mass from the multiple focal plane images; Using the mask of the inner cell mass in the target focal plane image as a segmentation mask, segmenting the multiple focal plane images to obtain multiple segmented focal plane images; According to the sharpness of the plurality of segmented focal plane images, the best focal plane of the inner cell mass is determined from the plurality of focal planes.

8. A three-dimensional fitting reconstruction device for blastocysts, characterized in that: include: An acquisition module, used for acquiring multiple focal plane images of the blastocyst; an identification module, configured to identify a mask of trophoblast cells and a mask of an inner cell mass in the blastocyst based on the multiple focal plane images; A construction module, used to construct a trophoblast cell grid model and an inner cell mass grid model respectively according to the trophoblast cell mask and the inner cell mass mask; The construction module is further used to construct a depth map of the blastocyst according to the multiple focal plane images; A fusion module, used for performing image fusion on the multiple focal plane images to obtain a texture image; A mapping module, used for performing plane mapping according to the texture image and the depth map to generate an internal three-dimensional point cloud model of the blastocyst; The construction module is further used to construct a three-dimensional geometric model of the blastocyst according to the trophoblast cell grid model, the inner cell mass grid model and the internal three-dimensional point cloud model; The mapping module is also used to map the texture image onto the surface of the three-dimensional geometric model to generate a three-dimensional model of the blastocyst.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is executed.

Images