Nouritrophoblast cell analysis method, device and equipment and storage medium

Abstract

The invention provides a trophoblast cell analysis method, device and equipment and a storage medium, and relates to the technical field of embryo quality evaluation. Comprising the steps of collecting a multi-focal plane image of a target blastocyst; performing image fusion on the multi-focal plane image based on a pre-trained image fusion model to generate a first fusion image and a second fusion image corresponding to the target blastocyst; performing main body region segmentation on the first fusion image and the second fusion image to obtain a first target image and a second target image; segmenting the trophoblast cells in the first target image and the second target image by adopting a pre-trained image segmentation model to obtain a first mask image and a second mask image; according to the first mask image and the second mask image, multiple feature parameters of the trophoblast cells are extracted, and based on the feature parameters, correlation between the feature parameters and the development result of the embryo is obtained through analysis. According to the method, a comprehensive interpretable feature analysis method is realized, and the analysis accuracy is improved.

Description

Technical Field

[0001] The present application relates to the technical field of embryo quality assessment, and in particular to a trophoblast cell analysis method, device, equipment and storage medium. Background Art

[0002] Currently in the field of assisted reproduction, the assessment of embryo development potential has become a very important part of the entire assisted reproductive process; assessing embryo development potential not only helps to understand the survival and developmental capacity of the embryo, but also provides key basis for clinical decision-making, thereby improving the success rate of assisted reproductive technology.

[0003] At present, embryo quality is often evaluated based on a consensus system, mainly by counting the number of trophoblast cells located on the equatorial plane of the blastocyst, evaluating the developmental morphology of the blastocyst, and thus determining the developmental quality of the embryo.

[0004] However, the above evaluation indicators are relatively simple, and due to the characteristics of the blastocyst's own structure, the statistical results of the number of trophoblast cells are less accurate, resulting in poor accuracy of the embryo quality analysis results. Summary of the invention

[0005] The purpose of the present application is to provide a trophoblast cell analysis method, device, equipment and storage medium to address the deficiencies in the above-mentioned prior art, so as to improve the accuracy of trophoblast cell feature extraction and analysis, thereby improving the accuracy of embryonic development result evaluation.

[0006] To achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0007] In a first aspect, the present application provides a method for analyzing trophoblast cells, comprising:

[0008] Acquire multi-focal plane images of the target blastocyst, with each focal plane separated by a preset distance;

[0009] Performing image fusion on the multi-focal plane images based on a pre-trained image fusion model to generate a first fused image and a second fused image corresponding to the target blastocyst;

[0010] Performing subject area segmentation on the first fused image and the second fused image respectively to obtain a first target image and a second target image;

[0011] Using a pre-trained image segmentation model, segment the trophoblast cells in the first target image and the second target image respectively to obtain a first mask image and a second mask image;

[0012] According to the first mask image and the second mask image, a plurality of characteristic parameters of the trophoblast cells are extracted, and based on the characteristic parameters, the correlation between the characteristic parameters and the developmental results of the embryo is analyzed and obtained.

[0013] Optionally, before performing image fusion on the multi-focal plane images based on a pre-trained image fusion model to generate a first fused image and a second fused image corresponding to the target blastocyst, the method further includes:

[0014] Filling the color channels of each focal plane image to obtain a first color space image corresponding to each focal plane image;

[0015] Performing color model conversion on the first color space image corresponding to each focal plane image to obtain a second color space image corresponding to each focal plane image;

[0016] Dividing the focal plane images into a first image set and a second image set according to a positional relationship of the focal plane images;

[0017] Generate first input data according to color information of a specified color channel in a second color space image corresponding to each focal plane image in the first image set;

[0018] The second input data is generated according to the color information of the designated color channel in the second color space image corresponding to each focal plane image in the second image set.

[0019] Optionally, performing image fusion on the multi-focal plane images based on a pre-trained image fusion model to generate a first fused image and a second fused image corresponding to the target blastocyst includes:

[0020] The first input data is input into the image fusion model, the image fusion model processes the first input data to generate first tensor data, and the first tensor data is normalized to obtain the first target image; and the second input data is input into the image fusion model, the image fusion model processes the second input data to generate second tensor data, and the second tensor data is normalized to obtain the second target image.

[0021] Optionally, the using a pre-trained image segmentation model to segment the trophoblast cells in the first target image and the second target image respectively to obtain a first mask image and a second mask image includes:

[0022] The first target image is input into the image segmentation model, and the image segmentation model identifies and segments the trophoblast cells in the first target image to obtain the first mask image; and the second target image is input into the image segmentation model, and the image segmentation model identifies and segments the trophoblast cells in the second target image to obtain the second mask image.

[0023] Optionally, the training process of the image fusion model includes:

[0024] Collecting a first training sample data set, the first training sample data set includes: multiple groups of input data, each group of input data is generated by color information of a specified color channel of multiple adjacent focal plane images of a sample blastocyst; each group of input data uses the central focal plane image of the corresponding sample blastocyst as label information;

[0025] The image fusion model is trained using the first training sample data set.

[0026] Optionally, the adopting the first training sample data set to train the image fusion model includes:

[0027] Inputting the first training sample data set into an initial image fusion model, processing each group of input data by the initial image fusion model to generate tensor data corresponding to each group of input data, and normalizing each tensor data to obtain a predicted fusion image corresponding to each group of input data;

[0028] Calculating loss information of the initial image fusion model according to the predicted fusion images corresponding to each group of input data and the label information corresponding to each group of input data;

[0029] The network parameters of the initial image fusion model are optimized according to the loss information until the loss information meets a preset condition, thereby obtaining the image fusion model.

[0030] Optionally, the training process of the image segmentation model includes:

[0031] Collect multi-focal plane images of multiple sample blastocysts;

[0032] Annotating the trophoblast cells in each focal plane image of each sample blastocyst, and generating a second training sample data set according to the annotation results;

[0033] The image segmentation model is trained using the second training sample data set.

[0034] Optionally, the labeling of trophoblast cells in each focal plane image of each sample blastocyst includes:

[0035] Configure the cell recognition type and segmentation channel properties of the initial image segmentation model;

[0036] Using the initial image segmentation model, according to the pre-measured diameter of a standard trophoblast cell, a single trophoblast cell in each focal plane image of each sample blastocyst is identified and segmented;

[0037] According to the segmentation result of the single trophoblast cell in each focal plane image of each sample blastocyst, the single trophoblast cell in each focal plane image of each sample blastocyst is labeled.

[0038] Optionally, the labeling of the single trophoblast cells in each focal plane image of each sample blastocyst according to the segmentation result of the single trophoblast cells in each focal plane image of each sample blastocyst includes:

[0039] Pre-labeling the single trophoblast cells in each focal plane image of each sample blastocyst according to the segmentation results of the single trophoblast cells in each focal plane image of each sample blastocyst;

[0040] According to the morphological properties of standard trophoblast cells, the pre-labeled results of each single trophoblast cell are corrected to obtain the target labeling results of the single trophoblast cell in each focal plane image of each sample blastocyst.

[0041] Optionally, extracting multiple characteristic parameters of trophoblast cells according to the first mask image and the second mask image includes:

[0042] According to the first mask image and the second mask image, the trophoblast cells in the target blastocyst are analyzed for morphological characteristics, and multiple characteristic parameters of the trophoblast cells are extracted. The multiple characteristic parameters include at least: the number of trophoblast cells, the area of ​​a single trophoblast cell, the circumference of a single trophoblast cell, the first axis length and the second axis length of a single trophoblast cell, and the roundness of a single trophoblast cell.

[0043] In a second aspect, an embodiment of the present application further provides a trophoblast cell analysis device, including: a collection module, a processing module and an analysis module;

[0044] The acquisition module is used to acquire multi-focal plane images of the target blastocyst, with each focal plane being separated by a preset distance;

[0045] The processing module is used to perform image fusion on the multi-focal plane images based on a pre-trained image fusion model to generate a first fused image and a second fused image corresponding to the target blastocyst;

[0046] The processing module is used to perform subject area segmentation on the first fused image and the second fused image respectively to obtain a first target image and a second target image;

[0047] The processing module is used to use a pre-trained image segmentation model to segment the trophoblast cells in the first target image and the second target image respectively to obtain a first mask image and a second mask image;

[0048] The analysis module is used to extract multiple characteristic parameters of trophoblast cells according to the first mask image and the second mask image, and based on each characteristic parameter, analyze and obtain the correlation between each characteristic parameter and the development result of the embryo.

[0049] Optionally, the processing module is further used to perform color channel filling on each focal plane image to obtain a first color space image corresponding to each focal plane image;

[0050] Performing color model conversion on the first color space image corresponding to each focal plane image to obtain a second color space image corresponding to each focal plane image;

[0051] Dividing the focal plane images into a first image set and a second image set according to a positional relationship of the focal plane images;

[0052] Generate first input data according to color information of a specified color channel in a second color space image corresponding to each focal plane image in the first image set;

[0053] The second input data is generated according to the color information of the designated color channel in the second color space image corresponding to each focal plane image in the second image set.

[0054] Optionally, the processing module is specifically used to input the first input data into the image fusion model, have the image fusion model process the first input data to generate first tensor data, and normalize the first tensor data to obtain the first target image; and input the second input data into the image fusion model, have the image fusion model process the second input data to generate second tensor data, and normalize the second tensor data to obtain the second target image.

[0055] Optionally, the processing module is specifically used to input the first target image into the image segmentation model, and the image segmentation model identifies and segments the trophoblast cells in the first target image to obtain the first mask image; and input the second target image into the image segmentation model, and the image segmentation model identifies and segments the trophoblast cells in the second target image to obtain the second mask image.

[0056] Optionally, it also includes: a training module;

[0057] The training module is used to collect a first training sample data set, wherein the first training sample data set includes: multiple groups of input data, each group of input data is generated by color information of a specified color channel of multiple adjacent focal plane images of a sample blastocyst; each group of input data uses the central focal plane image of the corresponding sample blastocyst as label information;

[0058] The image fusion model is trained using the first training sample data set.

[0059] Optionally, the training module is specifically used to input the first training sample data set into an initial image fusion model, and the initial image fusion model processes each group of input data to generate tensor data corresponding to each group of input data, and normalizes each tensor data to obtain a predicted fusion image corresponding to each group of input data;

[0060] Calculating loss information of the initial image fusion model according to the predicted fusion images corresponding to each group of input data and the label information corresponding to each group of input data;

[0061] The network parameters of the initial image fusion model are optimized according to the loss information until the loss information meets a preset condition, thereby obtaining the image fusion model.

[0062] Optionally, the training module is further used to collect multi-focal plane images of multiple sample blastocysts;

[0063] Annotating the trophoblast cells in each focal plane image of each sample blastocyst, and generating a second training sample data set according to the annotation results;

[0064] The image segmentation model is trained using the second training sample data set.

[0065] Optionally, the training module is specifically used to configure the cell recognition type and segmentation channel attributes of the initial image segmentation model;

[0066] Using the initial image segmentation model, according to the pre-measured diameter of a standard trophoblast cell, a single trophoblast cell in each focal plane image of each sample blastocyst is identified and segmented;

[0067] According to the segmentation result of the single trophoblast cell in each focal plane image of each sample blastocyst, the single trophoblast cell in each focal plane image of each sample blastocyst is labeled.

[0068] Optionally, the training module is specifically used to pre-label the single trophoblast cells in each focal plane image of each sample blastocyst according to the segmentation result of the single trophoblast cells in each focal plane image of each sample blastocyst;

[0069] According to the morphological properties of standard trophoblast cells, the pre-labeled results of each single trophoblast cell are corrected to obtain the target labeling results of the single trophoblast cell in each focal plane image of each sample blastocyst.

[0070] Optionally, the analysis module is specifically used to perform morphological characteristic analysis on the trophoblast cells in the target blastocyst according to the first mask image and the second mask image, and extract multiple characteristic parameters of the trophoblast cells, wherein the multiple characteristic parameters include at least: the number of trophoblast cells, the area of ​​a single trophoblast cell, the circumference of a single trophoblast cell, the first axis length and the second axis length of a single trophoblast cell, and the roundness of a single trophoblast cell.

[0071] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the storage medium communicate through the bus, and the processor executes the machine-readable instructions to implement the trophoblast cell analysis method provided in the first aspect.

[0072] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the trophoblast cell analysis method provided in the first aspect is executed.

[0073] The beneficial effects of this application are:

[0074] The present application provides a trophoblast cell analysis method, device, equipment and storage medium, including: collecting multi-focal plane images of a target blastocyst, with each focal plane separated by a preset distance; fusing the multi-focal plane images based on a pre-trained image fusion model to generate a first fused image and a second fused image corresponding to the target blastocyst; segmenting the main area of ​​the first fused image and the second fused image respectively to obtain a first target image and a second target image; using a pre-trained image segmentation model to segment the trophoblast cells in the first target image and the second target image respectively to obtain a first mask image and a second mask image; extracting multiple characteristic parameters of the trophoblast cells based on the first mask image and the second mask image, and analyzing and obtaining the correlation between each characteristic parameter and the developmental results of the embryo based on each characteristic parameter. This method collects multi-focal plane images of the blastocyst and fuses them to generate a fused blastocyst image. Then, the trophoblast cell mask image is extracted from the fused blastocyst image through image segmentation technology, and then the trophoblast cell morphological characteristics are extracted and analyzed based on the extracted trophoblast cell mask image, so as to analyze and obtain the trophoblast morphological characteristics related to the clinical outcome of the embryo, and provide a new analysis method and evaluation index for the evaluation of embryos in the blastocyst stage. Among them, the acquired multi-focal plane images can be fused into two complete blastocyst images through image fusion technology, which facilitates the observation and analysis of the complete texture of the trophoblast cells. Based on the image segmentation technology, the trophoblast cell mask image can be extracted from the generated fused image, thereby realizing the automated extraction and analysis of the morphological characteristics of the trophoblast cells, realizing a comprehensive and explainable trophoblast cell feature analysis method, and avoiding the problem of inaccurate analysis results caused by subjective difference evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0076] Figure 1 A schematic diagram of a trophoblast cell analysis method provided in an embodiment of the present application;

[0077] Figure 2 A schematic diagram of a blastocyst structure provided in an embodiment of the present application;

[0078] Figure 3 A schematic diagram of another trophoblast cell analysis method provided in an embodiment of the present application;

[0079] Figure 4A schematic diagram of a fused image provided in an embodiment of the present application;

[0080] Figure 5 A schematic diagram of comparison before and after segmentation of a main area provided in an embodiment of the present application;

[0081] Figure 6 A schematic diagram of an image enhancement processing result provided in an embodiment of the present application;

[0082] Figure 7 A schematic diagram of a segmentation result provided in an embodiment of the present application;

[0083] Figure 8 Another schematic diagram of segmentation results provided in an embodiment of the present application;

[0084] Fig. 9 A schematic diagram of a process for analyzing trophoblast cells provided in an embodiment of the present application;

[0085] Fig.10 A schematic diagram of another trophoblast cell analysis method provided in an embodiment of the present application;

[0086] Fig.11 A schematic diagram of another trophoblast cell analysis method provided in an embodiment of the present application;

[0087] Fig.12 A schematic diagram of a process for analyzing trophoblast cells provided in an embodiment of the present application;

[0088] Fig.13 A schematic diagram of another trophoblast cell analysis method provided in an embodiment of the present application;

[0089] Fig.14 A schematic diagram of trophoblast cells provided in an embodiment of the present application;

[0090] Fig.15 A schematic diagram of a trophoblast cell analysis device provided in an embodiment of the present application;

[0091] Fig.16 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0092] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0093] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0094] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0095] First, let’s briefly explain the relevant background involved in this plan:

[0096] Currently in the field of assisted reproduction, time-difference incubators are widely used in embryo culture. In the time-difference culture system, embryology obtains multi-focal plane image data of embryos through time-difference imaging technology and uses it to observe and select well-developed embryos for transplantation or freezing. The evaluation of embryo development potential has become a very important part of the entire assisted reproductive process. Although a lot of consensus on embryo evaluation has been formed in the rapid development of assisted reproductive technology, which can greatly improve the success of embryo transplantation and reduce the heavy observation and evaluation tasks of embryologists, for embryologists, the observation and evaluation of embryos is still a complicated process. Therefore, there are still many problems to be solved in realizing the automation process of embryo evaluation.

[0097] In the existing and common blastocyst stage embryo assessment methods, embryologists will evaluate the blastocyst according to the degree of expansion of the blastocyst, the size, shape and density of the inner cell mass, the number of trophoblast cells on the equatorial plane of the blastocyst and the uniformity of their distribution to determine whether the embryo is usable. The evaluation process requires rich experience and professional embryological knowledge, and the evaluation results often diverge due to individual subjective differences. Therefore, the evaluation of embryos requires objective and quantifiable indicators as the basis for evaluation. In addition, there are various potential factors in the embryonic development process that may determine whether the embryo can be born alive. These potential factors are usually reflected in the morphological characteristics of the embryonic development process. How to obtain as many potential characteristics related to the quality of embryonic development as possible through embryo images has become the main direction of the current development of embryology and ART (Artificial Reproductive Technology) technology.

[0098] Based on this, this scheme provides a trophoblast cell analysis method, which collects multi-focal plane images of blastocysts for image fusion to generate a fused blastocyst image, and then extracts a trophoblast cell mask image from the fused blastocyst image through image segmentation technology, thereby extracting and analyzing the morphological characteristics of trophoblast cells based on the extracted trophoblast cell mask image, so as to analyze and obtain the trophoblast morphological characteristics related to the clinical outcome of the embryo, and provide a new analysis method and evaluation index for the evaluation of embryos in the blastocyst stage. Among them, the acquired multi-focal plane images can be fused into two complete blastocyst images through image fusion technology, which facilitates the observation and analysis of the complete texture of trophoblast cells. Based on image segmentation technology, a mask image of a single trophoblast cell can be extracted from the generated fused image, thereby realizing the automated extraction and analysis of the morphological characteristics of trophoblast cells, realizing a comprehensive and explainable trophoblast cell feature analysis method, and avoiding the problem of inaccurate analysis results caused by subjective difference evaluation.

[0099] Figure 1 The flowchart of a trophoblast cell analysis method provided in the embodiment of the present application is shown below; the execution subject of the method can be a computer device such as a terminal or a server. Figure 1 As shown, the method may include:

[0100] S101, acquiring multi-focal plane images of a target blastocyst, with each focal plane being spaced by a preset distance.

[0101] Optionally, multi-focal plane images of the target blastocyst can be collected through a time-difference incubator. The time-difference incubator is mainly composed of two systems: a culture system and a time-difference imaging system. The culture system provides a stable development environment for the embryo, while the time-difference imaging system records the development process of the embryo by "snapshotting". Most time-difference imaging systems are based on HMC (Hoffman Modulation Contrast) imaging systems. Their advantage is that they can obtain clear "bright field light slice" data with a three-dimensional effect. Each layer of slice data contains the cell information of the current depth of field interval.

[0102] In one feasible manner, 11 focal plane images of the target blastocyst can be collected by the time-difference imaging system of the time-difference incubator, wherein the F0 focal plane (central focus, i.e., the equatorial plane) image can be collected first, and based on the F0 focal plane image, a focal plane image is collected every 15um above the F0 focal plane, for a total of 5 focal plane images, for example: the collected focal plane images are F15, F30, F45, F60, and F75; a focal plane image is collected every 15um below the F0 focal plane, for a total of 5 focal plane images, for example: the collected focal plane images are F-15, F-30, F-45, F-60, and F-75; then, including the 0 focal plane image, a total of 11 focal plane images are collected.

[0103] Of course, in practical applications, the multi-focal plane images collected are not limited to 11 focal planes, and more focal planes can be collected, and the collection interval is not limited to 15um, which can be adjusted adaptively according to needs. The more focal plane images collected, the more detailed textures and clearer the fused blastocyst image will be.

[0104] S102, performing image fusion on the multi-focal plane images based on a pre-trained image fusion model to generate a first fused image and a second fused image corresponding to the target blastocyst.

[0105] Optionally, a pre-trained image fusion model may be used to perform fusion processing on the collected multi-focal plane images to generate a first fused image and a second fused image corresponding to the target blastocyst.

[0106] It is worth noting that in the present embodiment, when performing image fusion, two fusion processes may be performed separately, and the F0 focal plane image and the focal plane images above the F0 focal plane may be fused to generate a first fused image, that is, to obtain the upper half image of the target blastocyst multi-focal plane (also referred to as Up_Map); and then the F0 focal plane image and the focal plane images below the F0 focal plane may be fused to generate a second fused image, that is, to obtain the lower half image of the target blastocyst multi-focal plane (also referred to as Down_Map).

[0107] Optionally, by fusing the first fused image and the second fused image respectively, it is more helpful to identify and extract trophoblast cells from different angles, thereby improving the completeness and accuracy of extracting trophoblast cell characteristics.

[0108] S103 , performing subject area segmentation on the first fused image and the second fused image respectively to obtain a first target image and a second target image.

[0109] In some embodiments, the generated first fused image and the second fused image may both be blastocyst images containing a background image, and the background image may interfere with the blastocyst image and the extraction of trophoblast cells. Therefore, the main area segmentation may be performed on the first fused image and the second fused image, respectively, to remove the background image from the first fused image, segment the blastocyst image, and obtain the first target image, and to remove the background image from the second fused image, segment the blastocyst image, and obtain the second target image.

[0110] Optionally, the main region of the blastocyst can be segmented by a Unet++ semantic segmentation network. The main region refers to the blastocyst region.

[0111] The Unet++ network retains the advantages of the Unet model and adds more paths and depth to improve the accuracy and robustness of image segmentation. Its network structure mainly includes the following key parts:

[0112] Encoder: used to extract image features. In Unet++, the encoder usually contains multiple convolutional layers and pooling layers to gradually extract high-level semantic features in the image.

[0113] Decoder: Generates segmentation results based on the features extracted by the encoder. The decoder usually contains multiple upsampling layers and convolutional layers to gradually restore the resolution of the image and generate the final segmentation map.

[0114] Dense skip connections: Unet++ introduces dense skip connections between the encoder and decoder. These connections allow information to be passed directly from the encoder to the decoder to help retain more detail information. Through the concatenation operation, feature maps at different levels can be fused together, thereby improving the accuracy of segmentation.

[0115] Redesigned skip path: Compared with the traditional Unet, Unet++ redesigns the skip path to make up for the semantic difference between the encoder and decoder subpaths. This helps reduce the loss of information during transmission and improves the accuracy of segmentation results.

[0116] Deep Supervision: Unet++ introduces a deep supervision mechanism that guides the training of the network by adding supervision information at different layers of the network. This helps ensure that the network can learn more effective feature representations during training, thereby improving segmentation performance.

[0117] S104 , using a pre-trained image segmentation model to segment trophoblast cells in the first target image and the second target image respectively, to obtain a first mask image and a second mask image.

[0118] In some embodiments, based on the obtained first target image and the second target image, image enhancement processing may be performed on the first target image and the second target image respectively to improve the clarity of the image so that the texture details of the blastocyst image are more clearly visible.

[0119] Next, the first target image and the second target image after image enhancement processing are segmented using a pre-trained image segmentation model to segment the first target image to obtain a first mask image, and to segment the second target image to obtain a second mask image, respectively. The purpose of the segmentation processing is to identify and segment the trophoblast cells from the first target image and the second target image, so that the obtained first mask image can refer to the mask image of the trophoblast cells in the first target image, and the obtained second mask image can refer to the mask image of the trophoblast cells in the second target image.

[0120] S105. Extract multiple characteristic parameters of trophoblast cells according to the first mask image and the second mask image, and analyze and obtain the correlation between each characteristic parameter and the development result of the embryo based on each characteristic parameter.

[0121] Optionally, based on the first mask image and the second mask image, characteristic analysis and extraction of trophoblast cells can be performed, and multiple characteristic parameters of trophoblast cells can be extracted, so as to analyze the correlation between various characteristic parameters and the developmental outcomes of the embryo. Subsequently, in the study of embryonic development, the developmental outcomes of the embryo can be analyzed based on the correlation between the multiple characteristic parameters of trophoblast cells and the developmental outcomes of the embryo.

[0122] In summary, the trophoblast cell analysis method provided in this embodiment includes: acquiring multi-focal plane images of the target blastocyst, with each focal plane being spaced by a preset distance; performing image fusion on the multi-focal plane images based on a pre-trained image fusion model to generate a first fused image and a second fused image corresponding to the target blastocyst; performing main area segmentation on the first fused image and the second fused image, respectively, to obtain a first target image and a second target image; using a pre-trained image segmentation model, respectively segmenting the trophoblast cells in the first target image and the second target image, to obtain a first mask image and a second mask image; according to the first mask image and the second mask image, extracting multiple characteristic parameters of the trophoblast cells, and based on each characteristic parameter, analyzing and obtaining the correlation between each characteristic parameter and the developmental results of the embryo. This method collects multi-focal plane images of the blastocyst and fuses them to generate a fused blastocyst image. Then, the trophoblast cell mask image is extracted from the fused blastocyst image through image segmentation technology, and then the trophoblast cell morphological characteristics are extracted and analyzed based on the extracted trophoblast cell mask image, so as to analyze and obtain the trophoblast morphological characteristics related to the clinical outcome of the embryo, and provide a new analysis method and evaluation index for the evaluation of embryos in the blastocyst stage. Among them, the acquired multi-focal plane images can be fused into two complete blastocyst images through image fusion technology, which facilitates the observation and analysis of the complete texture of the trophoblast cells. Based on the image segmentation technology, the trophoblast cell mask image can be extracted from the generated fused image, thereby realizing the automated extraction and analysis of the morphological characteristics of the trophoblast cells, realizing a comprehensive and explainable trophoblast cell feature analysis method, and avoiding the problem of inaccurate analysis results caused by subjective difference evaluation.

[0123] Figure 2 A schematic diagram of a blastocyst structure provided in the present application example. Figure 2 The a in the figure shows the structure of the blastocyst. Figure 2 The image b in the figure shows the F0 focal plane image of the blastocyst (ie, the equatorial plane of the blastocyst).

[0124] like Figure 2 As shown in the figure, the blastocyst is mainly composed of three parts: ① inner cell mass; ② trophoblast layer; ③ cavity. Figure 2As shown in b, due to the three-dimensional structure of the blastocyst and the filling and expansion of the cystic fluid, the trophoblast cells in the central area on the defocused plane cannot be observed. In order to solve this problem, this solution fuses these data. The fusion process is equivalent to overlapping and combining multiple layers of focal plane data, overlapping and merging the image texture features of multiple layers, and then enhancing the fused image to enhance the cell edge, making the individual cells clearer and more recognizable. The trophoblast cells in the image are then labeled and segmented through the image segmentation model. According to the segmentation results, multiple characteristic parameters of the trophoblast cells can be extracted and analyzed. For example: the number of trophoblast cells, the area of ​​trophoblast cells, the roundness, etc.

[0125] Figure 3 A schematic flow chart of another trophoblast cell analysis method provided in an embodiment of the present application; optionally, in step S102, before performing image fusion on multi-focal plane images based on a pre-trained image fusion model to generate a first fused image and a second fused image corresponding to a target blastocyst, the method may further include:

[0126] S201 . Fill color channels of each focal plane image to obtain a first color space image corresponding to each focal plane image.

[0127] In some embodiments, before performing image fusion based on the acquired multi-focal plane images, color channel filling may be performed on each focal plane image to convert each focal plane image into a first color space image. The first color space image here may refer to an RGB (red, green, and blue) image.

[0128] S202: Perform color model conversion on the first color space image corresponding to each focal plane image to obtain a second color space image corresponding to each focal plane image.

[0129] Then, a color model conversion is performed on each RGB image obtained by the conversion, and each RGB image is converted into a second color space image. The second color space image here may refer to a YCrCb image.

[0130] The Y channel represents the grayscale value (grayscale characteristics); Cr reflects the difference between the red part of the RGB input signal and the brightness value of the RGB signal; and Cb reflects the difference between the blue part of the RGB input signal and the brightness value of the RGB signal.

[0131] S203 . Divide the focal plane images into a first image set and a second image set according to the positional relationship of the focal plane images.

[0132] Next, according to the positional relationship of the collected focal plane images, with F0 focal plane as the center, the F0 focal plane image and multiple adjacent focal plane images above the F0 focal plane are divided into a first image set, and the F0 focal plane image and multiple adjacent focal plane images below the F0 focal plane are divided into a second image set.

[0133] For example, assuming that the 11 focal plane images collected are: F75, F60, F45, F30, F15, F0, F-15, F-30, F-45, F-60, and F-75. Then, F75, F60, F45, F30, F15, and F0 can be divided into the first image set, and F0, F-15, F-30, F-45, F-60, and F-75 can be divided into the second image set.

[0134] S204 . Generate first input data according to color information of a designated color channel in the second color space image corresponding to each focal plane image in the first image set.

[0135] After each focal plane image is converted into a YCrCb image, the image may be separated into color channels, thereby extracting Y channel data of each focal plane image.

[0136] For the first image set, the Y channel data of each focal plane image in the first image set may be connected into 6*1*800*800 tensor data as the first input data.

[0137] S205 . Generate second input data according to color information of a designated color channel in the second color space image corresponding to each focal plane image in the second image set.

[0138] Similarly, for the second image set, the Y channel data of each focal plane image in the second image set may be connected into 6*1*800*800 tensor data as the second input data.

[0139] Optionally, in step S102, image fusion is performed on multi-focal plane images based on a pre-trained image fusion model to generate a first fused image and a second fused image corresponding to the target blastocyst, which may include: inputting first input data into the image fusion model, processing the first input data by the image fusion model to generate first tensor data, and normalizing the first tensor data to obtain a first target image; and inputting second input data into the image fusion model, processing the second input data by the image fusion model to generate second tensor data, and normalizing the second tensor data to obtain a second target image.

[0140] In some embodiments, the first input data generated above can be input into the image fusion model, and the image fusion model processes the first input data to generate 1*800*800 tensor data, that is, the first tensor data, and normalizes the first tensor data to obtain the fused first target image. In other words, the image fusion model fuses the input data of the 6 Y channels to obtain a tensor data, and normalizes the tensor data to obtain a grayscale image as the first target image.

[0141] Similarly, after the second input data is input into the image fusion model, the image fusion model uses the same processing method to obtain a second target image.

[0142] Figure 4 A schematic diagram of a fused image provided in an embodiment of the present application, such as Figure 4 The a in the figure shows the second fusion image obtained after fusion (that is, the Down_Map mentioned above, the lower half of the blastocyst multi-focal plane). Figure 4 The image b in the figure shows the focal plane sequence images (F0~F-75). Figure 4 The focal plane images in b are fused to obtain Figure 4 The second fused image is shown in a in FIG. Figure 4 The image d in the figure shows the first fusion image obtained after fusion (i.e. the Up_Map mentioned above, the upper part of the multi-focal plane of the blastocyst). Figure 4 The image c in the figure shows the focal plane sequence (F75~F0). Figure 4 The focal plane images in c are fused to obtain Figure 4 The first fused image shown in d.

[0143] Figure 5 This is a schematic diagram of a comparison before and after the segmentation of a main area provided in an embodiment of the present application. Figure 5 The a in the figure shows the original image at the F0 focal plane. Figure 5 Figure b shows the image obtained after extracting the main area through semantic segmentation. It can be seen that by segmenting and extracting the main area, background interference can be effectively removed, which is conducive to subsequent feature extraction and analysis.

[0144] For the first target image and the second target image obtained after the main area is segmented, in this embodiment, two image enhancement processing methods can be used to improve the clarity of the images.

[0145] First, we can use the RLTV (Richardson-lucy with tolar variation, deconvolution enhancement) technology. RLTV is a deconvolution algorithm that uses iterative deblurring. RL (Richardson-lucy) is an iterative deblurring algorithm based on Bayesian theory independently proposed by Richardson and Lucy. This algorithm can effectively restore images. Its basic principle is to compare the differences between the reconstructed image and the target image, and update the reconstructed image based on these differences. Tolar Variation is a metric used to evaluate image quality and is used to control the convergence of the iterative process in the RL algorithm. Tolar Variation is a standard deviation used to measure the magnitude of the change in intensity of adjacent pixels in an image. By using Tolar Variation as a convergence indicator, it can be ensured that the iterative process will not stop before reaching a certain threshold, thereby avoiding falling into a local minimum.

[0146] After RLTV processing, the second image enhancement is performed. The second enhancement is achieved through the Equalize AdaotiveHitogram (adaptive histogram equalization) technology, which is a method used to improve image contrast, especially when the histogram of the image presents a skewed distribution. Due to the particularity of HMC imaging, the image will appear bright on the left and dark on the right, and the image brightness distribution is uneven. Therefore, it is necessary to use adaptive histogram equalization to improve the overall visual effect of the image and make the details in the image clearer.

[0147] Figure 6 A schematic diagram of an image enhancement processing result provided in an embodiment of the present application, Figure 6 Figure a shows the original blastocyst schematic diagram obtained after segmentation of the main area. Figure 6 The b in the figure shows the image after being enhanced by RLTV. Figure 6 Figure c shows the image enhanced by adaptive histogram equalization. It can be seen that after two image enhancement processes, the details of the blastocyst image are clearer.

[0148] Optionally, in step S104, a pre-trained image segmentation model is used to segment the trophoblast cells in the first target image and the second target image, respectively, to obtain a first mask image and a second mask image, which may include: inputting the first target image into the image segmentation model, and using the image segmentation model to identify and segment the trophoblast cells in the first target image to obtain a first mask image; and inputting the second target image into the image segmentation model, and using the image segmentation model to identify and segment the trophoblast cells in the second target image to obtain a second mask image.

[0149] In some embodiments, a pre-trained image segmentation model can be used to annotate and segment the trophoblast cells in the first target image after image enhancement processing and the second target image after image enhancement processing, respectively, to obtain the first mask image and the second mask image, as well as the horizontal and vertical gradient maps of the topological structure generated based on the mask images.

[0150] Optionally, the image segmentation model in this embodiment can be obtained based on cellpose (cell segmentation) network training. Cellpose is a universal cell and cell nucleus segmentation network that only requires a small amount of data for training and adaptation to achieve better recognition effects. Cellpose also has topological XY gradient map reasoning, which can assist in determining the boundaries between cells and is more conducive to obtaining the texture features of trophoblast cells.

[0151] The image segmentation model can be used to identify and segment the trophoblast cells in the first target image after image enhancement and the second target image after image enhancement to obtain the first mask image and the second mask image. In addition, the horizontal and vertical gradient maps of the topological structure generated based on the mask image can also be obtained.

[0152] Figure 7 A schematic diagram of a segmentation result provided in an embodiment of the present application is shown in FIG. Figure 7 a represents the first target image after image enhancement processing. Figure 7 b represents the first mask image obtained after segmentation by the image segmentation model. Figure 7 In the figure, c represents the gradient map in the horizontal and vertical directions of the topological structure generated based on the first mask image.

[0153] Figure 8 Another segmentation result schematic diagram provided in the embodiment of the present application is as follows: Figure 8 a represents the second target image after image enhancement processing. Figure 8 b represents the second mask image obtained after segmentation by the image segmentation model. Figure 8 In the figure, c represents the gradient map in the horizontal and vertical directions of the topological structure generated based on the second mask image.

[0154] Fig. 9 A schematic flow chart of another trophoblast cell analysis method provided in an embodiment of the present application; Optionally, the training process of the above-mentioned image fusion model may include:

[0155] S301. Collect a first training sample data set, where the first training sample data set includes: multiple groups of input data, each group of input data is generated by color information of a specified color channel of multiple adjacent focal plane images of a sample blastocyst; each group of input data uses the central focal plane image of the corresponding sample blastocyst as label information.

[0156] In some embodiments, multi-focus sample plane images of multiple sample blastocysts may be collected to generate multiple sets of input data to constitute a first training sample data set to train an image fusion model.

[0157] Among them, each sample blastocyst can collect multi-focal sample plane images. According to the above-mentioned collection method, a focal plane image can be collected at an interval of 15um, and 11 focal sample plane images can be collected for each sample blastocyst.

[0158] Similarly, by dividing the multi-focus sample plane images of each sample blastocyst into two groups, performing channel filling and color model conversion on each group, and finally fusing the Y channel data of each focus sample plane image in each group, two sets of input data corresponding to each sample blastocyst can be generated, so that multiple sample blastocysts can correspond to multiple sets of input data, forming the first training sample data set.

[0159] It is worth noting that the two sets of input data corresponding to each sample blastocyst use the F0 focal plane image of the sample blastocyst as label information, because the F0 focal plane (also known as the central focal plane) is the best clear focal plane.

[0160] S302: Use the first training sample data set to train and obtain an image fusion model.

[0161] Optionally, each group of input data in the first training sample data set is used as training data, and an image fusion model can be trained.

[0162] Fig.10 A schematic flow chart of another trophoblast cell analysis method provided in an embodiment of the present application; Optionally, in step S302, using a first training sample data set to train an image fusion model may include:

[0163] S401. Input the first training sample data set into the initial image fusion model, and use the initial image fusion model to process each group of input data, generate tensor data corresponding to each group of input data, and normalize each tensor data to obtain a predicted fusion image corresponding to each group of input data.

[0164] Optionally, each group of input data in the first training sample data set may be input into an initial image fusion model, and the initial image fusion model may perform fusion processing on each group of input data to obtain a predicted fusion image corresponding to each group of input data.

[0165] Among them, the initial image fusion model and the image fusion model obtained by final training have the same network structure, but the network parameters of the initial image fusion model have not been trained. After training with the first training sample data set, the network parameters reach the optimal level, and the obtained image fusion model can be used to achieve the required functions.

[0166] S402: Calculate loss information of the initial image fusion model according to the predicted fusion image corresponding to each group of input data and the label information corresponding to each group of input data.

[0167] Optionally, for any set of input data, the loss information generated by the set of input data can be calculated based on the predicted fusion image corresponding to the set of input data and the label information corresponding to the set of input data, so as to obtain the loss information of the initial image fusion model by taking the weighted sum of the loss information generated by each set of input data.

[0168] For example, the calculation of loss information generated by any set of input data can be implemented using the following formula:

[0169]

[0170] Where M represents the length of the image, N represents the width of the image, and I H represents the predicted fusion image, I W Represents label information (i.e., F0 focal plane image).

[0171] S403. Optimize the network parameters of the initial image fusion model according to the loss information until the loss information meets the preset conditions, thereby obtaining the image fusion model.

[0172] Optionally, the network parameters of the initial image fusion model can be optimized through multiple rounds of iterations by the above-mentioned method of calculating the loss until the loss information meets the loss accuracy, or the number of iterations meets the preset number of times, then the iteration is stopped, and the network parameters when the iteration is stopped are used as the target network parameters to obtain the image fusion model.

[0173] Fig.11 A schematic flow chart of another trophoblast cell analysis method provided in an embodiment of the present application; optionally, the training process of the above-mentioned image segmentation model may include:

[0174] S501, collecting multi-focal plane images of multiple sample blastocysts.

[0175] In this embodiment, cellpose is retrained to generate an image segmentation model, and the trophoblast cells in the blastocyst image are segmented by the image segmentation model.

[0176] Optionally, multi-focal plane images of the sample blastocyst may be acquired. The multi-focal plane images of the sample blastocyst may reuse the images used in the aforementioned training of the image fusion model, or may be generated by reselecting new sample blastocyst images.

[0177] S502: Label the trophoblast cells in each focal plane image of each sample blastocyst, and generate a second training sample data set according to the labeling results.

[0178] The trophoblast cells in each focal plane image of each sample blastocyst collected may be annotated, and each annotated focal plane image may be used as a training sample to form a second training sample data set.

[0179] S503: Use the second training sample data set to train and obtain an image segmentation model.

[0180] The image segmentation model can be trained using the second training sample data set.

[0181] Fig.12 A schematic flow chart of another trophoblast cell analysis method provided in an embodiment of the present application; Optionally, in step S502, labeling the trophoblast cells in each focal plane image of each sample blastocyst may include:

[0182] S601, configuring the cell recognition type and segmentation channel attributes of the initial image segmentation model.

[0183] In this embodiment, trophoblast cells may be labeled based on the pre-labeling function provided by cellpose to improve labeling efficiency.

[0184] Before pre-labeling, you can configure the cell recognition type and segmentation channel of the initial image segmentation model. For example, the model type of cellpose is configured as cyto2, which means that the cell type segmented by this model is closer to trophoblast cells, and the segmentation channel is configured as gray, which means that it supports the processing of grayscale data. At the same time, measure and set the diameter of a single standard trophoblast cell as a hyperparameter of the model.

[0185] S602 , using the initial image segmentation model, according to the pre-measured diameter of the standard trophoblast cells, identify and segment the single trophoblast cells in each focal plane image of each sample blastocyst.

[0186] Optionally, based on the configured initial image segmentation model, the cell recognition capability of the model itself can be relied upon, and the measured diameter of the standard trophoblast cells can be used as prior knowledge to identify and segment individual trophoblast cells in each focal plane image.

[0187] S603 . Labeling the single trophoblast cells in each focal plane image of each sample blastocyst according to the segmentation result of the single trophoblast cells in each focal plane image of each sample blastocyst.

[0188] Based on the segmentation result of the single trophoblast cell in each focal plane image, the single trophoblast cell in each focal plane image may be labeled to obtain a labeling result.

[0189] Fig.13 A flow chart of another trophoblast cell analysis method provided in an embodiment of the present application; Optionally, in step S603, labeling the single trophoblast cells in each focal plane image of each sample blastocyst according to the segmentation result of the single trophoblast cells in each focal plane image of each sample blastocyst may include:

[0190] S701 . Pre-labeling the single trophoblast cells in each focal plane image of each sample blastocyst according to the segmentation result of the single trophoblast cells in each focal plane image of each sample blastocyst.

[0191] In some embodiments, based on the segmentation results, individual trophoblast cells in each focal plane image of each sample blastocyst may be pre-labeled.

[0192] S702. According to the morphological properties of standard trophoblast cells, the pre-labeled results of each single trophoblast cell are corrected to obtain the target labeling results of the single trophoblast cells in each focal plane image of each sample blastocyst.

[0193] When labeling trophoblast cells, extra attention should be paid. There are interactions between individual trophoblast cells. Not all cells can be seen with clear boundaries during labeling. Therefore, the pre-labeling results can be further corrected manually.

[0194] Optionally, the trophoblast cells with unclear boundaries may be supplementally labeled according to the morphological properties of the standard trophoblast cells, for example, according to the property information such as the shape and size of the standard trophoblast cells, to obtain the target labeling result.

[0195] For example, some trophoblast cells in the focal plane image can only be seen as semicircles, but trophoblast cells are actually round, so the semicircles need to be completed according to the shape and size of standard trophoblast cells and closely connected with adjacent cells to obtain new annotations.

[0196] Fig.14 A schematic diagram of trophoblast cells provided in an embodiment of the present application is shown in FIG. Fig.14 The a in the figure shows the image before labeling. Fig.14 The image b shows the labeled trophoblast cells.

[0197] Optionally, in step S105, extracting multiple characteristic parameters of trophoblast cells according to the first mask image and the second mask image may include: performing morphological characteristic analysis on the trophoblast cells in the target blastocyst according to the first mask image and the second mask image, and extracting multiple characteristic parameters of the trophoblast cells, the multiple characteristic parameters including at least: the number of trophoblast cells, the area of ​​a single trophoblast cell, the circumference of a single trophoblast cell, the first axis length and the second axis length of a single trophoblast cell, and the roundness of a single trophoblast cell.

[0198] In some embodiments, based on the results of image segmentation, multiple characteristic parameters of trophoblast cells can be extracted, including but not limited to: the number of trophoblast cells (TE_Num), the area of ​​a single trophoblast cell (Area), the perimeter of a single trophoblast cell (Perimeter), the first axis length of a single trophoblast cell (i.e., the short axis, minor_length), and the second axis length of a single trophoblast cell (i.e., the long axis, major_length).

[0199] In addition, based on the above-mentioned multiple characteristic parameters, multiple extended characteristic parameters can be further calculated, including but not limited to: the ratio of the short axis to the long axis of a single trophoblast cell (TE_LDR) and the roundness of a single trophoblast cell (TE_Roundness).

[0200] Among them, the roundness TE_Roundness of a single trophoblast cell can be calculated using the following formula:

[0201]

[0202] Among them, L i is the circumference of trophoblast cells, S i is the area of ​​trophoblast cells.

[0203] The short-to-long axis ratio TE_LDR of a single trophoblast cell can be calculated using the following formula:

[0204]

[0205] Through the chi-square test of each characteristic parameter, it can be analyzed that the number of trophoblast cells, the mean and standard deviation of the perimeter of trophoblast cells, the area and standard deviation of trophoblast cells are correlated with live birth, pregnancy and gestational age. The mean perimeter of trophoblast cells, the number of trophoblast cells, the mean area of ​​trophoblast cells and other characteristics are all related to the ploidy outcome.

[0206] In summary, the trophoblast cell analysis method provided in this embodiment includes: acquiring multi-focal plane images of the target blastocyst, with each focal plane being spaced by a preset distance; performing image fusion on the multi-focal plane images based on a pre-trained image fusion model to generate a first fused image and a second fused image corresponding to the target blastocyst; performing main area segmentation on the first fused image and the second fused image, respectively, to obtain a first target image and a second target image; using a pre-trained image segmentation model, respectively segmenting the trophoblast cells in the first target image and the second target image, to obtain a first mask image and a second mask image; according to the first mask image and the second mask image, extracting multiple characteristic parameters of the trophoblast cells, and based on each characteristic parameter, analyzing and obtaining the correlation between each characteristic parameter and the developmental results of the embryo. This method collects multi-focal plane images of the blastocyst and fuses them to generate a fused blastocyst image. Then, the trophoblast cell mask image is extracted from the fused blastocyst image through image segmentation technology, and then the trophoblast cell morphological characteristics are extracted and analyzed based on the extracted trophoblast cell mask image, so as to analyze and obtain the trophoblast morphological characteristics related to the clinical outcome of the embryo, and provide a new analysis method and evaluation index for the evaluation of embryos in the blastocyst stage. Among them, the acquired multi-focal plane images can be fused into two complete blastocyst images through image fusion technology, which facilitates the observation and analysis of the complete texture of the trophoblast cells. Based on the image segmentation technology, the trophoblast cell mask image can be extracted from the generated fused image, thereby realizing the automated extraction and analysis of the morphological characteristics of the trophoblast cells, realizing a comprehensive and explainable trophoblast cell feature analysis method, and avoiding the problem of inaccurate analysis results caused by subjective difference evaluation.

[0207] The following describes the apparatus, equipment, storage medium, etc. used to perform the trophoblast cell analysis method provided in the present application. The specific implementation process and technical effects are described above and will not be repeated below.

[0208] Fig.15 A schematic diagram of a trophoblast cell analysis device provided in an embodiment of the present application, the functions implemented by the trophoblast cell analysis device correspond to the steps performed by the above method. The device can be understood as the above server, or the processor of the server, or can be understood as a component independent of the above server or processor that implements the functions of the present application under the control of the server, such as Fig.15 As shown, the device may include: a collection module 150, a processing module 151 and an analysis module 152;

[0209] An acquisition module 150 is used to acquire multi-focal plane images of the target blastocyst, with each focal plane being separated by a preset distance;

[0210] A processing module 151 is used to perform image fusion on the multi-focal plane images based on a pre-trained image fusion model to generate a first fused image and a second fused image corresponding to the target blastocyst;

[0211] The processing module 151 is used to perform subject area segmentation on the first fused image and the second fused image respectively to obtain a first target image and a second target image;

[0212] A processing module 151 is used to segment the trophoblast cells in the first target image and the second target image using a pre-trained image segmentation model to obtain a first mask image and a second mask image;

[0213] The analysis module 152 is used to extract multiple characteristic parameters of the trophoblast cells according to the first mask image and the second mask image, and analyze and obtain the correlation between each characteristic parameter and the development result of the embryo based on each characteristic parameter.

[0214] Optionally, the processing module 151 is further configured to perform color channel filling on each focal plane image to obtain a first color space image corresponding to each focal plane image;

[0215] Performing color model conversion on the first color space image corresponding to each focal plane image to obtain a second color space image corresponding to each focal plane image;

[0216] Dividing the focal plane images into a first image set and a second image set according to a positional relationship of the focal plane images;

[0217] Generate first input data according to color information of a specified color channel in a second color space image corresponding to each focal plane image in the first image set;

[0218] The second input data is generated according to the color information of the designated color channel in the second color space image corresponding to each focal plane image in the second image set.

[0219] Optionally, the processing module 151 is specifically used to input the first input data into the image fusion model, have the image fusion model process the first input data to generate first tensor data, and normalize the first tensor data to obtain a first target image; and input the second input data into the image fusion model, have the image fusion model process the second input data to generate second tensor data, and normalize the second tensor data to obtain a second target image.

[0220] Optionally, the processing module 151 is specifically used to input the first target image into an image segmentation model, and the image segmentation model identifies and segments the trophoblast cells in the first target image to obtain a first mask image; and input the second target image into the image segmentation model, and the image segmentation model identifies and segments the trophoblast cells in the second target image to obtain a second mask image.

[0221] Optionally, it also includes: a training module;

[0222] A training module is used to collect a first training sample data set, the first training sample data set comprising: multiple groups of input data, each group of input data is generated by color information of a specified color channel of multiple adjacent focal plane images of a sample blastocyst; each group of input data uses the central focal plane image of the corresponding sample blastocyst as label information;

[0223] The first training sample data set is used to train an image fusion model.

[0224] Optionally, the training module is specifically used to input the first training sample data set into the initial image fusion model, and the initial image fusion model processes each group of input data to generate tensor data corresponding to each group of input data, and normalizes each tensor data to obtain a predicted fusion image corresponding to each group of input data;

[0225] Calculate the loss information of the initial image fusion model according to the predicted fusion image corresponding to each group of input data and the label information corresponding to each group of input data;

[0226] The network parameters of the initial image fusion model are optimized according to the loss information until the loss information meets the preset conditions to obtain the image fusion model.

[0227] Optionally, the training module is further used to acquire multi-focal plane images of multiple sample blastocysts;

[0228] Annotating the trophoblast cells in each focal plane image of each sample blastocyst, and generating a second training sample data set according to the annotation results;

[0229] The second training sample data set is used to train an image segmentation model.

[0230] Optionally, a training module is specifically used to configure the cell recognition type and segmentation channel attributes of the initial image segmentation model;

[0231] The initial image segmentation model identifies and segments individual trophoblast cells in each focal plane image of each sample blastocyst according to the pre-measured diameter of a standard trophoblast cell;

[0232] According to the segmentation result of the single trophoblast cell in each focal plane image of each sample blastocyst, the single trophoblast cell in each focal plane image of each sample blastocyst is labeled.

[0233] Optionally, the training module is specifically used to pre-label the single trophoblast cells in each focal plane image of each sample blastocyst according to the segmentation result of the single trophoblast cells in each focal plane image of each sample blastocyst;

[0234] According to the morphological properties of standard trophoblast cells, the pre-labeled results of each single trophoblast cell are corrected to obtain the target labeling results of the single trophoblast cell in each focal plane image of each sample blastocyst.

[0235] Optionally, the analysis module 152 is specifically used to perform morphological characteristic analysis on the trophoblast cells in the target blastocyst according to the first mask image and the second mask image, and extract multiple characteristic parameters of the trophoblast cells, the multiple characteristic parameters including at least: the number of trophoblast cells, the area of ​​a single trophoblast cell, the circumference of a single trophoblast cell, the first axis length and the second axis length of a single trophoblast cell, and the roundness of a single trophoblast cell.

[0236] The above-mentioned device is used to execute the method provided by the aforementioned embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0237] The above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more digital singnal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0238] The above modules can be connected or communicated with each other via a wired connection or a wireless connection. The wired connection may include a metal cable, an optical cable, a hybrid cable, etc., or any combination thereof. The wireless connection may include a connection in the form of a LAN, a WAN, Bluetooth, a ZigBee, or NFC, or any combination thereof. Two or more modules may be combined into a single module, and any one module may be divided into two or more units. Those skilled in the art will clearly understand that for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application.

[0239] Fig.16 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, which may be a computing device with data processing capabilities.

[0240] The device may include: a processor 801 and a storage medium 802 .

[0241] The storage medium 802 is used to store programs, and the processor 801 calls the programs stored in the storage medium 802 to execute the above method embodiment. The specific implementation method and technical effect are similar and will not be repeated here.

[0242] Among them, the storage medium 802 stores program code, and when the program code is executed by the processor 801, the processor 801 executes various steps in the trophoblast cell analysis method according to various exemplary embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0243] Processor 801 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor (DigitalSignal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as a hardware processor to be executed, or a combination of hardware and software modules in the processor can be executed.

[0244] Storage medium 802, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules. Storage medium may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type storage medium, random access storage medium (Random Access Memory, RAM), static random access storage medium (Static Random Access Memory, SRAM), programmable read-only storage medium (Programmable Read Only Memory, PROM), read-only storage medium (Read Only Memory, ROM), electrically erasable programmable read-only storage medium (Electrically Erasable Programmable Read-Only Memory, EEPROM), magnetic storage medium, disk, optical disk, etc. Storage medium is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The storage medium 802 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0245] Optionally, the present application also provides a program product, such as a computer-readable storage medium, including a program, which is used to execute the above method embodiment when executed by a processor.

[0246] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0247] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0248] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0249] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only storage medium (English: Read-Only Memory, referred to as: ROM), random access storage medium (English: Random Access Memory, referred to as: RAM), disk or optical disk and other media that can store program code.

Claims

1. A method for analyzing trophoblast cells, characterized in that: include: Acquire multi-focal plane images of the target blastocyst, with each focal plane separated by a preset distance; Performing image fusion on the multi-focal plane images based on a pre-trained image fusion model to generate a first fused image and a second fused image corresponding to the target blastocyst; Performing subject area segmentation on the first fused image and the second fused image respectively to obtain a first target image and a second target image; Using a pre-trained image segmentation model, segment the trophoblast cells in the first target image and the second target image respectively to obtain a first mask image and a second mask image; According to the first mask image and the second mask image, a plurality of characteristic parameters of the trophoblast cells are extracted, and based on the characteristic parameters, the correlation between the characteristic parameters and the developmental results of the embryo is analyzed and obtained.

2. The method according to claim 1, characterized in that Before performing image fusion on the multi-focal plane images based on the pre-trained image fusion model to generate the first fused image and the second fused image corresponding to the target blastocyst, the method further includes: Filling the color channels of each focal plane image to obtain a first color space image corresponding to each focal plane image; Performing color model conversion on the first color space image corresponding to each focal plane image to obtain a second color space image corresponding to each focal plane image; Dividing the focal plane images into a first image set and a second image set according to a positional relationship of the focal plane images; Generate first input data according to color information of a specified color channel in a second color space image corresponding to each focal plane image in the first image set; The second input data is generated according to the color information of the designated color channel in the second color space image corresponding to each focal plane image in the second image set.

3. The method according to claim 2, characterized in that The method of performing image fusion on the multi-focal plane images based on a pre-trained image fusion model to generate a first fused image and a second fused image corresponding to the target blastocyst comprises: The first input data is input into the image fusion model, the image fusion model processes the first input data to generate first tensor data, and the first tensor data is normalized to obtain the first target image; and the second input data is input into the image fusion model, the image fusion model processes the second input data to generate second tensor data, and the second tensor data is normalized to obtain the second target image.

4. The method according to claim 1, characterized in that: The method adopts a pre-trained image segmentation model to segment the trophoblast cells in the first target image and the second target image respectively to obtain a first mask image and a second mask image, including: The first target image is input into the image segmentation model, and the image segmentation model identifies and segments the trophoblast cells in the first target image to obtain the first mask image; and the second target image is input into the image segmentation model, and the image segmentation model identifies and segments the trophoblast cells in the second target image to obtain the second mask image.

5. The method according to any one of claims 1 to 4, characterized in that: The training process of the image fusion model includes: Collecting a first training sample data set, the first training sample data set includes: multiple groups of input data, each group of input data is generated by color information of a specified color channel of multiple adjacent focal plane images of a sample blastocyst; each group of input data uses the central focal plane image of the corresponding sample blastocyst as label information; The image fusion model is trained using the first training sample data set.

6. The method according to claim 5, characterized in that The adopting the first training sample data set to train the image fusion model includes: Inputting the first training sample data set into an initial image fusion model, processing each group of input data by the initial image fusion model to generate tensor data corresponding to each group of input data, and normalizing each tensor data to obtain a predicted fusion image corresponding to each group of input data; Calculating loss information of the initial image fusion model according to the predicted fusion images corresponding to each group of input data and the label information corresponding to each group of input data; The network parameters of the initial image fusion model are optimized according to the loss information until the loss information meets a preset condition, thereby obtaining the image fusion model.

7. The method according to any one of claims 1 to 4, characterized in that: The training process of the image segmentation model includes: Collect multi-focal plane images of multiple sample blastocysts; Annotating the trophoblast cells in each focal plane image of each sample blastocyst, and generating a second training sample data set according to the annotation results; The image segmentation model is trained using the second training sample data set.

8. The method according to claim 7, characterized in that The labeling of the trophoblast cells in each focal plane image of each sample blastocyst includes: Configure the cell recognition type and segmentation channel properties of the initial image segmentation model; Using the initial image segmentation model, according to the pre-measured diameter of a standard trophoblast cell, a single trophoblast cell in each focal plane image of each sample blastocyst is identified and segmented; According to the segmentation result of the single trophoblast cell in each focal plane image of each sample blastocyst, the single trophoblast cell in each focal plane image of each sample blastocyst is labeled.

9. The method according to claim 8, characterized in that The step of labeling the single trophoblast cells in each focal plane image of each sample blastocyst according to the segmentation result of the single trophoblast cells in each focal plane image of each sample blastocyst comprises: Pre-labeling a single trophoblast cell in each focal plane image of each sample blastocyst according to the segmentation result of the trophoblast cells in each focal plane image of each sample blastocyst; According to the morphological properties of standard trophoblast cells, the pre-labeled results of each single trophoblast cell are corrected to obtain the target labeling results of the single trophoblast cell in each focal plane image of each sample blastocyst.

10. The method according to claim 1, characterized in that The step of extracting multiple characteristic parameters of trophoblast cells according to the first mask image and the second mask image includes: According to the first mask image and the second mask image, the trophoblast cells in the target blastocyst are analyzed for morphological characteristics, and multiple characteristic parameters of the trophoblast cells are extracted. The multiple characteristic parameters include at least: the number of trophoblast cells, the area of ​​a single trophoblast cell, the circumference of a single trophoblast cell, the first axis length and the second axis length of a single trophoblast cell, and the roundness of a single trophoblast cell.

11. A trophoblast cell analysis device, characterized in that: include: Acquisition module, processing module and analysis module; The acquisition module is used to acquire multi-focal plane images of the target blastocyst, with each focal plane being separated by a preset distance; The processing module is used to perform image fusion on the multi-focal plane images based on a pre-trained image fusion model to generate a first fused image and a second fused image corresponding to the target blastocyst; The processing module is used to perform subject area segmentation on the first fused image and the second fused image respectively to obtain a first target image and a second target image; The processing module is used to use a pre-trained image segmentation model to segment the trophoblast cells in the first target image and the second target image respectively to obtain a first mask image and a second mask image; The analysis module is used to extract multiple characteristic parameters of trophoblast cells according to the first mask image and the second mask image, and based on each characteristic parameter, analyze and obtain the correlation between each characteristic parameter and the development result of the embryo.

12. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor, and when the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the program instructions to implement the trophoblast cell analysis method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the method for analyzing trophoblast cells according to any one of claims 1 to 10 is implemented.

Images