Mouse embryo development evaluation method based on AI deep learning algorithm

Abstract

The application provides a mouse embryo development evaluation method based on an AI deep learning algorithm, morphological feature targets in an image are accurately and quickly recognized through a PP-YOLO algorithm, then rapid gestational age prediction is carried out through the morphological features, and the gestational age presented by a given embryo can be accurately to 0.25-0.5 days. The method solves the problems of large workload, low accuracy, great difference in knowledge reserves of judges, diversity of embryo morphology, poor experimental parallelism and the like in the current mouse embryo development evaluation.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of stereomicroscope image automatic identification, and particularly relates to a mouse embryo development evaluation method based on morphology and artificial intelligence deep learning algorithm. BACKGROUND

[0002] Rodent embryo related experiments have important value in toxicology and teratology research, and morphological grading technology is essential for these fields. After implantation, mouse embryos begin three germ layer differentiation from blastocyst, and important morphological features such as neural tube and heart begin to occur and develop in a specific spatiotemporal order of the embryo itself. The morphological changes are huge and rapid, the dynamics and molecular mechanisms involved are complex, and the morphological index evaluation is highly subjective, far from meeting the evaluation of post-implantation embryo development level and the prediction of a given embryo gestational age, especially the morphological evaluation accuracy of key development nodes.

[0003] At present, the evaluation of embryo development status is still mainly manual, and the evaluation results of embryo development show strong subjectivity due to the differences in the knowledge background of evaluators and the individual diversity of embryos. The parallelism of scientific research is poor, and the statistical analysis of embryo development indicators in the later stage is time-consuming and laborious, so it is urgent to build an automatic grading system with strong objectivity, high accuracy and rapidness. SUMMARY

[0004] The purpose of the application is to provide a mouse embryo development evaluation method based on AI deep learning algorithm, to solve the problems of large workload and low accuracy of current mouse embryo development evaluation, and the problems of great difference in knowledge reserve background of judges, diversity of embryo morphology, poor experimental parallelism, etc.

[0005] The technical scheme of the application is: a mouse embryo development evaluation method based on AI deep learning algorithm, comprising the following steps,

[0006] S1: selecting mouse embryo images during E8.0-10.0, marking morphological features in the embryo image and making name labels, training using a PP-YOLOv4 model to obtain an identification model that can identify morphological features in the image;

[0007] S2: The morphological features in the S1 embryo image are individually cropped, and a growth evaluation file corresponding to each morphological feature picture is made. Each morphological feature in the growth evaluation file corresponds to an embryo development stage through a development score. The embryo images selected in S1 during E8.0-10.0 are divided into six embryo development stages according to E8.0-8.5.0, E8.5-8.75, E8.75-9.0, E9.0-9.25, E9.25-9.5, and E9.5-10.0;

[0008] S3: The mouse embryo picture to be identified is input into the recognition model. The original picture outputs the rectangular box and name label data corresponding to the morphological features, and the morphological features in the picture are cropped. The cropped picture is input into the image scoring model to obtain the development score and development stage corresponding to each morphological feature of the picture to be identified,

[0009] S4: A gestational age verification model is established. The gestational age evaluation weight of each morphological feature in each development stage in the growth evaluation file of S2 is divided from high to low into a gestational age prediction morphological feature, a gestational age prediction reference morphological feature, and a non-reference morphological feature.

[0010] S5: Gestational age prediction. The output results in S3 are counted. When the scores of each morphological feature are concentrated in one development stage by more than half, the development stage is the predicted gestational age of the picture to be identified. When the output results are evenly distributed in two development stages, the corresponding development stages in the gestational age verification model of S4 are compared. The gestational age prediction reference morphological features and the non-reference morphological features with lower weights are removed. The predicted gestational age is the one with the most corresponding morphological features. The morphological features that are inconsistent with the scores of the gestational age prediction morphological features in the predicted gestational age stage are marked as suspicious morphological features.

[0011] The suspicious morphological features indicate that the morphological features may have developmental retardation or malformation.

[0012] The morphological features include the yolk sac, somite, allantois, forebrain, midbrain, hindbrain, eye vesicle, ear vesicle, olfactory vesicle, heart, forelimb, hindlimb, tail neural tube, branchial arch, upper jaw, and lower jaw.

[0013] The yolk sac and allantois are excluded from the weight division of each morphological feature in step S4. The yolk sac and allantois are used as separate evaluation features.

[0014] The morphological features of the E8.0-8.5 embryo development stage include seven items, namely, the yolk sac, allantois, somite, heart, tail neural tube, forebrain, and hindbrain. The gestational age prediction morphological features in this stage are the heart, tail neural tube, forebrain, and hindbrain, and the gestational age prediction reference morphological feature is the somite.

[0015] The morphological characteristics of the E 8.5-8.75 embryonic development stage include 10 items, respectively, yolk sac, allantois, somites, heart, tail neural tube, forebrain, midbrain, hindbrain, eye vesicle, ear vesicle, the gestational age prediction morphological characteristics are heart, tail neural tube, midbrain, forebrain, eye vesicle, ear vesicle, the gestational age prediction reference morphological characteristic is somite, and no morphological characteristic hindbrain is referenced;

[0016] The morphological characteristics of the E 8.75-9.0 embryonic development stage include 13 items, respectively, yolk sac, allantois, somites, heart, tail neural tube, forebrain, midbrain, hindbrain, eye vesicle, ear vesicle, branchial arch, maxilla, mandible, the gestational age prediction morphological characteristics are midbrain, hindbrain, ear vesicle, eye vesicle, maxilla and branchial arch, the gestational age prediction reference morphological characteristic is somite, tail neural tube and forebrain, and no morphological characteristic heart is referenced;

[0017] The morphological characteristics of the E 9.0-9.25 embryonic development stage include 15 items, respectively, yolk sac, allantois, somites, heart, tail neural tube, forebrain, midbrain, hindbrain, eye vesicle, ear vesicle, olfactory vesicle, forelimb, branchial arch, maxilla and mandible, the gestational age prediction morphological characteristics are heart, forebrain, eye vesicle, ear vesicle, forelimb, branchial arch and maxilla, the gestational age prediction reference morphological characteristic is tail neural tube, hindbrain, midbrain, olfactory vesicle, mandible and somite, and no morphological characteristic is referenced;

[0018] The morphological characteristics of the E 9.25-9.5 embryonic development stage include 16 items, respectively, yolk sac, allantois, somites, heart, tail neural tube, forebrain, midbrain, hindbrain, eye vesicle, ear vesicle, olfactory vesicle, forelimb, branchial arch, maxilla, mandible and hindlimb, the gestational age prediction morphological characteristics are heart, tail neural tube, forebrain, midbrain, hindbrain, eye vesicle, ear vesicle, branchial arch and hindlimb, the gestational age prediction reference morphological characteristic is somite, olfactory vesicle and mandible, and no morphological characteristic is referenced;

[0019] The morphological characteristics of the E 9.5-10.0 embryonic development stage include 16 items, respectively, yolk sac, allantois, somites, heart, tail neural tube, forebrain, midbrain, hindbrain, eye vesicle, ear vesicle, olfactory vesicle, forelimb, hindlimb, branchial arch, maxilla and mandible, the gestational age prediction morphological characteristics are heart, forebrain, midbrain, hindbrain, eye vesicle, ear vesicle, olfactory vesicle, forelimb, branchial arch and maxilla, the gestational age prediction reference morphological characteristic is somite and mandible, and no morphological characteristic is referenced;

[0020] Step S1 target recognition, PP-YOLOv4 algorithm is used for yolk sac, allantois, forebrain, midbrain, hindbrain, eye vesicle, ear vesicle, olfactory vesicle, heart, forelimb, hindlimb, body rotation, tail neural tube, branchial arch, upper jaw and lower jaw, and the model is trained under the ubuntu system using the NVIDIA Tesla V100 GPU, and IOU and mAP are used as evaluation indexes of recognition accuracy.

[0021] In step S1, after the suspected area obtained by the PP-YOLOv4 algorithm preliminary screening result, the image is first edge enhanced and sharpened, then the gray value of the embryonic morphological feature is grayed and binarized, the embryonic morphological feature is separated from the background, and then the contour is extracted through the Canny operator, after obtaining the clear contour, random points are taken on the contour for fitting, if the fitting area overlaps with the fitting area of the morphological feature after binarization by more than 95%, it can be judged that the shape of the morphological feature in the area conforms to the development shape characteristics of the morphological feature, and the morphological feature can be determined.

[0022] The mouse embryo development evaluation method based on the AI deep learning algorithm provided by the application can accurately and quickly identify the morphological feature target in the image through the PP-YOLO algorithm, and then quickly predict the gestational age through the morphological feature. The gestational age presented by the given embryo can be accurately predicted to 0.25-0.5 days. The method has high efficiency and high accuracy in gestational age prediction. After the gestational age is predicted, the development status of each morphological feature of the mouse embryo and the judgment of the overall development level of the embryo are quickly analyzed, which provides a data basis for later data statistical analysis. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 The flowchart of the application.

[0024] Figure 2 The morphological feature labeling diagram of the application.

[0025] Figure 3 The growth evaluation file of the application.

[0026] Figure 4 The matrix frame labeling diagram of step S3 of the application.

[0027] Figure 5 The weight division diagram in step S4 of the application.

[0028] Figure 6 The gestational age prediction flowchart of the first case of the embodiment of the application.

[0029] Figure 7 The gestational age prediction flowchart of the second case of the embodiment of the application. DETAILED DESCRIPTION

[0030] In order to more clearly illustrate the technical means and beneficial effects of the present application, the present application will be described in detail below in conjunction with the drawings. The flow of the present embodiment is shown in Figure 1 .

[0031] S1: select 300 mouse embryo images during E8.0-10.0 as test images, use PP-YOLOv4 model for training, identify all morphological features from mouse embryo images, morphological features include yolk sac, allantois, forebrain, midbrain, hindbrain, eye vesicle, ear vesicle, olfactory vesicle, heart, forelimb, hindlimb, posterior neuropore, branchial arch, somite, maxilla and mandible, morphological features detected by PP-YOLOv4 model will be marked with a matrix frame in the image, use labelme software to label each measurable morphological feature of the image in the test image, the labeled image is shown in Figure 2 .

[0032] Change the brightness of each picture of the labeled image to 0.3 times, 0.5 times, 1.2 times and 1.6 times of the original picture respectively, and perform 90°, 180°, 270°, horizontal and mirror image flipping for data enhancement; divide the enhanced data set into 16 groups, and divide each group into training set and test set according to 7:3 respectively.

[0033] Use target detection task training, process the labeled image and name label file json one by one to form the training set, and apply the data to the neural convolution network training to obtain the image recognition model of each morphological feature.

[0034] S2: Morphological features in the labeled embryo image are cut out from the original image, and the image of each morphological feature and its corresponding growth evaluation file are separately used as training set data, and are trained through a convolutional neural network respectively. After training, an image scoring model for predicting gestational age of each morphological feature is obtained. Specifically, the PP-YOLOv4 algorithm is used, and the model is trained under the ubuntu system using the NVIDIA Tesla V100 GPU. The accuracy index calculation formula is IUO = the intersection over union of the predicted box and the real box. IOU and mAP (Mean Average Precision) are used as evaluation indexes of recognition accuracy. IOU is obtained by calculating the intersection over union of the real box and the predicted box, which reflects that the model can accurately locate the position of a certain morphological feature. When the IOU is higher than 50%, it is considered that the position of the box is correctly predicted. mAP is the AP (Average Precision) of each class, that is, the area under the PR curve of each class, and the average value of the AP values of all classes is taken, which reflects the accuracy performance of the model for scoring morphological features; the mAP value greater than 70% indicates high recognition accuracy.

[0035] The growth evaluation file is shown in Figure 3 Each morphological feature in the growth evaluation file corresponds to an embryo development stage through development score. The embryo images selected in S1 during E8.0-10.0 are divided into six embryo development stages according to E8.0-8.5.0, E8.5-8.75, E8.75-9.0, E9.0-9.25, E9.25-9.5, E9.5-10.0.

[0036] S3: The mouse embryo picture to be identified is input into the recognition model. The morphological features detected by the PP-YOLOv4 model are labeled in the image with a matrix box, and the fixed-point coordinates of the matrix box are output, as shown in Figure 4According to the vertex coordinates of the matrix frame, the morphological features are cut out from the original image and stored according to the categories. The specific identification method is as follows: after obtaining the clear suspected area screened by the PP-YOLOv4 algorithm, the image is first subjected to edge enhancement and sharpening, then the gray value of the embryo morphological feature is subjected to gray-scale and binary segmentation, and the embryo morphological feature is separated from the background, and then the Canny operator is used for contour extraction. After obtaining the clear contour, random points are taken on the contour for fitting. If the fitting area overlaps with the fitting area of the morphological feature after binary segmentation by more than a certain value, it can be determined that the shape feature in this area is the morphological feature. If the fitting area overlaps with the fitting area of the embryo morphological feature after binary segmentation by more than 95%, it can be determined that the shape of the morphological feature in this area conforms to the shape of the morphological feature, and the morphological feature can be determined. At the same time, the size threshold and the aspect ratio threshold are set, and the size of the rotated rectangle of the contour is effectively screened to filter out invalid areas, reduce the interference of invalid areas that are too large or valid areas that are too small on the identification of embryo morphological features, and improve the operation speed and identification efficiency.

[0037] S4: Establish a gestational age verification model. On the basis of the growth evaluation file in S2, the gestational age evaluation weight of each morphological feature in each development stage is divided. The division basis is the difficulty of achieving agreement between the morphological feature recognition degree and the evaluation of the embryo development stage. Specifically, in the present application, the embryo shape is greatly affected by the dissection operation, the view of the embryo is greatly affected by the shooting angle of the view in a specific development stage, or the development is rapid in the development stage, and the morphological performance is complex and diverse, which is difficult to form a unified evaluation standard. The morphological features are divided into gestational age prediction reference morphological features. The morphological features without obvious development and change in the development stage are divided into non-reference morphological features. The rest are divided into main development organs in the development stage as gestational age prediction morphological features, as shown in Figure 5 The gestational age prediction morphological features, the gestational age prediction reference morphological features, and the non-reference morphological features are divided from high to low. The yolk sac and the allantois are excluded during weight division. Since the yolk sac and the allantois involve the problem of whether the embryo extraction is complete, they are excluded. If necessary, they can be scored separately.

[0038] S5: Gestational age prediction. The output results in S3 are counted. There are two cases. Case one: when more than half of the scores of each morphological feature are concentrated in one development stage, as shown in Figure 6 There are 12 in the E9.5-10.0 development stage, two in the E9.25-9.5 development stage, and the E9.5-10.0 development stage is in an absolute advantage. Therefore, the development stage is the predicted gestational age of the picture to be identified. Case two: when the output results are evenly distributed in two gestational age stages, as shown in Figure 7As shown, the morphological characteristics at E9.5-10.0 and the morphological characteristics at E9.25-9.5 are both 6, at this time, the corresponding gestational age stages in the gestational age verification model of S4 are compared respectively, the gestational age prediction reference morphological characteristics with lower weights are removed, the morphological characteristics are not referenced, the more the number of gestational age prediction morphological characteristics consistent with the simulated gestational age, the higher the confidence, it can be seen that E9.5-10.0 has higher confidence, the development stage is taken as the gestational age prediction result, at the same time, the eye vesicle and the gill arch of the visual system in the output result of S3 are inconsistent with the gestational age prediction morphological characteristic score in the predicted gestational age stage, which is marked as a doubtful morphological characteristic, there may be developmental retardation or malformation, which needs to be evaluated by artificial.

[0039] In summary, the mouse embryo development evaluation method based on the AI deep learning algorithm of the present application is more intelligent and rapid than the traditional manual screening and scoring method, can meet the needs of embryonic development, and can judge the mouse gestational age, the development status of each morphological characteristic, and the overall development level of the embryo, further improve the judgment accuracy and speed, and provide a data basis for later data statistical analysis.

[0040] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for evaluating mouse embryonic development based on AI deep learning algorithms, comprising the following steps: S1: Select embryonic images of mice during the E8.0-10.0 period, mark the morphological features in the embryonic images and create name labels, and use the PP-YOLOv4 model for training to obtain a recognition model that can recognize the morphological features in the images. S2: Extract the morphological features from the embryo images in S1 separately, and create growth evaluation files corresponding to each morphological feature image. Train each file using a convolutional neural network to obtain an image scoring model. Each morphological feature in the growth evaluation file corresponds to an embryonic development stage through a developmental score. The embryo images selected in S1 during the period E8.0-10.0 are divided into six embryonic development stages: E8.0-8.5.0, E8.5-8.75, E8.75-9.0, E9.0-9.25, E9.25-9.5, and E9.5-10.

0. S3: Input the image of the mouse embryo to be identified into the recognition model. The original image outputs the bounding boxes and name labels corresponding to the morphological features, and takes screenshots of each morphological feature in the image. Input the screenshots into the image scoring model to obtain the developmental score and developmental stage corresponding to each morphological feature of the image to be identified. S4: Establish a gestational age verification model. Based on the morphological features of each developmental stage in the growth evaluation file of S2, divide the gestational age evaluation weights into morphological features for gestational age prediction, morphological features for gestational age prediction reference, and morphological features without reference, in descending order of high weight. S5: Gestational Age Prediction. The output results in S3 are statistically analyzed. When more than half of the scores of each morphological feature are concentrated in one developmental stage, that developmental stage is the predicted gestational age of the image to be identified. When the output results show that the scores of each morphological feature are evenly distributed in two developmental stages, they are compared with the corresponding developmental stages in the gestational age verification model in S4. Gestational age prediction reference morphological features and non-reference morphological features with lower weights are removed. The gestational age prediction morphological features with the largest number of matching results are the predicted gestational age. At the same time, morphological features whose output results in S3 are inconsistent with the gestational age prediction morphological feature scores within the predicted gestational age stage are marked as questionable morphological features.

2. The method for evaluating mouse embryonic development based on AI deep learning algorithm according to claim 1, characterized in that: The questionable morphological features indicate that the morphological features may indicate developmental delay or malformation.

3. The method for evaluating mouse embryonic development based on AI deep learning algorithm according to claim 1, characterized in that: The morphological features include the yolk sac, body rotation, allantois, forebrain, midbrain, hindbrain, eye vesicles, ear vesicles, olfactory vesicles, heart, forelimbs, hindlimbs, caudal nerve canal, gill arches, upper jaw, and lower jaw.

4. The mouse embryonic development evaluation method based on AI deep learning algorithm according to claim 1, characterized in that: In step S4, when assigning weights to morphological features for gestational age evaluation, the yolk sac and allantoic follicle are excluded, and the yolk sac and allantoic follicle are evaluated as separate features.

5. The method for evaluating mouse embryonic development based on AI deep learning algorithm according to claim 1, characterized in that: The morphological characteristics of the embryonic development stage E 8.0-8.5 include 7 items, namely yolk sac, allantois, rotation, heart, caudal nerve tube, forebrain, and hindbrain; the morphological characteristics for gestational age prediction at this stage are heart, caudal nerve tube, forebrain, and hindbrain, and the morphological characteristic for gestational age prediction is rotation. The morphological characteristics of the embryonic development stage E 8.5-8.75 include 10 items, namely yolk sac, allantois, rotation, heart, caudal nerve tube, forebrain, midbrain, hindbrain, eye vesicle, and ear vesicle. The morphological characteristics for gestational age prediction at this stage are heart, caudal nerve tube, midbrain, forebrain, eye vesicle, and ear vesicle. The morphological characteristic for gestational age prediction is rotation, and the morphological characteristic of hindbrain is not referenced. The morphological features of the embryonic development stage E 8.75-9.0 include 13 items, namely yolk sac, allantois, rotation, heart, caudal nerve tube, forebrain, midbrain, hindbrain, eye vesicles, ear vesicles, branchial arches, maxilla, and mandible. The morphological features for gestational age prediction at this stage are midbrain, hindbrain, ear vesicles, eye vesicles, maxilla, and branchial arches. The morphological features for gestational age prediction are rotation, caudal nerve tube, and forebrain, but not the morphological feature of heart. The morphological characteristics of the embryonic development stage E 9.0-9.25 include 15 items, namely yolk sac, allantois, rotation, heart, caudal nerve tube, forebrain, midbrain, hindbrain, eye vesicles, ear vesicles, olfactory vesicles, forelimbs, gill arches, maxilla, and mandible. The morphological characteristics for gestational age prediction at this stage are heart, forebrain, eye vesicles, ear vesicles, forelimbs, gill arches, and maxilla. The morphological characteristics for gestational age prediction reference are caudal nerve tube, hindbrain, midbrain, olfactory vesicles, mandible, and rotation. The morphological characteristics of the embryonic development stage E 9.25-9.5 include 16 items, namely yolk sac, allantois, rotation, heart, caudal nerve tube, forebrain, midbrain, hindbrain, eye vesicles, ear vesicles, olfactory vesicles, forelimbs, gill arches, maxilla, mandible, and hindlimbs. The morphological characteristics for gestational age prediction at this stage are heart, caudal nerve tube, forebrain, midbrain, hindbrain, eye vesicles, ear vesicles, gill arches, and hindlimbs. The morphological characteristics for gestational age prediction are rotation, olfactory vesicles, and mandible, while the morphological characteristics of maxilla and forelimbs are not considered. The morphological features of the embryonic development stage E 9.5-10.0 include 16 items, namely yolk sac, allantois, rotation, caudal nerve tube, heart, forebrain, midbrain, hindbrain, eye vesicles, ear vesicles, olfactory vesicles, forelimbs, hindlimbs, gill arches, maxilla, and mandible. The morphological features for predicting gestational age at this stage are heart, forebrain, midbrain, hindbrain, eye vesicles, ear vesicles, olfactory vesicles, forelimbs, gill arches, and maxilla. The morphological features for gestational age prediction are rotation and mandible, while the morphological features of caudal nerve tube and hindlimbs are not considered.

6. The method for evaluating mouse embryonic development based on AI deep learning algorithm according to claim 1, characterized in that: Step S1: Target identification. The PP-YOLOV4 model algorithm is used for yolk sac, allantois, forebrain, midbrain, hindbrain, eye vesicles, ear vesicles, olfactory vesicles, heart, forelimbs, hindlimbs, body rotation, caudal nerve canal, gill arches, maxilla, and mandible. The model is trained on an NVIDIA Tesla V100 GPU under the Unbuntu system, and IOU and mAP are used as evaluation indicators for identification accuracy.

7. The mouse embryonic development evaluation method based on AI deep learning algorithm according to claim 6, characterized in that: In step S1, after the suspected areas are obtained from the initial screening results of the PP-YOLO V4 algorithm, the image is first enhanced and sharpened at the edges. Then, grayscale and binarization are performed based on the color values ​​of the embryonic morphological features to separate the embryonic morphological features from the background. The contour is then extracted using the Canny operator. After obtaining a clear contour, points are randomly selected on the contour for fitting. If the fitted area overlaps with the fitted area of ​​the binarized morphological features by more than 95%, it can be determined that the shape of the morphological feature in this area conforms to the developmental shape characteristics of the morphological feature and can be identified as that morphological feature.

Images