A method for automatic segmentation and area calculation of in vitro fertilization embryo blastomere image
By combining deep learning and image enhancement, the problem of low accuracy caused by interference factors in embryo blastomere identification was solved, and accurate segmentation and area calculation of blastomere and embryo were achieved, meeting the high precision requirements of clinical practice.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-16
- Publication Date
- 2026-04-10
AI Technical Summary
Existing methods for identifying embryonic blastomeres cannot accurately outline and calculate areas when faced with interference from the zona pellucida, air bubbles, and fragments, differences in images taken from different devices, and overlapping blastomeres, thus failing to meet the high-precision requirements of clinical practice.
A deep learning-based target detection model was used to initially locate blastomeres and embryo candidate boxes. Combined with image enhancement processing and interactive image segmentation algorithms, blastomeres and embryo masks were obtained for accurate segmentation and area calculation.
It enables precise segmentation and outlining of blastomeres and embryos in complex environments, improving recognition accuracy and meeting clinical needs.
Smart Images
Figure CN116091421B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of embryo light microscopy image processing technology, and in particular to a method for automatic segmentation and area calculation of blastomere images of in vitro fertilized embryos. Background Technology
[0002] During embryo transfer, it is necessary to assess the quality of the embryo and blastomeres in the light micrographs of the embryo. To reduce the manual burden, researchers have developed a series of intelligent auxiliary detection technologies based on deep learning or machine learning. Currently, the identification of blastomeres in light micrographs of cleavage-stage embryos after in vitro fertilization mainly includes ellipse fitting methods based on machine learning and target detection methods based on deep learning.
[0003] In the area of machine learning:
[0004] Conaghan et al., in their paper "Improving embryo selection using a computer-automated time-lapse image analysis test plus day 3 morphology: results from a prospective multicenter trial," proposed a cell tracking software that can approximate the blastomeres of day 3 embryos with ellipses to assist doctors in diagnosis, increasing the specificity of doctors' diagnoses from 79.5% to 86.6%.
[0005] Patil et al. proposed a machine learning-based algorithm for detecting blastomeres on the second and third days of embryonic development in “Application of Vessel Enhancement Filtering for Automated Classification of Human In-Vitro Fertilized (IVF) Images”. The algorithm consists of four steps: boundary extraction, filtering, finding blastomeres, and verification.
[0006] In their paper "Ellipse detection on embryo image using modification of arc Particle Swarm Optimization (ArcPSO) based arc segment", Syulistyo et al. proposed a machine learning algorithm for detecting blastomeres on day 3 of an embryo. The algorithm consists of three steps: extracting line segments, filtering, and finding ellipses. It achieves an accuracy of 42% in the multi-blastomere detection task.
[0007] The methods proposed by Conaghan et al., Patil et al., and Syulistyo et al. all use circles or ellipses to fit blastomeres. However, blastomeres in the cleavage stage are not all centrally symmetrical ellipses in embryo light microscopy images, and there is interference from substances such as the zona pellucida, air bubbles, and debris. Therefore, these methods can only roughly estimate the number of blastomeres and cannot meet the requirements for accurate blastomere outlining and area calculation. Moreover, the above algorithms do not provide corresponding measures to address interference caused by differences in the proportion of the embryo to the whole image, contrast, light intensity, and signal-to-noise ratio between images taken from different devices and between different batches taken from the same device.
[0008] In deep learning:
[0009] In their paper "Methods, Systems, Equipment and Storage Media for Cell Identification in Embryo Light Microscopy Images", Wang Jianbo et al. proposed using the minimum tangent frame to detect the position and number of blastomeres. However, this method cannot perform edge segmentation and area statistics on blastomeres, and the minimum tangent frame is not necessarily strictly tangent to the blastomeres when they are highly overlapping.
[0010] The model proposed by He et al. in "Machine learning for automated cell segmentation in embryos" requires manual outlining as labels during training, and can only segment blastomeres in embryos with 4 blastomeres, with a counting accuracy of only 70%, which cannot meet clinical requirements.
[0011] Therefore, current identification and detection methods are insufficient for accurate blastomere outlining and area calculation due to interference from substances such as zona pellucida, air bubbles, and fragments, as well as interference caused by differences in the proportion of the embryo body to the whole image, contrast, light intensity, and signal-to-noise ratio between different devices and different batches of the same device, and the high degree of overlap of blastomeres. These factors result in low accuracy and failure to meet the high precision requirements of clinical practice. Summary of the Invention
[0012] The purpose of this invention is to provide a method for automatic segmentation and area calculation of blastomeres in in vitro fertilized embryo images, addressing the limitations of existing technologies. This method can accurately segment blastomeres and embryo images, achieve precise outlining and area calculation of blastomeres and embryos, and has high recognition accuracy.
[0013] To achieve the above objectives, the present invention adopts the following technical solution:
[0014] A method for automatic segmentation and area calculation of blastomeres in in vitro fertilized embryo images, comprising:
[0015] A deep learning-based object detection model was used to detect blastomeres in embryo light microscopy images and obtain blastomer candidate boxes.
[0016] By fusing the blastomere candidate frames, an embryo candidate frame is obtained;
[0017] Regions of interest are extracted based on the embryo candidate boxes;
[0018] Image enhancement processing is performed on the region of interest;
[0019] After image enhancement processing, the region of interest in the embryo light microscopy image is automatically segmented using an interactive image segmentation algorithm to obtain the blastomere mask and the embryo mask.
[0020] The blastome mask and the embryo mask are used to outline the blastome and the embryo respectively on the light microscopic image of the embryo.
[0021] Calculate the areas of the blastomere mask and the embryo mask to obtain the blastomere area and the embryo area.
[0022] In some embodiments, the image enhancement process includes the following steps:
[0023] Adjust the color levels of the region of interest;
[0024] Noise reduction is applied to the region of interest after color level adjustment;
[0025] Contrast enhancement is performed on the noise-reduced region of interest.
[0026] In some embodiments, the step of adjusting the color levels includes:
[0027] Convert the region of interest into an HSV channel;
[0028] Calculate the mean value of the H channel, denoted as h;
[0029] Calculate the mean of the V channel, denoted as v;
[0030] If a preset threshold T is set, then... If the color level adjustment is not performed, then the color level adjustment operation based on the color level adjustment formula will be executed; otherwise, the color level adjustment will be exited.
[0031] The formula for adjusting color levels is:
[0032]
[0033] Where I is the image data before color adjustment, I' is the image data after color adjustment, s, h, and m are constants, and satisfy s∈[0,170], h∈(s,255), m∈[1,3], and numbers less than 0 in Is are set to 0.
[0034] In some embodiments, the noise reduction step includes:
[0035] Construct a total variational noise reduction function and preset the noise reduction parameter range;
[0036] The optimal J-invariance method is used to iterate the input image within the noise reduction parameter range with a preset step size to obtain the optimal noise reduction parameters;
[0037] The input image is denoised using the optimal denoising parameters.
[0038] In some embodiments, the step of using a deep learning-based target detection model to detect blastomeres in embryonic light microscopy images and obtaining blastomere candidate boxes includes:
[0039] Read embryo light microscopy image data;
[0040] Perform neighborhood histogram equalization on the read data;
[0041] The processed image is input into a deep learning-based object detection model to detect blastomeres and obtain primary blastomer bounding boxes.
[0042] The primary blastomere rectangles are expanded using a preset bias value, and the vertex coordinates are recorded. If the expanded coordinates exceed the size of the original embryo light microscopy image, the corresponding original coordinates are retained, thereby obtaining the blastomere candidate boxes.
[0043] In some embodiments, the step of extracting the region of interest based on the embryo candidate box includes:
[0044] Using the center of the embryo candidate box as the center, the embryo candidate box is expanded according to the expansion formula to form an expanded candidate box, which is used as the region of interest, and the corresponding vertex coordinates are recorded.
[0045] The expansion formula is: L'=L*k
[0046] Where L is the side length of the embryo candidate box, L' is the side length of the expanded candidate box, k is the expansion coefficient, and k∈[1.2,2].
[0047] In some embodiments, the step of automatically segmenting the embryonic light microscopy image using an interactive image segmentation algorithm to obtain the blastomere mask and the embryo mask includes:
[0048] Input a light microscopic image of the embryo after image enhancement processing of the region of interest, and use the blastomere candidate bounding box as the target candidate bounding box to perform the Grabcut algorithm to obtain the blastomere mask.
[0049] In some embodiments, the step of automatically segmenting the embryonic light microscopy image using an interactive image segmentation algorithm to obtain the blastomere mask and the embryo mask includes:
[0050] The blastomere masks are superimposed, and the image is binarized. Connected components are marked and only the connected components with the largest area are retained. Then, the holes are filled to obtain the fused mask.
[0051] Input the region of interest after image enhancement, use the embryo candidate box as the target candidate box and perform the Grabcut algorithm to obtain the initial mask, perform image binarization, mark connected components and retain only the connected components with the largest area, and then fill holes to obtain the initial mask of the embryo.
[0052] Perform a logical AND operation on the fusion mask and the initial embryo mask to obtain the intermediate embryo mask;
[0053] Construct a gen-level tag map based on the embryonic intermediate mask;
[0054] Using the aforementioned label map as the precise label map, the Grabcut algorithm is applied to the light microscopic image of the embryo after image enhancement processing of the region of interest to obtain the embryo mask.
[0055] In some embodiments, the step of constructing a genotype-level tag map based on the embryonic intermediate mask includes:
[0056] The intermediate mask of the embryo is eroded and then inverted to mark it as the background;
[0057] The intermediate mask of the embryo is eroded and marked as the foreground;
[0058] The transition area between the background and the foreground is marked as a suspicious foreground;
[0059] A pixel-level label map is constructed based on the foreground, the background, and the suspected foreground.
[0060] In some embodiments, the steps of superimposing blastosphere masks, binarizing the image, marking connected components and retaining only the connected components with the largest area, and then filling holes to obtain a fused mask include:
[0061] The blastomere masks are superimposed, and pixels with values greater than 0 are set to 255 to obtain a binary image.
[0062] Label connected components in a binary graph;
[0063] Retain the pixels with the largest connected component area, and set the rest of the pixels to 0;
[0064] Perform hole filling and obtain the fusion mask.
[0065] The beneficial effects of this invention are as follows:
[0066] This invention first utilizes a deep learning-based object detection model to initially obtain candidate bounding boxes for blastomeres and embryos, performing preliminary localization of blastomeres and embryos, and accurately identifying the region of interest (ROI) to fix the value range of the embryo range. This minimizes interference from factors such as culture medium impurities, the proportion of the embryo in the image, varying light intensity, and contrast. Subsequently, image enhancement processing is performed only on the ROI. This reduces differences between different images and enhances the image difference between the ROI and the main subject of the image, expanding the difference between the embryo and the background within the same image. This increases the robustness of the subsequent interactive image segmentation algorithm, enabling accurate segmentation of blastomeres and embryos when the interactive image segmentation algorithm is used to segment the entire image. It combines the high accuracy of object detection in deep learning with the advantages of machine learning methods, such as no training required and high interpretability, effectively achieving accurate segmentation, outlining, and area calculation of blastomeres and embryos, resulting in higher recognition accuracy. Attached Figure Description
[0067] Figure 1 This is a flowchart of a method for automatic segmentation and area calculation of blastomeres in in vitro fertilized embryos according to the present invention.
[0068] Figure 2 This is a schematic diagram of the process for extracting the region of interest according to the present invention.
[0069] Figure 3 This is a schematic diagram of the image enhancement processing flow of the present invention.
[0070] Figure 4 This is a schematic diagram of the process for obtaining the embryo mask according to the present invention.
[0071] Figure 5 This is a schematic diagram illustrating the operation flow of a method for automatic segmentation and area calculation of blastomere images of in vitro fertilized embryos according to the present invention. Detailed Implementation
[0072] The present invention will be further described below with reference to the accompanying drawings and embodiments:
[0073] Please see Figure 1 and Figure 5 As shown, this invention discloses a method for automatic segmentation and area calculation of blastomeres in in vitro fertilized embryo images, comprising:
[0074] S100. A deep learning-based target detection model is used to detect blastomeres in embryo light microscopy images to obtain blastomer candidate boxes;
[0075] S200. Fuse blastomeres to obtain embryo candidate frames;
[0076] S300. Extract the region of interest based on the embryo candidate box;
[0077] S400. Perform image enhancement processing on the region of interest;
[0078] S500. After image enhancement processing, the region of interest in the embryo light microscopy image is automatically segmented using an interactive image segmentation algorithm to obtain the blastomere mask and the embryo mask.
[0079] S600. Blastome outlining and embryo outlining are performed on the light micrograph of the embryo using blastome mask and embryo mask, respectively;
[0080] S700. Calculate the area of the blastomere mask and the embryo mask to obtain the area of the blastomere and the area of the embryo.
[0081] This invention first utilizes a deep learning-based object detection model to initially obtain candidate bounding boxes for blastomeres and embryos, performing preliminary localization of blastomeres and embryos, and accurately identifying the region of interest (ROI) to fix the value range of the embryo range. This minimizes interference from factors such as culture medium impurities, the proportion of the embryo in the image, varying light intensity, and contrast. Subsequently, image enhancement processing is performed only on the ROI. This reduces differences between different images and enhances the image difference between the ROI and the main subject of the image, expanding the difference between the embryo and the background within the same image. This increases the robustness of the subsequent interactive image segmentation algorithm, enabling accurate segmentation of blastomeres and embryos when the interactive image segmentation algorithm is used to segment the entire image. It combines the high accuracy of object detection in deep learning with the advantages of machine learning methods, such as no training required and high interpretability, effectively achieving accurate segmentation, outlining, and area calculation of blastomeres and embryos, resulting in higher recognition accuracy.
[0082] In some embodiments, step S700 may be executed before S600 or simultaneously with S600.
[0083] In some embodiments, see Figure 3 As shown, the steps of image enhancement processing include:
[0084] S410. Adjust the color levels of the area of interest;
[0085] S420. Noise reduction is performed on the region of interest after color level adjustment;
[0086] S430. Enhances the contrast of the noise-reduced region of interest.
[0087] By adjusting color levels, reducing noise, and enhancing contrast, image enhancement processing is performed on the region of interest to reduce the differences between different images and increase the differences between the embryo and the background in the same image, which is beneficial for the high-precision segmentation of the embryo light microscopy image by the subsequent interactive image segmentation algorithm.
[0088] In some embodiments, the steps of adjusting color levels include:
[0089] S411. Convert the region of interest into an HSV channel;
[0090] S412. Calculate the mean of the H channel, denoted as h; calculate the mean of the V channel, denoted as v;
[0091] S413. Preset judgment threshold T, if If the color level adjustment is not performed, then the color level adjustment operation based on the color level adjustment formula will be executed; otherwise, the color level adjustment will be exited.
[0092] The formula for adjusting color levels is:
[0093]
[0094] Where I is the image data before color adjustment, I' is the image data after color adjustment, s, h, and m are constants, and satisfy s∈[0,170], h∈(s,255), m∈[1,3], and numbers less than 0 in Is are set to 0.
[0095] Preferably, the decision threshold T∈[100, 150].
[0096] Traditional color level adjustment algorithms require manual setting of adjustment ranges. This invention, however, uses a formula based on hue and brightness to automatically filter embryo micrographs, ensuring the color level adjustment algorithm only applies to images requiring adjustment, i.e., satisfying certain criteria. Adjust the image under this condition. 。
[0097] In color level adjustment, traditional methods such as histogram equalization and histogram stretching change the histogram trend and variance of the original image while adjusting the color levels. The color level adjustment formula used in this invention can shift pixels towards the bright end of the image while maintaining the original histogram trend, with less change in variance.
[0098] In some embodiments, the noise reduction step includes:
[0099] S421. Construct a total variational noise reduction function (TV) and preset the noise reduction parameter range;
[0100] S422. The optimal denoising parameters are obtained by iterating the input image in the denoising parameter range with a preset step size using the optimal J invariance method;
[0101] S423. Use the optimal noise reduction parameters to reduce noise in the input image.
[0102] Preferably, the noise reduction parameter range is [0.1, 0.9], and the preset step size is 0.02.
[0103] Specifically, the processing steps of the optimal J-invariant method can be adopted from the method described in "Noise2Self: Blind Denoising by Self-Supervision" (Authors: J. Batson & L. Royer, Source: International Conference on Machine Learning, pp. 524-533 (2019)).
[0104] The noise reduction method of this invention is automatic variable parameter noise reduction, which is more robust than the traditional single parameter noise reduction method. Moreover, the noise reduction method used in this invention is total variational noise reduction, which has the strongest noise reduction capability while preserving edges in embryo light microscopy imaging tasks compared with traditional bilateral filtering, Gaussian blur and other algorithms.
[0105] In some embodiments, the contrast enhancement step includes:
[0106] S431. Select the lowest probability threshold from the interval [0,1], denoted as q1, and select the highest probability threshold, denoted as q2, such that 0 <q1<q2<1;
[0107] S432. Calculate the pixel histogram of the input image and obtain the pixel values t1 and t2 corresponding to q1 and q2.
[0108] S433. Use t1 and t2 to perform histogram stretching on the pixel histogram of the input image.
[0109] The present invention uses the above-mentioned machine learning-based methods for color level adjustment, noise reduction, and contrast enhancement to form an automatic and standardized process that can adapt to the interference caused by differences in the proportion of the embryo subject to the whole image, contrast, light intensity, and signal-to-noise ratio when taking pictures between different devices and between different batches of the same device.
[0110] In some embodiments, the step of using a deep learning-based object detection model to detect blastomeres in embryo light microscopy images and obtaining blastomer candidate boxes includes:
[0111] S110. Read the embryo light microscopy image data;
[0112] S120. Perform neighborhood histogram equalization on the read data;
[0113] S130. Input the processed image into a deep learning-based object detection model to detect blastomeres and obtain primary blastomere bounding boxes. The deep learning-based object detection model can be a YOLO series model or an RCNN series model.
[0114] S140. Expand the primary blastomere rectangles using a preset bias value and record the vertex coordinates. If the expanded coordinates exceed the original size of the embryo light microscopy image, retain the corresponding original coordinates to obtain blastomere candidate frames. Preferably, select a value from the pixel range [10, 50] as the preset bias value.
[0115] This invention, by combining a deep learning-based target detection model with the expansion of step S140, can locate blastomeres while avoiding, to some extent, the risks caused by the inaccuracy of the minimum tangent box of the target detection model.
[0116] In some embodiments, see Figure 2 As shown, the steps for extracting the region of interest based on the embryo candidate box include:
[0117] S310. Using the center of the embryo candidate box as the center, expand the embryo candidate box according to the expansion formula to form an expanded candidate box, and use this as the region of interest, and record the corresponding vertex coordinates;
[0118] The expansion formula is: L' = L*k
[0119] Where L is the side length of the embryo candidate box, L' is the side length of the expanded candidate box, k is the expansion coefficient, and k∈[1.2,2].
[0120] This invention extracts the region of interest by combining a target detection model with a fixed magnification factor, which reduces the processing load in subsequent steps and also avoids the risk of inaccurate minimum tangent boxes in the target detection model to some extent.
[0121] In some embodiments, see Figure 4 As shown, the steps for automatically segmenting embryonic light microscopy images using an interactive image segmentation algorithm to obtain blastomere masks and embryo masks include:
[0122] S510. Input the embryo light microscopy image after image enhancement processing of the region of interest, and use the blastomere candidate box as the target candidate box to perform the Grabcut algorithm to obtain the blastomere mask.
[0123] In blastomere segmentation, if only the minimum tangent bounding box of each blastomere is used as the target bounding box, the segmentation effect is poor because other blastomeres also exist in the background. This invention uses an image enhancement algorithm that only performs image enhancement on the region of interest and applies a single blastomere segmentation algorithm to the entire image. This significantly increases the distinction between the background and foreground, allowing the Grabcut algorithm to segment even highly overlapping blastomeres.
[0124] In some embodiments, see Figure 4 and Figure 5 As shown, the steps for automatically segmenting embryonic light microscopy images using an interactive image segmentation algorithm to obtain blastomere masks and embryo masks include:
[0125] S521. Overlay the blastomere masks, binarize the image, mark the connected components and retain only the connected components with the largest area, and then fill the holes to obtain the fused mask;
[0126] S522. Input the region of interest after image enhancement, use the embryo candidate box as the target candidate box to perform the Grabcut algorithm to obtain the initial mask, perform image binarization, mark connected components and retain only the connected components with the largest area, and then fill the holes to obtain the initial mask of the embryo.
[0127] S523. Perform a logical AND operation on the fusion mask and the initial embryo mask to obtain the intermediate embryo mask;
[0128] S524. Constructing a genotype-level tag map based on the embryonic intermediate mask;
[0129] S525. Using the label image as the precise label image, perform the Grabcut algorithm on the embryo light microscopy image after image enhancement processing of the region of interest to obtain the embryo mask.
[0130] Existing Grabcut algorithms require manual setting of target candidate boxes or the provision of precise pixel-level label images. However, when faced with unclear tunica albuginea, highlights at the edges, debris adhering to blastomeres, or impurities such as extracellular granulosa cells and sperm adhering to the embryo, simple target candidate box-based Grabcut algorithms cannot effectively identify the differences between the foreground and background, leading to over-segmentation or incomplete segmentation. This invention separately segments the total embryo area and blastomeres, then merges the segments to generate background labels. This allows the background labels to penetrate more accurately into the inner part of the tunica albuginea. Furthermore, the pixel-level label image generated by this invention enables the Grabcut algorithm to perform recursive segmentation more accurately near the blurred embryo edges, thereby precisely identifying the embryo and allowing for accurate embryo outline drawing and area calculation.
[0131] In some embodiments, the step of constructing a genotype-level tag map based on an embryonic intermediate mask includes:
[0132] 1) Perform an erosion operation on the middle mask of the embryo and invert its colors to mark it as the background;
[0133] 2) Perform an erosion operation on the middle mask of the embryo and mark it as the foreground;
[0134] 3) Mark the transition area between the background and foreground as a suspicious foreground;
[0135] 4) Construct pixel-level label maps based on foreground, background, and suspected foreground.
[0136] This invention first etches the middle mask of the embryo and then marks it, further ensuring that the background label penetrates into the inner side of the zona pellucida. This overcomes the interference caused by factors such as unclear zona pellucida, highlights at the edges, fragments adhering to blastomeres, and impurities such as granulosa cells and sperm adhering to the embryo, which affect the precise segmentation of the embryo.
[0137] Furthermore, the steps for constructing a genotype-level tag map based on the embryonic intermediate mask include:
[0138] 1) Select a value in the interval [3, 13] as the side length, generate a square with a pixel value of 1 as the kernel, and denote it as k;
[0139] 2) Perform an erosion operation on the intermediate mask of the embryo, with 1 iteration and k kernels, and invert the colors to mark it as the background;
[0140] 2) Perform an erosion operation on the intermediate mask of the embryo, with the number of iterations being i, the kernel being k, and the label being the foreground, i∈[3,15];
[0141] 3) Mark the transition area between the background and foreground as a suspicious foreground;
[0142] 4) Construct pixel-level label maps based on foreground, background, and suspected foreground.
[0143] The erosion operation before background and foreground annotation is iterated to different degrees to further avoid the occurrence of foreground and background segmentation exceeding the boundary or being insufficient.
[0144] In some embodiments, the steps of superimposing blastosphere masks, binarizing the image, marking connected components and retaining only the connected components with the largest area, and then filling holes to obtain the fused mask include:
[0145] 1) Overlay the blastosphere masks and set pixels greater than 0 to 255 to obtain a binary image;
[0146] 2) Label the connected components of the binary graph;
[0147] 3) Retain the pixels with the largest connected component area, and set the rest of the pixels to 0;
[0148] 4) Fill the holes and obtain the fusion mask.
[0149] Furthermore, the input region of interest after image enhancement is used as the input image enhancement candidate box. The Grabcut algorithm is then applied to obtain the initial mask, and the image is binarized. Connected components are marked, and only the connected components with the largest area are retained. Subsequently, hole filling is performed. The steps to obtain the initial mask of the embryo include:
[0150] 1) Input the region of interest after image enhancement, use the embryo candidate box as the target candidate box and perform the Grabcut algorithm for n iterations to obtain the initial mask, n∈[5,20];
[0151] 2) Binarize the initial mask, setting pixels greater than 0 to 255 to obtain the binary image;
[0152] 3) Label the connected components of the binary graph;
[0153] 4) Retain the pixels with the largest connected component area, and set the remaining pixels to 0;
[0154] 5) Fill the holes to obtain the initial mask for the embryo.
[0155] Furthermore, the steps of outlining the blastomere and the embryo on the embryo light microscopy image using blastomere and embryo masks respectively include:
[0156] 1) Edge detection is performed on embryo light microscopy images based on blastomere masks, followed by outlining;
[0157] 2) Edge detection is performed on embryo light microscopy images based on embryo masks, followed by outlining.
[0158] Accordingly, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method. Simultaneously, the present invention also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described method.
[0159] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.
Claims
1. A method for automatic segmentation and area calculation of in vitro fertilization embryo blastomere image, characterized in that, The method comprises the following steps: An embryo light microscope picture is subjected to blastomere detection by using a deep learning-based target detection model to obtain a blastomere candidate frame; The blastomere candidate frame is fused to obtain an embryo candidate frame; An area of interest is extracted based on the embryo candidate frame; The area of interest is subjected to image enhancement processing; After the image enhancement processing of the area of interest of the embryo light microscope picture, an interactive image segmentation algorithm is used to automatically segment the embryo light microscope picture to obtain a blastomere mask and an embryo mask; The blastomere mask and the embryo mask are used to respectively perform blastomere outlining and embryo outlining on the embryo light microscope picture; The areas of the blastomere mask and the embryo mask are calculated to obtain a blastomere area and an embryo area; The step of using the interactive image segmentation algorithm to automatically segment the embryo light microscope picture to obtain the blastomere mask and the embryo mask comprises the following steps: The blastomere mask is superimposed and subjected to image binarization, connected domains are marked and only the connected domain with the largest area is retained, and then hole filling is performed to obtain a fused mask; The area of interest subjected to the image enhancement processing is input, the Grabcut algorithm is performed on the area of interest with the embryo candidate frame as a target candidate frame to obtain an initial mask, the initial mask is subjected to image binarization, connected domains are marked and only the connected domain with the largest area is retained, and then hole filling is performed to obtain an embryo initial mask; The fused mask and the embryo initial mask are subjected to logical AND operation to obtain an embryo intermediate mask; A pixel-level label map is constructed based on the embryo intermediate mask; The Grabcut algorithm is performed on the embryo light microscope picture subjected to the image enhancement processing of the area of interest with the label map as an accurate label map to obtain an embryo mask; The step of constructing the pixel-level label map based on the embryo intermediate mask comprises the following steps: The embryo intermediate mask is subjected to erosion operation and color inversion to be marked as a background; The embryo intermediate mask is subjected to erosion operation to be marked as a foreground; A transition region between the background and the foreground is marked as a suspicious foreground; The pixel-level label map is constructed based on the foreground, the background and the suspicious foreground.
2. The method for automatic segmentation and area calculation of in vitro fertilization embryo blastomere image according to claim 1, characterized in that, The step of the image enhancement processing comprises the following steps: The area of interest is subjected to color scale adjustment; The area of interest subjected to the color scale adjustment is subjected to noise reduction; The area of interest subjected to the noise reduction is subjected to contrast enhancement.
3. The method for automatic segmentation and area calculation of in vitro fertilization embryo blastomere image according to claim 2, characterized in that, The step of the color scale adjustment comprises the following steps: The area of interest is converted into an HSV channel; The mean value of the H channel is calculated and denoted as h; The mean value of the V channel is calculated and denoted as v; A preset determination threshold T is set, if then an operation of color scale adjustment based on a color scale adjustment formula is performed, otherwise, the color scale adjustment is exited. The color scale adjustment formula is as follows: wherein I is image data before color scale adjustment, I' is image data after color scale adjustment, s, h and m are constants, and s∈[0, 170], h∈(s, 255), m∈[1, 3], and numbers less than 0 in I-s are set to 0.
4. The method for automatic segmentation and area calculation of in vitro fertilization embryo blastomere image according to claim 2, characterized in that, The step of the noise reduction comprises the following steps: A total variation noise reduction function is constructed, and a noise reduction parameter interval is preset; An optimal J invariance method is used to iteratively obtain an optimal noise reduction parameter in the noise reduction parameter interval with a preset step length; The input image is subjected to noise reduction by using the optimal noise reduction parameter.
5. The method for automatic segmentation and area calculation of in vitro fertilization embryo blastomere image according to claim 1, characterized in that, The step of detecting blastomeres in the embryo light microscope picture by using the target detection model based on deep learning to obtain the blastomere candidate frame comprises: reading the embryo light microscope picture data; performing field histogram equalization on the read data; inputting the processed picture into the target detection model based on deep learning to detect blastomeres and obtain the primary blastomere rectangular frame; expanding the primary blastomere rectangular frame by using a preset offset value, and recording the vertex coordinates, if the expanded coordinates exceed the size of the original embryo light microscope picture, the corresponding original coordinates are retained, thereby obtaining the blastomere candidate frame.
6. The method for automatic segmentation and area calculation of in vitro fertilization embryo blastomere image according to claim 1, characterized in that, The step of extracting the region of interest based on the embryo candidate frame comprises: expanding the embryo candidate frame according to the expansion formula with the center of the embryo candidate frame as the center, the expanded embryo candidate frame is taken as the region of interest, and the corresponding vertex coordinates are recorded; the expansion formula is: L'=L*k wherein, L is the side length of the embryo candidate frame, L' is the side length of the expanded candidate frame, k is the expansion coefficient, and k∈[1.2, 2].
7. The method for automatic segmentation and area calculation of in vitro fertilization embryo blastomere image according to claim 1, characterized in that, The step of automatically segmenting the embryo light microscope picture by using the interactive image segmentation algorithm to obtain the blastomere mask and the embryo mask comprises: inputting the embryo light microscope picture after image enhancement processing of the region of interest, taking the blastomere candidate frame as the target candidate frame to perform Grabcut algorithm to obtain the blastomere mask.
8. The method for automatic segmentation and area calculation of in vitro fertilization embryo blastomere image according to claim 1, characterized in that, The step of superimposing the blastomere mask, performing image binarization, marking the connected domain and retaining only the largest area connected domain, and then filling the hole to obtain the fusion mask comprises: superimposing the blastomere mask and setting the pixels greater than 0 to 255 to obtain a binary image; marking the connected domain of the binary image; retaining the pixels of the largest area connected domain, and setting the rest of the pixels to 0; performing hole filling to obtain the fusion mask.
Citation Information
Patent Citations
Embryo quality comprehensive evaluation device based on deep learning
CN111539308A