Cell image processing methods, devices and computer equipment

Abstract

This application relates to the field of image processing technology, and provides a cell image processing method, apparatus, and computer device that can automatically obtain the distance of the perivitelline space. In this application, the inner contour region of the zona pellucida and the cytoplasmic contour region in the cell image are determined; the non-overlapping region between the inner contour region of the zona pellucida and the cytoplasmic contour region is determined; when there are multiple non-overlapping regions, the region with the largest area is selected as the target non-overlapping region, and the tilt angle of the minimum bounding rectangle of the target non-overlapping region is determined; based on the tilt angle and the cell center point, a straight line equation is obtained; the intersection points of the straight line equation with the cytoplasmic contour and the inner contour of the zona pellucida are calculated, and the distance of the perivitelline space is obtained based on the distance between the intersection points.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a cell image processing method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] Oocytes are a key factor determining the pregnancy rate of IVF-ET (In Vitro Fertilization and Embryo Transfer), and oocyte quality can affect fertilization rate, early embryonic development, pregnancy maintenance, and fetal development. Therefore, it is necessary to perform oocyte characterization analysis.

[0003] The perivitelline space is a narrow gap between the cytoplasm of the oocyte or pronuclear embryo and the inner layer of the zona pellucida. Under normal circumstances, the perivitelline space should be of moderate distance and free of foreign objects. An excessively large perivitelline space is often associated with abnormalities in the oocyte or pronuclear embryo. However, the current method of obtaining the perivitelline space is artificial, which is inefficient. Summary of the Invention

[0004] Therefore, it is necessary to provide a cell image processing method, apparatus, computer equipment, storage medium, and computer program product to address the aforementioned technical problems.

[0005] This application provides a cell image processing method, the method comprising:

[0006] Identify the inner contour region of the zona pellucida and the cytoplasmic contour region in the cell image;

[0007] Obtain the non-overlapping region between the inner contour region of the zona pellucida and the cytoplasmic contour region;

[0008] When there are multiple non-overlapping regions, the region with the largest area is selected as the target non-overlapping region, and the tilt angle of the minimum bounding rectangle of the target non-overlapping region is determined.

[0009] Based on the tilt angle and the cell center point, the equation of the straight line is obtained;

[0010] Calculate the intersection points of the linear equation with the cytoplasmic contour and the inner contour of the zona pellucida, respectively, and obtain the distance of the periovarian space based on the distance between the intersection points.

[0011] In one embodiment, the cell image may be an oocyte image or a pronuclear embryo image.

[0012] In one embodiment, before calculating the intersections of the linear equation with the cytoplasmic contour and the inner contour of the zona pellucida, the method further includes:

[0013] The cell image is segmented to obtain a cell mask;

[0014] The contour of the cell mask is extracted to obtain the cytoplasmic contour.

[0015] In one embodiment, after extracting the contour of the cell mask to obtain the cytoplasmic contour, the method further includes:

[0016] Based on the cytoplasmic outline, the cell's perimeter and area are obtained;

[0017] The roundness of the cell is obtained based on its perimeter and area.

[0018] In one embodiment, after extracting the contour of the cell mask to obtain the cytoplasmic contour, the method further includes:

[0019] Obtain the average distance from the contour pixel point to the cell center point; the contour pixel point is the pixel point on the cytoplasmic contour.

[0020] Obtain the deviation between the distance from the contour pixel point to the cell center point and the average distance;

[0021] The roundness of the cell is obtained based on the relationship between the average distance and the deviation.

[0022] In one embodiment, the method further includes:

[0023] The cell image is input into the target detection network to obtain the polar body prediction box;

[0024] The diameter of the polar body is calculated based on the size of the polar body prediction frame.

[0025] In one embodiment, the method further includes:

[0026] A mask for segmenting the zona pellucida from the cell image;

[0027] When the mask of the transparent strip is completely divided, the inner contour and outer contour of the transparent strip are detected, and the outer contour of the transparent strip is sampled to obtain multiple outer contour sampling points;

[0028] Obtain the lines connecting multiple outer contour sampling points to the center point of the outer contour of the transparent band, and based on the intersection of the connecting lines with the inner contour of the transparent band, obtain the inner contour sampling points corresponding to the outer contour sampling points.

[0029] The thickness of the transparent band is determined based on the distance between the outer contour sampling point and the corresponding inner contour sampling point.

[0030] In one embodiment, before determining the thickness of the transparent band based on the distance between the outer contour sampling point and the corresponding inner contour sampling point, the method further includes:

[0031] When the mask of the transparent band is not completely segmented, an incomplete transparent band outline is detected;

[0032] Obtain the linear equations between the center point of the incomplete transparent band contour and each azimuth angle in the azimuth angle set;

[0033] Based on the two intersection points generated by the linear equation and the incomplete transparent band contour, the corresponding outer contour sampling points and inner contour sampling points are determined.

[0034] This application provides a cell image processing apparatus, the apparatus comprising:

[0035] The region extraction module is used to determine the contour regions within the zona pellucida and the cytoplasmic contour regions in the cell image;

[0036] The non-overlapping region acquisition module is used to acquire the non-overlapping region between the inner contour region of the zona pellucida and the cytoplasmic contour region.

[0037] An angle determination module is used to select the area with the largest area as the target non-overlapping area when there are multiple non-overlapping areas, and to determine the tilt angle of the minimum bounding rectangle of the target non-overlapping area.

[0038] The straight line determination module is used to obtain the straight line equation based on the tilt angle and the cell center point;

[0039] The perivitelline distance acquisition module is used to calculate the intersection points of the linear equation with the cytoplasmic contour and the inner contour of the zona pellucida, respectively, and obtain the perivitelline distance based on the distance between the intersection points.

[0040] This application provides a computer device, including a memory and a processor, wherein the memory stores a computer program and the processor executes the above-described method.

[0041] This application provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor using the methods described above.

[0042] This application provides a computer program product having a computer program stored thereon, the computer program being executed by a processor using the above-described method.

[0043] In the aforementioned cell image processing methods, devices, computer equipment, storage media, and computer program products, the inner contour region of the zona pellucida and the cytoplasmic contour region in the cell image are determined; the non-overlapping region between the inner contour region of the zona pellucida and the cytoplasmic contour region is determined; when there are multiple non-overlapping regions, the region with the largest area is selected as the target non-overlapping region, and the tilt angle of the minimum bounding rectangle of the target non-overlapping region is determined; based on the tilt angle and the cell center point, a straight line equation is obtained; the intersection points of the straight line equation with the cytoplasmic contour and the inner contour of the zona pellucida are calculated respectively, and the distance of the perivitelline space is automatically obtained based on the distance between the intersection points, without the need for manual intervention, thus improving the efficiency of obtaining the perivitelline space distance. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating a cell image processing method in one embodiment;

[0045] Figure 2 Here is a diagram illustrating the correlations involved in analyzing periovarian distance in one embodiment;

[0046] Figure 3 Here is a network structure diagram of FCN in one embodiment;

[0047] Figure 4 Here is a network structure diagram of a cell segmentation model in one embodiment;

[0048] Figure 5 This is a flowchart illustrating a cell image processing method in one embodiment;

[0049] Figure 6 This is a schematic diagram of the polar body in one embodiment;

[0050] Figure 7(a) is a schematic diagram of a transparent strip mask that is completely segmented in one embodiment;

[0051] Figure 7(b) is a schematic diagram of contour sampling in one embodiment where the mask is completely segmented;

[0052] Figure 8(a) is a schematic diagram of a transparent strip mask that is not completely segmented in one embodiment;

[0053] Figure 8(b) is a schematic diagram of contour sampling in one embodiment where the mask is not completely segmented;

[0054] Figure 9 A waveform diagram showing the thickness of the transparent strip in one embodiment;

[0055] Figure 10(a) is an illustration of an embryo with a smaller perivitelline gap in one embodiment;

[0056] Figure 10(b) is an illustration of an embryo with a larger perivitelline gap in one embodiment;

[0057] Figure 10(c) is a diagram of the ROI of an oocyte with a smaller perivitelline space in one embodiment;

[0058] Figure 10(d) is a diagram of the ROI corresponding to a larger perivitelline space in one embodiment;

[0059] Figure 10(e) is an example of a smaller periovarian space in one embodiment;

[0060] Figure 10(f) is an example of a larger periovarian space in one embodiment;

[0061] Figure 11 This is a structural block diagram of a cell image processing device in one embodiment;

[0062] Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0064] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0065] The following describes the technical terms related to this application:

[0066] IVF-ET: (In Vitro Fertilization and Embryo Transfer) Artificial insemination and in vitro fertilization-embryo transfer.

[0067] ZP: Zona Pellucida. The zona pellucida is a ring of unstructured, eosinophilic gelatinous material surrounding the oocyte. It is secreted by the oocyte and the surrounding follicular cells during the growth and development of the egg.

[0068] ART: Assisted Reproductive Technology, refers to techniques that use medical assistance to help infertile couples conceive.

[0069] Oocytes: Oocytes are oogonia that undergo meiosis during oogenesis.

[0070] ICSI: Intracytoplasmic Sperm Injection, is a technique that uses a micromanipulator to inject a single sperm into an egg to fertilize it.

[0071] FCN: Fully Convolutional Network, a framework for image semantic segmentation.

[0072] The cell image processing method provided in this application mainly performs feature analysis on oocytes or embryos, including measuring the perivitelline distance. This method includes... Figure 1 The steps shown are as follows:

[0073] Step S101: Determine the inner contour region of the zona pellucida and the cytoplasmic contour region in the cell image.

[0074] The cell image can be an oocyte image or a pronuclear embryo image. Using a zona pellucida segmentation model, a zona pellucida mask is extracted from the cell image to determine the inner contour region of the zona pellucida within the cell image, such as... Figure 2 As shown in (a); using a cell segmentation model, cell masks are extracted from the cell image to determine the cytoplasmic contour region in the cell image, such as... Figure 2 As shown in (b); when the cell image is an oocyte image, the cytoplasmic contour region is the oocyte contour region, and when the cell image is a pronuclear embryo image, the cytoplasmic contour region is the pronuclear embryo contour region.

[0075] Translucency segmentation models can be built based on networks such as deeplabV3, Unet, UneXt, and FCN. In this embodiment, the transparency segmentation model is based on... Figure 3 The FCN network shown can accept input images of any size. It uses deconvolution to upsample the feature map of the last convolutional layer, restoring it to the same size as the input image. This allows it to generate a pixelwise prediction for each pixel while preserving the original spatial information in the input image. Finally, it performs pixel-wise classification on the upsampled feature map.

[0076] To achieve good segmentation accuracy for semantic segmentation of the transparency band, various semantic segmentation methods can be tested, including deeplabV3, Unet, UneXt, FCN-resnet50, and FCN-resnet101. FCN-Resnet101 is more suitable for completing the segmentation task of the transparency band, balancing the requirements of feature extraction for detail and classification accuracy.

[0077] To obtain a zona pellucida segmentation model, segmentation training can be performed. During segmentation training, data labeling tools can be used to select cell images for labeling, marking the segmentation regions, and setting classification labels. The labels can be divided into two categories: 0: background (background does not need labeling) and 1: zona pellucida region. The labeled cell image dataset is divided into training and validation sets at a ratio of 3:1. The original input data is then planned and data augmented, and iterative training is performed to obtain the zona pellucida segmentation model. Finally, a zona pellucida mask is obtained based on the zona pellucida segmentation model.

[0078] When the cell image is an oocyte image, the cell segmentation model is mainly used to segment the oocytes in the cell image. When the cell image is a pronuclear embryo image, the cell segmentation model is mainly used to segment the pronuclear embryos in the cell image. The cell segmentation model can use instance segmentation networks, such as Solov2, BCnet, and Mask R-CNN, to perform instance segmentation of cells in the cell image. In the embodiments of this application, the cell segmentation model adopts Mask R-CNN. Mask R-CNN is based on the Faster R-CNN framework, and a fully connected segmentation network is added after the basic feature network, changing the original two tasks (classification + regression) to three tasks (classification + regression + segmentation). Mask R-CNN adopts the same two stages as Faster R-CNN, and uses FCN for semantic segmentation of each proposal box of Fast R-CNN. The backbone network of Mask R-CNN consists of ResNet50 and FPN. The residual network structure ensures that the network will not degrade during the stacking process, while FPN utilizes the spatial information of the low-level feature maps and the semantic information of the high-level feature maps at the same time. By training the MaskRCNN network model and inputting the original cell image, the cell segmentation model can infer the cell mask C_mask and cell center point O_c of the oocyte or pronuclear embryo. Simultaneously, the cytoplasmic contour C_p can be extracted from the cell mask C_mask of the oocyte or pronuclear embryo, such as... Figure 4 As shown.

[0079] For cell instance segmentation, methods such as Solov2, BCnet, and Mask R-CNN can be used for testing. Mask R-CNN is chosen because it can handle not only object detection but also semantic segmentation. Furthermore, to address the issue of Faster R-CNN's double quantization of the ROI pooling layer failing to accurately align the feature map with the original pixels, Mask R-CNN replaces ROI pooling with ROIAlign. ROIAlign can align pixels, meeting the accuracy requirements for image semantic segmentation.

[0080] Step S102: Obtain the non-overlapping region between the inner contour region of the zona pellucida and the cytoplasmic contour region.

[0081] The result is as follows Figure 2 (a) The transparent inner contour region shown is denoted as Region1. The obtained... Figure 2 (b) The cytoplasmic contour region shown is denoted as Region2. The difference between Region1 and Region2 is calculated to obtain the non-overlapping region between Region1 and Region2, denoted as Uor. Figure 2 As shown in (c).

[0082] Step S103: When there are multiple non-overlapping regions, select the region with the largest area as the target non-overlapping region, and determine the tilt angle of the minimum bounding rectangle of the target non-overlapping region.

[0083] It can be seen Figure 2 (c) There are multiple non-overlapping regions, some of which have scattered and independent boundaries. To remove these non-overlapping regions, we can select the non-overlapping region with the largest area from among the multiple non-overlapping regions, resulting in, as shown in the figure. Figure 2 (d) shows the non-overlapping target area.

[0084] Next, determine the minimum bounding rectangle of the target non-overlapping region, and determine the tilt angle of the minimum bounding rectangle, which is determined by the slope of the minimum bounding rectangle.

[0085] Step S104: Obtain the equation of the straight line based on the tilt angle and the center point of the cell;

[0086] When the cell image is an oocyte image, the center point of the cell is the center point of the oocyte; when the cell image is a pronuclear embryo image, the center point of the cell is the center point of the pronuclear embryo.

[0087] Step S105: Calculate the intersection points of the straight line equation with the cytoplasmic outline and the inner outline of the zona pellucida, respectively, and obtain the distance of the periovarian space based on the distance between the intersection points.

[0088] The equation of the line mentioned above can be in the form of a point direction equation, that is: (u,v) is the direction vector of the line equation. After obtaining the line equation, the intersection points between the line equation and the non-overlapping target region can be determined. There are two intersection points. The intersection point closer to the cell center is the point where the line equation in the point direction intersects the cytoplasmic contour, denoted as P1(X). p1 Y p1 The intersection point furthest from the cell center is the intersection of the point-direction linear equation and the inner contour of the zona pellucida, denoted as P2(X). p2 Y p2 Next, the distance between the two intersection points mentioned above... As the distance of the periovarian space, such as Figure 2 As shown in (e).

[0089] Furthermore, the reference range for the perivitelline space distance is (8.5–12.3 μm). When the perivitelline space distance is greater than 15 μm (the threshold setting can be adjusted based on clinical experience and consensus on embryonic development), it can be determined that the perivitelline space distance is too large.

[0090] In the above cell image processing method, the inner contour region of the zona pellucida and the cytoplasmic contour region in the cell image are determined; the non-overlapping region between the inner contour region of the zona pellucida and the cytoplasmic contour region is determined; when there are multiple non-overlapping regions, the region with the largest area is selected as the target non-overlapping region, and the tilt angle of the minimum bounding rectangle of the target non-overlapping region is determined; the equation of a straight line is obtained based on the tilt angle and the cell center point; the intersection points of the straight line equation with the cytoplasmic contour and the inner contour of the zona pellucida are calculated respectively, and the distance of the perivitelline space is automatically obtained based on the distance between the intersection points, without the need for manual intervention, thus improving the efficiency of obtaining the perivitelline space distance.

[0091] In one embodiment, before calculating the intersection points of the linear equation with the cytoplasmic contour and the inner contour of the zona pellucida, the method provided in this application further includes: segmenting the cell image to obtain a cell mask; and extracting the contour of the cell mask to obtain the cytoplasmic contour.

[0092] Specifically, a cell segmentation model can be used to segment the cell image to obtain a cell mask, and then the contour of the cell mask can be extracted to obtain the cytoplasmic contour.

[0093] In the above embodiments, the cell image is first segmented to obtain a cell mask; then the contour of the cell mask is extracted to obtain the cytoplasmic contour, making the contour more accurate.

[0094] In one embodiment, such as Figure 5As shown, the feature analysis of the embryo may also include: calculating the roundness of the cells, specifically including the following steps: obtaining the perimeter and area of ​​the cells based on the cytoplasmic outline; and obtaining the roundness of the cells based on the perimeter and area of ​​the cells.

[0095] This embodiment mainly obtains cell roundness. By judging the cell roundness Ri, it detects whether there are abnormal cell morphology phenomena in the embryo.

[0096] When calculating the roundness of a cell, the cell's perimeter L can be obtained. i and area S i Using the formula The roundness of the cells is obtained. The roundness obtained in this way is between [0,1]. The closer the calculated value is to 1, the rounder the cell is.

[0097] In the above embodiments, the roundness of the cell is obtained by measuring the cell's perimeter and area, which helps to detect whether the embryo exhibits abnormal cell morphology.

[0098] Regarding the roundness of cells, this application provides another implementation method, which specifically includes the following steps: obtaining the average distance from the contour pixel point to the cell center point; the contour pixel point is a pixel point on the cytoplasmic contour; obtaining the deviation between the distance from the contour pixel point to the cell center point and the average distance; and obtaining the roundness of the cell based on the relationship between the average distance and the deviation.

[0099] Let p be the cell center point; p_i be the outline pixel point; and F be the area of ​​the image region enclosed by the cytoplasmic outline, which can be represented by the number of pixels within the cytoplasmic outline.

[0100]

[0101]

[0102]

[0103] Where Dis represents the average distance from the contour pixel to the cell center point, and Sigma is the deviation between the distance from the contour pixel to the cell center point and the average distance. Roundness represents the relationship between the average distance and the deviation, and can be used to measure cell roundness. The roundness evaluation result obtained using Roundness is between [0,1], and the closer the calculated value is to 1, the greater the roundness of the cell.

[0104] In the above embodiments, the average distance from the contour pixel point to the cell center point is obtained, as well as the deviation between the distance from the contour pixel point to the cell center point and the average distance. Based on the relationship between the average distance and the deviation, the roundness of the cell is obtained. The roundness of the cell can be obtained without relying on the cell perimeter and area, thus improving the flexibility of calculation.

[0105] In one embodiment, such as Figure 5 As shown, feature analysis can also include: polar body detection; the purpose of polar body detection is to find the location and number of polar bodies and measure the diameter of polar bodies. The diameter of the polar body is used to determine whether the polar body is too large, thereby reducing the impact of abnormal phenomena of giant polar bodies on embryonic development. In addition, the expulsion of polar bodies is also an important marker of the embryonic development process. Usually, the expulsion of the first polar body is used as a marker of oocyte maturation, while the expulsion of the second polar body represents successful embryonic fertilization.

[0106] The method provided in this application further includes: inputting a cell image into a target detection network to obtain a polar body prediction box; and calculating the diameter of the polar body based on the size of the polar body prediction box.

[0107] Object detection networks can include YOLOv5 and Faster-RCNN. These networks can be used to detect and count the extruded polar bodies of oocytes, such as... Figure 6 As shown, 601 represents the bounding box during labeling, and 602 represents the polar body identified through inference. Through inference by the object detection network, the polar body prediction box bbox[x1,y1,x2,y2] can be obtained. Then, according to the formula D_pa=(abs(x2-x1)+abs(y2-y1)) / 2, the diameter D_pa of the polar body can be calculated.

[0108] For polar body detection, this embodiment selected YOLOv5 and FasterCNN for testing. The test results show that the single-level network model YOLOv5 is more suitable for polar body detection requirements.

[0109] In the above embodiments, the diameter of the polar body is automatically analyzed and predicted by the target detection network without human intervention, thus improving processing efficiency.

[0110] In one embodiment, such as Figure 5 As shown, feature analysis can also include zona pellucida thickness analysis. The zona pellucida surrounds the oocyte, and measuring its thickness is not only used to assess oocyte quality but also serves as an indicator for embryologists when selecting treatment methods. With a thickness ranging from 10-31 μm, the zona pellucida thickness is related to the fertilization efficiency of the oocyte. Therefore, calculating the thickness of the oocyte helps embryologists provide reference values ​​for embryo analysis and processing.

[0111] The method provided in this application also includes the following steps:

[0112] Step S1: Segment the mask of the zona pellucida from the cell image.

[0113] The cell image can be an oocyte image or a pronuclear embryo image.

[0114] Step S2: When the mask of the transparent band is completely segmented, the inner contour and outer contour of the transparent band are detected, and the outer contour of the transparent band is sampled to obtain multiple outer contour sampling points.

[0115] This step can be implemented using the `findContours` function in OpenCV, with the detection mode set to `RETR_TREE`. All contours are extracted and the network structure is reconstructed. All contours are retrieved and the complete hierarchical structure of nested contours is reconstructed. The enumerator for the contour approximation method is `CHAIN_APPROX_SIMPLE`. The `findContours` function is set to `RETR_TREE` mode. The mask of the zona pellucida is completely segmented, indicating that the zona pellucida is not occluded or missing in the cell image. At this point, the zona pellucida ring structure is in a closed loop state, and two contours, the inner and outer contours, can be detected, as shown in Figure 7(a). The outer contour of the zona pellucida can be denoted as `ZP_Polygon_out`, and the inner contour can be denoted as `ZP_Polygon_in`.

[0116] Step S3: Obtain the lines connecting multiple outer contour sampling points and the center point of the outer contour of the transparent band. Based on the intersection of the connecting lines with the inner contour of the transparent band, obtain the inner contour sampling points corresponding to the outer contour sampling points.

[0117] When the inner and outer contours of the transparent band are detected, this step mainly involves first sampling the outer contour of the transparent band, and then sampling the inner contour of the transparent band based on the obtained outer contour sampling points to obtain the inner contour sampling points.

[0118] Specifically, the outer contour of the transparent band can be extracted using a contour approximation method. The principle is to approximate the outer contour of the transparent band with another contour having fewer vertices, such that the distance between them is less than or equal to a specified precision. This is achieved using `approxPolyDP()`. The second parameter is used for the precision of the contour approximation, representing the maximum distance between the original contour and its approximate contour. The smaller the value, the more closely the approximate contour fits the original contour. The third parameter specifies whether the approximate contour is closed. This results in the set of sampling points for the outer contour, which can be set as C_n.

[0119] Traverse the outer contour sampling points in the set C_n. When traversing to the i-th outer contour sampling point (C_i_nx, C_i_ny), the equation of the straight line between the outer contour sampling point (C_i_nx, C_i_ny) and the center point O_z of the outer contour of the transparent band can be calculated and denoted as L_a[i].

[0120] Calculate the intersection point (C_i_mx,C_i_my) between the straight line equation L_a[i] and the inner contour ZP_Polygon_in of the transparent band. Use this intersection point as the inner contour sampling point and store (C_i_mx,C_i_my) in C_m.

[0121] It should be noted that the outer contour sampling points (C_i_nx, C_i_ny) and the inner contour sampling points (C_i_mx, C_i_my) mentioned above correspond to each other.

[0122] After traversing all the outer contour sampling points in the outer contour sampling point set C_n, the inner contour sampling points corresponding to each outer contour sampling point can be determined, as shown in Figure 7(b).

[0123] Step S4: Analyze the thickness of the transparent band based on the distance between the outer contour sampling point and the corresponding inner contour sampling point.

[0124] For example, after obtaining the i-th outer contour sampling point (C_i_nx, C_i_ny) and its corresponding inner contour sampling point (C_i_mx, C_i_my), the distance between these two sampling points can be obtained. This distance can be called the point distance and can be denoted as k__nm[i]. Based on {k__nm[1], k__nm[2], ..., k__nm[i]}, the thickness of the transparent band is analyzed.

[0125] In the above embodiments, a mask of the zona pellucida is segmented from the cell image. When the mask of the zona pellucida is completely segmented, the inner contour and outer contour of the zona pellucida are detected, and the outer contour of the zona pellucida is sampled to obtain multiple outer contour sampling points. The lines connecting the multiple outer contour sampling points and the center point of the outer contour of the zona pellucida are obtained. Based on the intersection of each line with the inner contour of the zona pellucida, the inner contour sampling points corresponding to the outer contour sampling points are obtained. Based on the distance between the corresponding outer contour sampling points and the inner contour sampling points, the thickness of the zona pellucida is automatically analyzed without the need for manual marking and calculation, thus improving efficiency.

[0126] In one embodiment, before analyzing the thickness of the transparent band based on the distance between the outer contour sampling point and the corresponding inner contour sampling point, the method provided in this application further includes: detecting an incomplete transparent band contour when the mask of the transparent band is not completely segmented; obtaining the linear equations between the center point of the incomplete transparent band contour and each azimuth angle in the azimuth angle set; and determining the corresponding outer contour sampling point and inner contour sampling point based on the two intersection points generated by the linear equations and the incomplete transparent band contour.

[0127] This embodiment can be implemented based on the `findContours` function in OpenCV, with the detection mode set to `RETR_TREE`. It extracts all contours and reconstructs the network structure, retrieving all contours and reconstructing the complete hierarchical structure of nested contours. The enumerator for the contour approximation method is `CHAIN_APPROX_SIMPLE`. The `findContours` function is set to `RETR_TREE` mode. Incomplete segmentation of the zona pellucida mask indicates that the zona pellucida is occluded or missing in the cell image. In this case, the ring structure of the zona pellucida is disrupted, and only one contour can be detected. This contour is called an incomplete zona pellucida contour, as shown in Figure 8(a), where the incomplete zona pellucida contour can be denoted as `ZP_Polygon`.

[0128] When a contour is detected, the azimuth angle method can be used to extract the outer contour sampling points and their corresponding inner contour sampling points from the incomplete transparent band contour.

[0129] When the azimuth angle set An_s is set to {An_s[1],An_s[2],…,An_s[i]}, when traversing to the i-th azimuth angle An_s[i], the equation of the straight line between the azimuth angle and the center point of the non-complete transparent strip contour ZP_Polygon can be obtained, denoted as L_b[i]. Next, the intersection of the linear equation L_b[i] with the incomplete transparent band contour ZP_Polygon is calculated. At this time, the linear equation L_b[i] will have two intersection points with the incomplete transparent band contour ZP_Polygon, as shown in Figure 8(b). These two intersection points are the corresponding outer contour sampling point and inner contour sampling point. Among these two intersection points, the point closer to the center of the incomplete transparent band contour ZP_Polygon is the inner contour sampling point, denoted as (C_i_mx, C_i_my), and added to the inner contour sampling point set C_m. The point farther away from the center of the incomplete transparent band contour ZP_Polygon is the outer contour sampling point, denoted as (C_i_nx, C_i_ny), and added to the outer contour sampling point set C_n.

[0130] In this embodiment, the appropriate contour sampling method is selected based on whether the zona pellucida mask is completely segmented. This takes into account the situation where the zona pellucida is occluded in the cell image during the oocyte stage, further adapting to the functions required by the computer equipment and the universality of the algorithm. When the zona pellucida mask is completely segmented, the contour sampling method as in steps S2 and S3 is used. When using this method, the sparsity and density of sampling can be flexibly adjusted by setting the coefficient δ. When the zona pellucida mask is not completely segmented, the azimuth contour sampling method described above is used.

[0131] Further, obtaining the center point of the incomplete transparency band contour includes: obtaining the zero-order spatial moment, the first-order spatial moment about the horizontal axis, and the first-order spatial moment about the vertical axis of the image region enclosed by the incomplete transparency band contour; and determining the center point of the incomplete transparency band contour in the horizontal and vertical directions based on the ratio of the first-order spatial moment to the zero-order spatial moment about the horizontal axis and the ratio of the first-order spatial moment to the zero-order spatial moment about the vertical axis.

[0132] In this embodiment, the image region enclosed by the incomplete transparent band contour can be determined, and the zero-order spatial moment m of the image region can be calculated. 00 The first spatial moment m about the horizontal axis 01 and the first spatial moment m about the vertical axis 10 .

[0133] The formula for calculating the space moment is: Here, array(x, y) is the pixel value at pixel (x, y). When i = j = 0, it corresponds to the zeroth-order spatial moment; when i = 0 and j = 1, it corresponds to the first-order spatial moment about the horizontal axis; when i = 1 and j = 0, it corresponds to the first-order spatial moment about the vertical axis.

[0134] For a binary image, the zeroth-order spatial moment m 00 It is the area of ​​the image region enclosed by the non-complete transparent band outline.

[0135] Next, we can obtain the first-order spatial moment m about the horizontal axis. 01 With zeroth order space moment m 00 The ratio of the two values ​​is used to obtain the first-order spatial moment m on the vertical axis. 10 The ratio m to the zeroth-order space moment 00 Thus, the center point O of the incomplete transparent band outline is obtained. T coordinates in,

[0136] In the above embodiments, from the perspective of image spatial moments, the center point of the incomplete transparent band contour is accurately determined in the horizontal and vertical directions based on the ratio of the first-order spatial moment to the zero-order spatial moment about the horizontal axis and the ratio of the first-order spatial moment to the zero-order spatial moment about the vertical axis.

[0137] Furthermore, the azimuth angles in the azimuth angle set take values ​​from 0° to 360°, which belongs to an arithmetic sequence. That is, in the azimuth angle set An_s, the difference between two adjacent azimuth angles is fixed, and the difference can be 45°. The corresponding azimuth angle set An_s can be {0,45,90,135,180,225,270,315}.

[0138] In the above embodiments, the azimuth angles in the azimuth angle set are taken from 0° to 360°, which belongs to an arithmetic sequence, so as to achieve uniform sampling of the non-complete transparent band contour and ensure the accuracy of the transparent band thickness analysis.

[0139] In one embodiment, analyzing the thickness of the transparent band based on the distance between the outer contour sampling points and the corresponding inner contour sampling points includes: obtaining the distance between the points based on the outer contour sampling points and the corresponding inner contour sampling points; mapping the corresponding distance between the points to a coordinate system according to the sampling order in which the sampling points are formed to obtain a waveform; and analyzing the thickness of the transparent band based on the waveform.

[0140] For example, after obtaining the corresponding outer contour sampling points (C_i_nx, C_i_ny) and inner contour sampling points (C_i_mx, C_i_my), the distance between these two sampling points can be obtained. This distance can be called the inter-point distance, denoted as k__nm[i]. The sampling order that forms the above two sampling points is i. According to this sampling order, the inter-point distance k__nm[i] is mapped to the coordinate system to obtain the waveform, as shown below. Figure 9 As shown, the thickness of the transparent band is analyzed based on the waveform.

[0141] Furthermore, based on the waveform, the thickness of the transparent band is analyzed, including: determining the thickest position on the transparent band based on the peaks in the waveform and the sampling order corresponding to the peaks; and determining the thinnest position on the transparent band based on the troughs in the waveform and the sampling order corresponding to the troughs.

[0142] In this embodiment, the "thickest" and "thinnest" areas of the zona pellucida are located. The thicker areas of the zona pellucida can be determined by the peaks, and the thinner areas by the troughs. During the laser thinning process on the embryo, the thinner areas of the zona pellucida are automatically analyzed and located, providing the operator with a reference point for membrane breaking.

[0143] When the method provided in this application is applied to a real-world scenario, the cell images can be obtained from an embryo time-lapse incubator. Relevant data for real-world scenarios are shown in Table 1.

[0144]

[0145] Table 1

[0146] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0147] In one embodiment, such as Figure 11 As shown, a cell image processing apparatus is provided, comprising:

[0148] Region extraction module 1101 is used to determine the inner contour region of the zona pellucida and the cytoplasmic contour region in the cell image;

[0149] The non-overlapping region acquisition module 1102 is used to acquire the non-overlapping region between the inner contour region of the zona pellucida and the cytoplasmic contour region.

[0150] Angle determination module 1103 is used to select the area with the largest area as the target non-overlapping area when there are multiple non-overlapping areas, and to determine the tilt angle of the minimum bounding rectangle of the target non-overlapping area.

[0151] The straight line determination module 1104 is used to obtain the straight line equation based on the tilt angle and the cell center point;

[0152] The perivitelline distance acquisition module 1105 is used to calculate the intersection points of the straight line equation with the cytoplasmic contour and the inner contour of the zona pellucida, respectively, and obtain the perivitelline distance based on the distance between the intersection points.

[0153] In one embodiment, the device further includes a contour extraction module for segmenting the cell image to obtain a cell mask; and extracting the contour of the cell mask to obtain a cytoplasmic contour.

[0154] In one embodiment, the device further includes a roundness calculation module for obtaining the perimeter and area of ​​the cell based on the cytoplasmic contour; and obtaining the roundness of the cell based on the perimeter and area of ​​the cell.

[0155] In one embodiment, the roundness calculation module is further configured to obtain the average distance from the contour pixel to the cell center point; the contour pixel is a pixel on the cytoplasmic contour; obtain the deviation between the distance from the contour pixel to the cell center point and the average distance; and obtain the roundness of the cell based on the relationship between the average distance and the deviation.

[0156] In one embodiment, the device further includes a polar body diameter calculation module, used to input the cell image into a target detection network to obtain a polar body prediction box; and to calculate the diameter of the polar body based on the size of the polar body prediction box.

[0157] In one embodiment, the device further includes a zona pellucida thickness analysis module, used to segment a mask of the zona pellucida from the cell image; when the mask of the zona pellucida is completely segmented, the inner contour and outer contour of the zona pellucida are detected, and the outer contour of the zona pellucida is sampled to obtain multiple outer contour sampling points; the lines connecting the multiple outer contour sampling points to the center point of the outer contour of the zona pellucida are obtained, and based on the intersection of the connecting lines with the inner contour of the zona pellucida, inner contour sampling points corresponding to the outer contour sampling points are obtained; the thickness of the zona pellucida is determined based on the distance between the outer contour sampling points and the corresponding inner contour sampling points.

[0158] In one embodiment, the device further includes a non-complete transparent band contour sampling module, used to detect a non-complete transparent band contour when the mask of the transparent band is not completely segmented; obtain the linear equations between the center point of the non-complete transparent band contour and each azimuth angle in the azimuth angle set; and determine the corresponding outer contour sampling point and inner contour sampling point based on the two intersection points generated by the linear equations and the non-complete transparent band contour.

[0159] Specific limitations regarding the cell image processing device can be found in the limitations of the cell image processing method described above, and will not be repeated here. Each module in the aforementioned cell image processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0160] In one embodiment, a computer device is provided, the internal structure of which can be shown as follows: Figure 12As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. The computer device also includes input / output interfaces, which are connection circuits for exchanging information between the processor and external devices. These interfaces are connected to the processor via a bus and are referred to as I / O interfaces. When the computer program is executed by the processor, it implements a cell image processing method. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0161] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0162] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in the various method embodiments described above.

[0163] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the various method embodiments described above.

[0164] In one embodiment, a computer program product is provided having a computer program stored thereon, the computer program being executed by a processor of the steps described in the various method embodiments above.

[0165] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0166] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0167] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0168] The above embodiments are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A cell image processing method, characterized in that, The method includes: Identify the inner contour region of the zona pellucida and the cytoplasmic contour region in the cell image; Obtain the non-overlapping region between the inner contour region of the zona pellucida and the cytoplasmic contour region; When there are multiple non-overlapping regions, the region with the largest area is selected as the target non-overlapping region, and the tilt angle of the minimum bounding rectangle of the target non-overlapping region is determined. Based on the tilt angle and the cell center point, the equation of the straight line is obtained; Calculate the intersection points of the linear equation with the cytoplasmic contour and the inner contour of the zona pellucida, respectively, and obtain the distance of the periovarian space based on the distance between the intersection points.

2. The method according to claim 1, characterized in that, Before calculating the intersection points of the linear equation with the cytoplasmic contour and the inner contour of the zona pellucida, the method further includes: The cell image is segmented to obtain a cell mask; The contour of the cell mask is extracted to obtain the cytoplasmic contour.

3. The method according to claim 2, characterized in that, After extracting the contour of the cell mask to obtain the cytoplasmic contour, the method further includes: Based on the cytoplasmic outline, the cell's perimeter and area are obtained; The roundness of the cell is obtained based on its perimeter and area.

4. The method according to claim 2, characterized in that, After extracting the contour of the cell mask to obtain the cytoplasmic contour, the method further includes: Obtain the average distance from the contour pixel point to the cell center point; the contour pixel point is the pixel point on the cytoplasmic contour. The deviation between the distance from the contour pixel point to the cell center point and the average distance is obtained; The roundness of the cell is obtained based on the relationship between the average distance and the deviation.

5. The method according to claim 1, characterized in that, The method further includes: The cell image is input into the target detection network to obtain the polar body prediction box; The diameter of the polar body is calculated based on the size of the polar body prediction frame.

6. The method according to claim 1, characterized in that, The method further includes: A mask for segmenting the zona pellucida from the cell image; When the mask of the transparent strip is completely divided, the inner contour and outer contour of the transparent strip are detected, and the outer contour of the transparent strip is sampled to obtain multiple outer contour sampling points; Obtain the lines connecting multiple outer contour sampling points to the center point of the outer contour of the transparent band, and based on the intersection of the connecting lines with the inner contour of the transparent band, obtain the inner contour sampling points corresponding to the outer contour sampling points. The thickness of the transparent band is determined based on the distance between the outer contour sampling point and the corresponding inner contour sampling point.

7. The method according to claim 6, characterized in that, Before determining the thickness of the transparent band based on the distance between the outer contour sampling point and the corresponding inner contour sampling point, the method further includes: When the mask of the transparent band is not completely segmented, an incomplete transparent band outline is detected; Obtain the linear equations between the center point of the incomplete transparent band contour and each azimuth angle in the azimuth angle set; Based on the two intersection points generated by the linear equation and the incomplete transparent band contour, the corresponding outer contour sampling points and inner contour sampling points are determined.

8. A cell image processing device, characterized in that, The device includes: The region extraction module is used to determine the contour regions within the zona pellucida and the cytoplasmic contour regions in the cell image; The non-overlapping region acquisition module is used to acquire the non-overlapping region between the inner contour region of the zona pellucida and the cytoplasmic contour region. An angle determination module is used to select the area with the largest area as the target non-overlapping area when there are multiple non-overlapping areas, and to determine the tilt angle of the minimum bounding rectangle of the target non-overlapping area. The straight line determination module is used to obtain the straight line equation based on the tilt angle and the cell center point; The perivitelline distance acquisition module is used to calculate the intersection points of the linear equation with the cytoplasmic contour and the inner contour of the zona pellucida, respectively, and obtain the perivitelline distance based on the distance between the intersection points.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images