Method, device and computer equipment for determining thickness of transparent zone

Abstract

The present application relates to the field of image processing technology, and provides a method, device and computer equipment for determining the thickness of the transparent band, which can improve the analysis efficiency of the thickness of the transparent band. In the present application, a transparent band mask is segmented from a cell image; when the transparent band mask is completely segmented, the transparent band inner contour and the transparent band outer contour are detected, and the transparent band outer contour is sampled to obtain a plurality of outer contour sampling points; a connecting line between the plurality of outer contour sampling points and the center point of the transparent band outer contour is obtained, and based on the intersection of the connecting line and the transparent band inner contour, an inner contour sampling point corresponding to the outer contour sampling point is obtained; based on the distance between the outer contour sampling point and the corresponding inner contour sampling point, the thickness of the transparent band is determined.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, computer equipment, storage medium and computer program product for determining the thickness of a transparent band. Background Art

[0002] Oocytes are the key factor in determining the pregnancy rate of IVF-ET (In Vitro Fertilization and Embryo Transfer). The quality of oocytes can affect the fertilization rate, early embryo development, pregnancy maintenance and fetal development. The zona pellucida is the outer cortex of the oocyte. The thickness of the zona pellucida affects the fertilization effect of the oocyte. In addition, the thickness of the zona pellucida also affects the development of pronuclear embryos.

[0003] Currently, the method of determining the thickness of the zona pellucida is generally through manual marking and calculation, which is inefficient. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, storage medium and computer program product for determining the thickness of a transparent band in order to solve the above technical problems.

[0005] The present application provides a method for determining the thickness of a transparent zone, the method comprising:

[0006] A mask to segment the zona pellucida from the cell image;

[0007] When the mask of the transparent band is completely segmented, an inner contour of the transparent band and an outer contour of the transparent band are detected, and the outer contour of the transparent band is sampled to obtain a plurality of outer contour sampling points;

[0008] Acquire a connection line between a plurality of the outer contour sampling points and a center point of the outer contour of the transparent band, and obtain an inner contour sampling point corresponding to the outer contour sampling point based on the intersection of the connection line and the inner contour of the transparent band;

[0009] The thickness of the transparent band is determined based on the distance between the outer contour sampling point and the corresponding inner contour sampling point.

[0010] In one embodiment, the cell image may be an oocyte image or a pronuclear embryo image.

[0011] In one embodiment, before determining the thickness of the transparent band based on the distance between the outer contour sampling point and the corresponding inner contour sampling point, the method further includes:

[0012] When the mask of the transparent zone is not completely segmented, an incomplete transparent zone contour is detected;

[0013] Obtaining equations of straight lines between the center point of the incomplete transparent band outline and each azimuth angle in the azimuth angle set;

[0014] According to the two intersection points generated by the straight line equation and the incomplete transparent band contour, the outer contour sampling points and the inner contour sampling points corresponding to each other are determined.

[0015] In one embodiment, obtaining the center point of the incomplete transparent zone outline includes:

[0016] Acquire the zero-order spatial moment, the first-order spatial moment about the horizontal axis, and the first-order spatial moment about the vertical axis of the image area enclosed by the incomplete transparent band outline;

[0017] According to the ratio of the first-order spatial moment about the horizontal axis to the zero-order spatial moment, and the ratio of the first-order spatial moment about the vertical axis to the zero-order spatial moment, the center point of the incomplete transparent band contour is determined in the horizontal and vertical directions.

[0018] In one embodiment, the azimuths in the azimuth set are in the range of 0° to 360° and belong to an arithmetic progression.

[0019] In one embodiment, determining the thickness of the transparent band based on the distance between the outer contour sampling point and the corresponding inner contour sampling point includes:

[0020] Based on the outer contour sampling points and the corresponding inner contour sampling points, obtaining the distance between points;

[0021] According to the sampling order of forming the sampling points, the corresponding distances between the points are mapped to the coordinate system to obtain the waveform;

[0022] Based on the waveform, the thickness of the transparent zone is determined.

[0023] In one embodiment, determining the thickness of the transparent band according to the waveform includes:

[0024] Determining the thickest position on the transparent band according to the peak of the waveform and the sampling order corresponding to the peak;

[0025] The thinnest position on the transparent band is determined according to the trough of the waveform and the sampling order corresponding to the trough.

[0026] The present application provides a device for determining the thickness of a transparent band, the device comprising:

[0027] An image segmentation module is used to segment the transparent zone mask from the cell image;

[0028] A contour sampling module, used for detecting and obtaining an inner contour and an outer contour of the transparent band when the mask of the transparent band is completely segmented, and sampling the outer contour of the transparent band to obtain a plurality of outer contour sampling points;

[0029] The contour sampling module is further used to obtain a connecting line between a plurality of the outer contour sampling points and a center point of the outer contour of the transparent band, and obtain an inner contour sampling point corresponding to the outer contour sampling point based on the intersection of the connecting line and the inner contour of the transparent band;

[0030] The thickness determination module is used to determine the thickness of the transparent band based on the distance between the outer contour sampling point and the corresponding inner contour sampling point.

[0031] The present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the above method.

[0032] The present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to execute the above method.

[0033] The present application provides a computer program product, on which a computer program is stored, and the computer program is executed by a processor to perform the above method.

[0034] In the above-mentioned transparent band thickness determination method, device, computer equipment, storage medium and computer program product, a transparent band mask is segmented from a cell image. When the transparent band mask is completely segmented, the transparent band inner contour and the transparent band outer contour are detected, and the transparent band outer contour is sampled to obtain multiple outer contour sampling points. The connecting lines between each of the multiple outer contour sampling points and the center point of the transparent band outer contour are obtained. Based on the intersection of each of the connecting lines with the transparent band inner contour, the inner contour sampling points corresponding to the outer contour sampling points are obtained. Based on the distance between the mutually corresponding outer contour sampling points and the inner contour sampling points, the transparent band thickness is automatically determined without the need for manual marking and calculation, thereby improving efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A flow chart of a method for determining the thickness of a transparent band in one embodiment;

[0036] Figure 2 A network structure diagram of FCN in one embodiment;

[0037] FIG3( a ) is a schematic diagram of a transparent band mask that is completely segmented in one embodiment;

[0038] FIG3( b ) is a schematic diagram of contour sampling when the mask is completely segmented in one embodiment;

[0039] FIG4( a ) is a schematic diagram of a transparent band mask that is not completely segmented in one embodiment;

[0040] FIG4( b ) is a schematic diagram of contour sampling when the mask is not completely segmented in one embodiment;

[0041] Figure 5 is a waveform diagram of the thickness of the transparent band in one embodiment;

[0042] Figure 6 is a schematic diagram of a cell image analysis method in one embodiment;

[0043] Figure 7 A correlation diagram for analyzing the periovular distance in one embodiment;

[0044] Figure 8 A network structure diagram of an oocyte segmentation model in one embodiment;

[0045] Fig. 9 is a schematic diagram of a polar body in one embodiment;

[0046] FIG. 10( a ) is a diagram of an embryo with a smaller perivitelline space in one embodiment;

[0047] FIG. 10( b ) is a diagram of an embryo corresponding to a larger perivitelline space in one embodiment;

[0048] FIG. 10( c ) is a diagram illustrating an oocyte ROI corresponding to a smaller perivitelline space in one embodiment;

[0049] FIG. 10( d ) is an example of an oocyte ROI corresponding to a larger perivitelline space in one embodiment;

[0050] FIG. 10( e ) is a diagram of a perivitelline space corresponding to a smaller perivitelline space in one embodiment;

[0051] FIG10( f ) is a diagram of a perivitelline space corresponding to a larger perivitelline space in one embodiment;

[0052] Fig.11 is a structural block diagram of a transparent band thickness determination device in one embodiment;

[0053] Fig.12 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0056] The following is an introduction to the technical terms related to this application:

[0057] IVF-ET: (In Vitro Fertilization and Embryo Transfer) artificial insemination and in vitro fertilization-embryo transfer.

[0058] ZP: Zona Pellucida, the zona pellucida is a circle of unstructured, eosinophilic gelatin-like substance surrounding the oocyte, which is secreted by the oocyte and its surrounding follicular cells during the growth and development of the egg.

[0059] ART: Assisted Reproductive Technology, a technology that uses medical assistance to help infertile couples become pregnant.

[0060] Oocytes: Oocytes, oogonia that undergo meiosis during oogenesis.

[0061] ICSI: intracytoplasmic sperm injection, intracytoplasmic sperm injection technology, uses a micromanipulation system to inject a single sperm into the egg to fertilize it.

[0062] FCN: fully convolutional network, fully convolutional neural network, fully convolutional neural network, fully convolutional network (Fully Convolutional Networks, FCN) is a framework for image semantic segmentation.

[0063] The zona pellucida wraps around the oocyte. The thickness of the zona pellucida can not only be used to evaluate the quality of the oocyte, but also is an indicator for embryologists to choose treatment methods. The thickness is 10-31μm. The thickness of the zona pellucida is related to the fertilization effect of the oocyte. Therefore, the calculation of the oocyte helps embryologists provide reference values ​​for the analysis and treatment of embryos.

[0064] The method for determining the thickness of the zona pellucida provided in the present application comprises: Figure 1 The steps shown are:

[0065] Step S101, segmenting the transparent zone mask from the cell image.

[0066] The cell image can be an oocyte image or a pronuclear embryo image. Specifically, the cell image can be input into a trained zona pellucida segmentation model to perform semantic segmentation on the cell image to obtain a zona pellucida mask. The zona pellucida segmentation model can be constructed based on networks such as deeplabV3, Unet, UneXt, FCN, FCN-resnet50, and FCN-resnet101. In the embodiment of the present application, the zona pellucida segmentation model is based on Figure 2 The FCN network shown in the figure can accept input images (Input_image) of any size, and uses deconvolution to upsample the Feature_map of the last convolutional layer to restore it to the same size as the input image, so that a prediction (Pixelwise prediction) can be generated for each pixel, while retaining the original spatial information in the input image, and finally pixel-by-pixel classification is performed on the upsampled feature map.

[0067] In order to obtain the zona pellucida segmentation model, segmentation training can be performed. During segmentation training, data annotation tools can be used to select cell images for labeling, mark segmentation areas, and set classification labels. Labels can be divided into two categories: 0: background (background does not need to be labeled), 1: zona pellucida area. The labeled cell image dataset is divided into a training set and a validation set at a ratio of 3:1. The input raw data is planned and data enhanced, and then iterative training is performed to obtain the zona pellucida segmentation model, and then the zona pellucida mask is obtained based on the zona pellucida segmentation model.

[0068] Step S102, when the mask of the transparent band is completely segmented, the inner contour and the outer contour of the transparent band are detected and obtained, and the outer contour of the transparent band is sampled to obtain a plurality of outer contour sampling points.

[0069] This step can be implemented based on the findContours function in Opencv. The detection mode is set to RETR_TREE. All contours are extracted and the network structure is reconstructed. All contours are retrieved and the complete hierarchical structure of nested contours is reconstructed. The enumerator selection of the contour approximation method is: CHAIN_APPROX_SIMPLE. The findContours function is set to RETR_TREE mode in the detection mode. The mask of the zona pellucida is completely segmented, indicating that the zona pellucida is not blocked or missing in the cell image. At this time, the zona pellucida ring structure is in a closed loop state, and the inner and outer contours of the zona pellucida can be detected, as shown in Figure 3(a), where the outer contour of the zona pellucida can be recorded as ZP_Polygon_out and the inner contour of the zona pellucida can be recorded as ZP_Polygon_in.

[0070] Step S103, obtaining a connecting line between a plurality of outer contour sampling points and the center point of the outer contour of the transparent band, and obtaining inner contour sampling points corresponding to the outer contour sampling points based on the intersection of the connecting line and the inner contour of the transparent band.

[0071] When the inner and outer contours of the transparent band are detected, this step mainly samples the outer contour of the transparent band first, and then samples the inner contour of the transparent band based on the obtained outer contour sampling points to obtain inner contour sampling points.

[0072] Specifically, the outer contour of the transparent band can be extracted by the contour approximation method. The principle is to approximate the outer contour of the transparent band to another contour with fewer vertices so that the distance between them is less than or equal to the specified precision. This is achieved by approxPolyDP(). The second parameter is used for the precision of contour approximation, which indicates the maximum distance between the original contour and its approximated contour. The smaller the value, the more the approximated contour fits the original contour. The third parameter specifies whether the approximated contour is closed. Thus, the outer contour sampling point set is obtained, which can be set to: C_n.

[0073] Traverse the outer contour sampling points in the outer contour sampling point set C_n. When traversing to the i-th outer contour sampling point (C_i_nx, C_i_ny), the straight line equation between the outer contour sampling point (C_i_nx, C_i_ny) and the center point O_z of the transparent band outer contour can be calculated and recorded as L_a[i].

[0074] Calculate the intersection point (C_i_mx, C_i_my) of the straight line equation L_a[i] and the inner contour of the transparent band ZP_Polygon_in, use the intersection point as the inner contour sampling point, and store (C_i_mx, C_i_my) in C_m.

[0075] It should be noted that the above outer contour sampling points (C_i_nx, C_i_ny) and inner contour sampling points (C_i_mx, C_i_my) correspond to each other.

[0076] After traversing the outer contour sampling points in the outer contour sampling point set C_n, the inner contour sampling points corresponding to each outer contour sampling point can be determined, as shown in FIG3(b).

[0077] Step S104, determining the thickness of the transparent band based on the distance between the outer contour sampling point and the corresponding inner contour sampling point.

[0078] For example, after obtaining the i-th outer contour sampling point (C_i_nx, C_i_ny) and its corresponding inner contour sampling point (C_i_mx, C_i_my), the distance between the two sampling points can be obtained, which can be called the point-to-point distance and can be recorded as k__nm[i]. According to {k__nm[1], k__nm[2], ..., k__nm[i]}, the thickness of the transparent zone is determined.

[0079] In the above-mentioned method for determining the thickness of the transparent band, the mask of the transparent band is segmented from the cell image. When the mask of the transparent band is completely segmented, the inner contour and the outer contour of the transparent band are detected, and the outer contour of the transparent band is sampled to obtain multiple outer contour sampling points. The connecting lines between the multiple outer contour sampling points and the center point of the outer contour of the transparent band are obtained. Based on the intersection of each connecting line with the inner contour of the transparent band, the inner contour sampling points corresponding to the outer contour sampling points are obtained. Based on the distance between the corresponding outer contour sampling points and the inner contour sampling points, the thickness of the transparent band is automatically determined without manual marking and calculation, thereby improving efficiency.

[0080] In one embodiment, before determining the thickness of the transparent band based on the distance between the outer contour sampling point and the corresponding inner contour sampling point, the method provided by the present application also includes: when the mask of the transparent band is not completely segmented, detecting an incomplete transparent band contour; obtaining straight line equations between the center point of the incomplete transparent band contour and each azimuth angle in the azimuth angle set; and determining the corresponding outer contour sampling points and inner contour sampling points based on the two intersection points generated by the straight line equation and the incomplete transparent band contour.

[0081] This embodiment can be implemented based on the findContours function in Opencv, with the detection mode set to RETR_TREE, extracting all contours and reconstructing the network structure, retrieving all contours and reconstructing the complete hierarchical structure of nested contours, and selecting the enumerator of the contour approximation method: CHAIN_APPROX_SIMPLE. The findContours function is set to RETR_TREE mode in the detection mode. The mask of the zona pellucida is not completely segmented, indicating that the zona pellucida is blocked or missing in the cell image. At this time, the ring structure of the zona pellucida is destroyed, and only one contour can be detected, which is called an incomplete zona pellucida contour, as shown in Figure 4(a), where the incomplete zona pellucida contour can be recorded as ZP_Polygon.

[0082] When a contour is detected, the outer contour sampling points and their corresponding inner contour sampling points can be extracted from the incomplete transparent band contour by using the azimuth angle method.

[0083] When the azimuth angle set An_s is set to {An_s[1], An_s[2], …, An_s[i]}, when traversing to the i-th azimuth angle An_s[i], the equation of the straight line between the azimuth angle and the center point of the incomplete transparent zone outline ZP_Polygon can be obtained, which is recorded as L_b[i]. Next, the intersection of the straight line equation L_b[i] and the incomplete transparent band outline ZP_Polygon is calculated. At this time, the straight line equation L_b[i] will generate two intersection points with the incomplete transparent band outline ZP_Polygon, as shown in Figure 4(b). The two intersection points are the corresponding outer contour sampling points and inner contour sampling points. Among them, among the two intersection points, the center point close to the incomplete transparent band outline ZP_Polygon is the inner contour sampling point, and is recorded as (C_i_mx, C_i_my), and is added to the inner contour sampling point set C_m. The center point far from the incomplete transparent band outline ZP_Polygon is the outer contour sampling point, and is recorded as (C_i_nx, C_i_ny), and is added to the outer contour sampling point set C_n.

[0084] In this embodiment, a corresponding contour sampling method is selected according to whether the mask of the transparent band is completely segmented, which further adapts the functions that need to be implemented by the computer device and the universality of the algorithm.

[0085] Furthermore, obtaining the center point of the incomplete transparent band outline includes: obtaining the zero-order spatial moment of the image area enclosed by the incomplete transparent band outline, the first-order spatial moment about the horizontal axis, and the first-order spatial moment about the vertical axis; determining the center point of the incomplete transparent band outline in the horizontal and vertical directions according to the ratio of the first-order spatial moment about the horizontal axis to the zero-order spatial moment, and the ratio of the first-order spatial moment about the vertical axis to the zero-order spatial moment.

[0086] In this embodiment, the image area surrounded by the incomplete transparent zone outline can be determined, and the zero-order spatial moment m of the image area can be calculated. 00 , the first-order spatial moment m about the horizontal axis 01 and the first-order spatial moment m about the vertical axis 10 .

[0087] The calculation formula of spatial moment is: Where array(x, y) is the pixel value at pixel (x, y). When i=j=0, it corresponds to the zero-order spatial moment; when i=0 and j=1, it corresponds to the first-order spatial moment about the horizontal axis; when i=1 and j=0, it corresponds to the first-order spatial moment about the vertical axis.

[0088] For a binary image, the zero-order spatial moment m 00 It is the area of ​​the image region enclosed by the outline of the incomplete transparent zone.

[0089] Next, we can obtain the first-order spatial moment m about the horizontal axis 01 and the zero-order spatial moment m 00 The ratio of the vertical axis to obtain the first-order spatial moment m 10 The ratio of the zero-order spatial moment m 00 , thus obtaining the center point O of the incomplete transparent zone outline T Coordinates in,

[0090] In the above embodiment, from the perspective of image space moment, according to the ratio of the first-order spatial moment to the zero-order spatial moment about the horizontal axis, and the ratio of the first-order spatial moment to the zero-order spatial moment about the vertical axis, the center point of the incomplete transparent band contour is accurately determined in the horizontal and vertical directions.

[0091] Furthermore, the azimuths in the azimuth set are in the range of 0° to 360°, which belongs to an arithmetic progression. That is, in the azimuth set An_s, the difference between two adjacent azimuths is fixed, which may be 45°. The corresponding azimuth set An_s may be {0, 45, 90, 135, 180, 225, 270, 315}.

[0092] In the above embodiment, the azimuth angle set belongs to an arithmetic progression, which achieves uniform sampling of the incomplete transparent zone contour and ensures the accuracy of the transparent zone thickness determination.

[0093] In one embodiment, the thickness of the transparent band is determined based on the distance between the outer contour sampling point and the corresponding inner contour sampling point, including: obtaining the distance between points based on the outer contour sampling point and the corresponding inner contour sampling point; mapping the corresponding distance between points to a coordinate system according to the sampling order of forming the sampling points to obtain a waveform; and determining the thickness of the transparent band according to the waveform.

[0094] For example, after obtaining the corresponding outer contour sampling points (C_i_nx, C_i_ny) and inner contour sampling points (C_i_mx, C_i_my), the distance between the two sampling points can be obtained, which can be called the point distance, denoted as k__nm[i]. The sampling order of the above two sampling points is i. According to the sampling order, the point distance k__nm[i] is mapped to the coordinate system to obtain a waveform, such as Figure 5 As shown, and based on the waveform, determine the thickness of the transparent zone.

[0095] Furthermore, the thickness of the transparent band is determined based on the waveform, including: determining the thickest position on the transparent band based on the peaks of the waveform and the sampling order corresponding to the peaks; determining the thinnest position on the transparent band based on the troughs of the waveform and the sampling order corresponding to the troughs.

[0096] In this embodiment, the "thickest" and "thinnest" of the zona pellucida are located. The thicker position on the zona pellucida can be determined by the peak, and the thinner position on the zona pellucida can be determined by the trough. During laser thin filming of the embryo, the area where the zona pellucida is thinner is automatically analyzed and located, providing a reference rupture point for the operating doctor.

[0097] When characterizing embryos, not only can the thickness of the zona pellucida be analyzed, but also the distance of the perivitelline space, the roundness of the cells, and the polar bodies, e.g. Figure 6 Measuring the distance of the perivitelline space specifically includes the following steps:

[0098] Step S1, determining the inner contour area of ​​the transparent zone and the cytoplasm contour area in the cell image.

[0099] The cell image can be an oocyte image or a pronuclear embryo image. The zona pellucida mask is extracted from the cell image through the zona pellucida segmentation model to determine the inner contour area of ​​the zona pellucida in the cell image, such as Figure 7 As shown in (a); through the cell segmentation model, the cell mask is extracted from the cell image to determine the cytoplasm contour area in the cell image, such as Figure 7 As shown in (b); when the cell image is an oocyte image, the cytoplasm contour area is the contour area of ​​the oocyte, and when the cell image is a pronuclear embryo image, the cytoplasm contour area is the contour area of ​​the pronuclear embryo.

[0100] When the cell image is an oocyte image, the cell segmentation model is mainly used to segment the oocyte in the cell image, and when the cell image is a prokaryotic embryo image, the cell segmentation model is mainly used to segment the prokaryotic embryo in the cell image, and the cell segmentation model can adopt an instance segmentation network, such as Solov2, BCnet, maskRCNN, etc. In one embodiment of the present application, maskRCNN is used to perform instance segmentation on the oocyte of the cell image. MaskRCNN is a framework based on Faster-RCNN. After the basic feature network, a fully connected segmentation network is added, and the original two tasks (classification + regression) are changed into three tasks (classification + regression + segmentation). Mask R-CNN adopts the same two stages as Faster R-CNN, and each proposal box of FastRCNN is semantically segmented using FCN. The backbone network of MaskRCNN is composed of ResNet50 and FPN, and the residual network structure ensures that in the process of stacking the network again, the network will not degenerate, and FPN simultaneously utilizes the spatial information of the underlying feature map and the semantic information of the high-level feature map. Through the training of the MaskRCNN network model, the original cell image is input, and the cell mask C_mask and cell center point O_c of the oocyte or pronuclear embryo can be obtained through cell segmentation model inference. At the same time, the cytoplasm contour C_p can be extracted based on the cell mask C_mask of the oocyte or pronuclear embryo, such as Figure 8 shown.

[0101] For instance segmentation of cells, Solov2, BCnet, maskRCNN and other methods can be used for testing. Mask-RCNN is selected. Mask-RCNN can not only handle object detection problems, but also semantic segmentation problems. At the same time, for the problem that the ROI pooling layer of Faster RCNN is quantized twice and the feature map cannot be accurately aligned with the original pixels, Mask RCNN replaces ROI pooling with ROIAlign. ROIAlign can align pixels and meet the accuracy requirements of image semantic segmentation.

[0102] Step S2, obtaining the non-overlapping area between the inner contour area of ​​the zona pellucida and the contour area of ​​the cytoplasm.

[0103] The obtained Figure 7 The inner contour area of ​​the transparent band shown in (a) is recorded as Region1. Figure 7 The cytoplasm outline region shown in (b) is denoted as Region2. The difference between Region1 and Region2 is calculated to obtain the non-overlapping region between Region1 and Region2, denoted as Uor, as follows: Figure 7(c) as shown.

[0104] Step S3: when there are multiple non-overlapping regions, select the region with the largest area as the target non-overlapping region, and determine the inclination angle of the minimum circumscribed rectangle of the target non-overlapping region.

[0105] It can be seen Figure 7 There are multiple non-overlapping regions in (c), among which there are some non-overlapping regions with scattered and independent boundaries. In order to remove these non-overlapping regions, the non-overlapping region with the largest area can be selected from multiple non-overlapping regions to obtain the following: Figure 7 (d) shows the target non-overlapping region.

[0106] Next, the minimum bounding rectangle of the target non-overlapping area is determined, and the inclination angle of the minimum bounding rectangle is determined, where the inclination angle is determined by the slope of the minimum bounding rectangle.

[0107] Step S4, obtaining a straight line equation according to the tilt angle and the cell center point;

[0108] When the cell image is an oocyte image, the center point of the cell is the center point of the oocyte; when the cell image is a pronuclear embryo image, the center point of the cell is the center point of the pronuclear embryo.

[0109] Step S5, calculating the intersection points of the straight line equation with the cytoplasm contour and the inner contour of the zona pellucida, and obtaining the distance of the perivitelline space according to the distance between the intersection points.

[0110] Among them, the above straight line equation can be in the form of point direction form, that is: (u,v) is the direction vector of the line equation. After obtaining the line equation, the intersection of the line equation and the target non-overlapping area can be determined. There are two intersection points. The intersection point close to the center of the cell is: the intersection point of the point-direction line equation and the cytoplasm contour, denoted as P1(X p1 , Y p1 ); the intersection point far from the center of the cell is the intersection point of the point-direction straight line equation and the inner contour of the zona pellucida, denoted as P2(X p2 , Y p2 ). Then, the distance between the two intersection points is As the distance of the perivitelline space, Figure 7 (e) as shown.

[0111] Furthermore, the reference interval of the perivitelline distance is (8.5-12.3 μm). When the measured perivitelline distance is greater than 15 μm (the threshold setting can be adjusted based on clinical experience and embryonic development consensus), it can be determined that the perivitelline distance is too large.

[0112] In the above-mentioned cell image analysis method, the inner contour area of ​​the transparent zone and the cytoplasm contour area in the cell image are determined; the non-overlapping area between the inner contour area of ​​the transparent zone and the cytoplasm contour area is determined; when there are multiple non-overlapping areas, the area with the largest area is selected as the target non-overlapping area, and the inclination angle of the minimum circumscribed rectangle of the target non-overlapping area is determined; the equation of the straight line is obtained according to the inclination angle and the center point of the oocyte; the intersection points of the straight line equation with the cytoplasm contour and the inner contour of the transparent zone are calculated, and the distance of the perivitelline space is automatically obtained according to the distance between the intersection points, without manual intervention, thereby improving the efficiency of obtaining the distance of the perivitelline space.

[0113] In one embodiment, before calculating the intersection points of the straight line equation with the cytoplasm contour and the inner contour of the zona pellucida, the method provided in the present application further includes: segmenting the cell image to obtain a cell mask; and extracting the contour of the cell mask to obtain a cytoplasm contour.

[0114] Specifically, the cell segmentation model can be used to segment the cell image to obtain a cell mask, and then the contour of the cell mask can be extracted to obtain the cytoplasm contour.

[0115] In the above embodiment, the cell image is first segmented to obtain a cell mask; then the contour of the cell mask is extracted to obtain the cytoplasm contour, so that the contour is more accurate.

[0116] In one embodiment, the feature analysis of the embryo may further include: calculating the roundness of the cell, specifically including the following steps: obtaining the circumference and area of ​​the cell according to the cytoplasm contour; and obtaining the roundness of the cell based on the circumference and area of ​​the cell.

[0117] This embodiment mainly obtains the cell roundness by measuring the cell roundness R i To determine whether the embryo has abnormal cell morphology.

[0118] When calculating the roundness of a cell, the cell circumference L can be obtained i and area S i , using the formula The roundness of the cell is obtained. The roundness obtained by this method is between [0,1]. The closer the calculated value is to 1, the greater the roundness of the cell.

[0119] In the above embodiment, the roundness of the cell is obtained through the perimeter and area of ​​the cell to assist in detecting whether the embryo has abnormal cell morphology.

[0120] Regarding the roundness of cells, the present application provides another implementation, which specifically includes the following steps: obtaining the average distance from contour pixels to the center of the cell; the contour pixels are pixels on the cytoplasm contour; obtaining the deviation between the distance from the contour pixels to the center of the cell and the average distance; and obtaining the roundness of the cell based on the relationship between the average distance and the deviation.

[0121] The cell center point is denoted as p; the contour pixel point is denoted as p_i; F is the area of ​​the image region enclosed by the cytoplasm contour, which can be represented by the number of pixels in the cytoplasm contour.

[0122]

[0123]

[0124]

[0125] Among them, Dis represents the average distance from the contour pixel to the cell center, and Sigma represents the deviation between the distance from the contour pixel to the cell center and the average distance. Roundness represents the relationship between the average distance and the deviation, which can be used to measure the roundness of the cell. The roundness evaluation result obtained by Roundness is between [0,1]. The closer the calculated value is to 1, the greater the roundness of the cell.

[0126] In the above embodiment, the average distance from the contour pixel point to the cell center point is obtained, and the deviation between the distance from the contour pixel point to the cell center point and the average distance is obtained. The roundness of the cell is obtained based on the relationship between the average distance and the deviation. The roundness of the cell can be obtained without the aid of the cell circumference and area, thereby improving the flexibility of the calculation.

[0127] In one embodiment, the characteristic analysis of the embryo may also include: polar body detection; the purpose of polar body detection is to find the location and number of polar bodies and measure the diameter of polar bodies, and to judge whether the polar bodies are too large by the diameter of polar bodies, so as to reduce the impact of abnormal giant polar bodies on embryonic development. In addition, the expulsion of polar bodies is also an important sign of the embryonic development process. Usually, the expulsion of the first polar body is regarded as a sign of oocyte maturity, while the expulsion of the second polar body represents the successful fertilization of the embryo.

[0128] The method provided in the present application also includes: inputting the cell image into the target detection network to obtain a polar body prediction frame; and predicting the diameter of the polar body according to the size of the polar body prediction frame.

[0129] The target detection network can include yolov5 and Faster-RCNN. The detection and counting of oocyte polar bodies can be realized through the target detection network, such as Fig. 9As shown, 901 represents the rectangular box when marked, and 902 represents the polar body identified by reasoning. Through the reasoning of the target detection network, the polar body prediction box bbox[x1,y1,x2,y2] can be obtained, and then the diameter D_pa of the polar body can be calculated according to the formula D_pa=(abs(x2-x1)+abs(y2-y1)) / 2.

[0130] Among them, for polar body detection, this embodiment selects yolov5 and fastercnn for testing, and the test results show that the single-stage network model yolov5 is more suitable for polar body detection needs.

[0131] In the above embodiment, the diameter of the polar body is automatically analyzed and predicted through the target detection network without manual intervention, thereby improving processing efficiency.

[0132] In one embodiment, the characteristic analysis of the embryo may also include: determining the thickness of the zona pellucida. The thickness of the zona pellucida can not only be used to evaluate the quality of the oocyte, but also is an indicator for embryologists to choose treatment methods. The thickness is 10-31 μm. The thickness of the zona pellucida is related to the fertilization effect of the oocyte, so it helps embryologists provide reference values ​​for embryo analysis and treatment.

[0133] When the method provided in the present application is applied to an actual scene, the cell image can be obtained from an embryo time-lapse incubator. The relevant data of the actual scene can be found in Table 1.

[0134]

[0135] Table 1

[0136] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0137] In one embodiment, Fig.11 As shown, a transparent band thickness determination device is provided, comprising:

[0138] An image segmentation module 1101 is used to segment the transparent zone mask from the cell image;

[0139] The contour sampling module 1102 is used for detecting the inner contour and the outer contour of the transparent band when the mask of the transparent band is completely segmented, and sampling the outer contour of the transparent band to obtain a plurality of outer contour sampling points;

[0140] The contour sampling module 1102 is further used to obtain a connection line between a plurality of the outer contour sampling points and a center point of the outer contour of the transparent band, and obtain an inner contour sampling point corresponding to the outer contour sampling point based on the intersection of the connection line and the inner contour of the transparent band;

[0141] The thickness determination module 1103 is used to determine the thickness of the transparent band based on the distance between the outer contour sampling point and the corresponding inner contour sampling point.

[0142] In one embodiment, the contour sampling module 1102 is also used to detect an incomplete transparent band contour when the mask of the transparent band is not completely segmented; obtain the straight line equations between the center point of the incomplete transparent band contour and each azimuth angle in the azimuth angle set; and determine the corresponding outer contour sampling points and inner contour sampling points based on the two intersection points generated by the straight line equation and the incomplete transparent band contour.

[0143] In one embodiment, the device also includes a center point acquisition module, which is used to obtain the zero-order spatial moment, the first-order spatial moment about the horizontal axis, and the first-order spatial moment about the vertical axis of the image area enclosed by the incomplete transparent band outline; according to the ratio of the first-order spatial moment about the horizontal axis to the zero-order spatial moment, and the ratio of the first-order spatial moment about the vertical axis to the zero-order spatial moment, the center point of the incomplete transparent band outline is determined in the horizontal and vertical directions.

[0144] In one embodiment, the azimuths in the azimuth set are in the range of 0° to 360° and belong to an arithmetic progression.

[0145] In one embodiment, the thickness determination module 1103 is also used to obtain the distance between points based on the outer contour sampling points and the corresponding inner contour sampling points; map the corresponding distance between points to a coordinate system according to the sampling order of forming the sampling points to obtain a waveform; and determine the thickness of the transparent band based on the waveform.

[0146] In one embodiment, the thickness determination module 1103 is also used to determine the thickest position on the transparent band based on the peaks in the waveform and the sampling order corresponding to the peaks; and to determine the thinnest position on the transparent band based on the troughs in the waveform and the sampling order corresponding to the troughs.

[0147] The specific definition of the transparent band thickness determination device can refer to the definition of the transparent band thickness determination method above, which will not be repeated here. Each module in the above transparent band thickness determination device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0148] In one embodiment, a computer device is provided, whose internal structure diagram can be as follows: Fig.12 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer device also includes an input and output interface, which is a connection circuit for exchanging information between the processor and the external device. They are connected to the processor through a bus, referred to as an I / O interface. When the computer program is executed by the processor, a method for determining the thickness of a transparent band is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0149] Those skilled in the art will understand that Fig.12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0150] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above-mentioned various method embodiments when executing the computer program.

[0151] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0152] In one embodiment, a computer program product is provided, on which a computer program is stored, and the computer program is used by a processor to execute the steps in the above-mentioned various method embodiments.

[0153] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0154] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the above-mentioned computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0155] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0156] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for determining the thickness of a transparent zone, characterized in that: The method comprises: A mask to segment the zona pellucida from the cell image; When the mask of the transparent band is completely segmented, an inner contour of the transparent band and an outer contour of the transparent band are detected, and the outer contour of the transparent band is sampled to obtain a plurality of outer contour sampling points; Acquire a connection line between a plurality of the outer contour sampling points and a center point of the outer contour of the transparent band, and obtain an inner contour sampling point corresponding to the outer contour sampling point based on the intersection of the connection line and the inner contour of the transparent band; When the mask of the transparent zone is not completely segmented, an incomplete transparent zone contour is detected; Acquire the zero-order spatial moment, the first-order spatial moment about the horizontal axis, and the first-order spatial moment about the vertical axis of the image area enclosed by the incomplete transparent band outline; Determine the center point of the contour of the incomplete transparent zone in the horizontal and vertical directions according to the ratio of the first-order spatial moment about the horizontal axis to the zero-order spatial moment, and the ratio of the first-order spatial moment about the vertical axis to the zero-order spatial moment; Obtaining equations of straight lines between the center point of the incomplete transparent band outline and each azimuth angle in the azimuth angle set; Determine corresponding outer contour sampling points and inner contour sampling points according to two intersection points generated by the straight line equation and the incomplete transparent band contour; The thickness of the transparent band is determined based on the distance between the outer contour sampling point and the corresponding inner contour sampling point.

2. The method according to claim 1, characterized in that The azimuths in the azimuth set are in the range of 0° to 360°, and belong to an arithmetic progression.

3. The method according to claim 1, characterized in that Determining the thickness of the transparent band based on the distance between the outer contour sampling point and the corresponding inner contour sampling point includes: Based on the outer contour sampling points and the corresponding inner contour sampling points, obtaining the distance between points; According to the sampling order of forming the sampling points, the corresponding distances between the points are mapped to the coordinate system to obtain the waveform; Based on the waveform, the thickness of the transparent zone is determined.

4. The method according to claim 3, characterized in that: Determining the thickness of the transparent zone according to the waveform includes: Determining the thickest position on the transparent band according to the peak of the waveform and the sampling order corresponding to the peak; The thinnest position on the transparent band is determined according to the trough of the waveform and the sampling order corresponding to the trough.

5. A transparent band thickness determination device, characterized in that: The device comprises: An image segmentation module is used to segment the transparent zone mask from the cell image; A contour sampling module, used for detecting and obtaining an inner contour and an outer contour of the transparent band when the mask of the transparent band is completely segmented, and sampling the outer contour of the transparent band to obtain a plurality of outer contour sampling points; The contour sampling module is further used to obtain a connecting line between a plurality of the outer contour sampling points and a center point of the outer contour of the transparent band, and obtain an inner contour sampling point corresponding to the outer contour sampling point based on the intersection of the connecting line and the inner contour of the transparent band; The contour sampling module is further used to detect an incomplete transparent band contour when the mask of the transparent band is not completely segmented, obtain the straight line equations between the center point of the incomplete transparent band contour and each azimuth angle in the azimuth angle set, and determine the corresponding outer contour sampling points and inner contour sampling points according to two intersection points generated by the straight line equation and the incomplete transparent band contour; The device further comprises a center point acquisition module, which is used to acquire the zero-order spatial moment, the first-order spatial moment about the horizontal axis, and the first-order spatial moment about the vertical axis of the image area enclosed by the incomplete transparent band outline, and determine the center point of the incomplete transparent band outline in the horizontal and vertical directions according to the ratio of the first-order spatial moment about the horizontal axis to the zero-order spatial moment, and the ratio of the first-order spatial moment about the vertical axis to the zero-order spatial moment; The thickness determination module is used to determine the thickness of the transparent band based on the distance between the outer contour sampling point and the corresponding inner contour sampling point.

6. The device according to claim 5, characterized in that The azimuths in the azimuth set are in the range of 0° to 360°, and belong to an arithmetic progression.

7. The device according to claim 5, characterized in that The thickness determination module is further used for: Based on the outer contour sampling points and the corresponding inner contour sampling points, obtaining the distance between points; According to the sampling order of forming the sampling points, the corresponding distances between the points are mapped to the coordinate system to obtain the waveform; Based on the waveform, the thickness of the transparent zone is determined.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images