Method and device for removing color block impurities of human sperm image

Abstract

The invention relates to a sperm image processing technology, and discloses a human sperm image color block impurity removal method and device, and the method comprises the steps: collecting a sperm image processed by a dyeing reagent; using a pre-trained instance segmentation model to perform segmentation processing on a sperm head, an acrosome region and an acrosome posterior region in the sperm image, and performing post-processing and matching assembly on a segmentation result to obtain segmentation identification information of all single independent sperms in the image; and searching for color block impurities in the segmentation identification result, and deleting the color block impurities to obtain a sperm segmentation result without the color block impurities. The method has the advantages of being simple and high in operation effect, and is very suitable for being used on the ground in industrial scenes. Moreover, the color block impurities which are segmented and identified as sperms in the sperm image can be accurately and effectively deleted by the method for filtering the color block impurities, and the final sperm image segmentation identification result has no color block impurities, so that the accuracy of computer-aided sperm morphological analysis can be improved.

Description

Technical Field

[0001] The invention relates to sperm image processing technology, and in particular to a method and device for removing color block impurities from human sperm images. Background Art

[0002] The sperm image in this patent refers to the microscopic image obtained by scanning human semen with a high-power microscope after staining it with a staining reagent in the human sperm morphology analysis scenario. For sperm images, the segmentation algorithm is used to segment and match the outline of the sperm, and the information of all independent sperm in the sperm image is obtained. Color block impurities are very similar to the shape of the sperm head in appearance, and are mistakenly identified as independent sperm by the segmentation algorithm. This will affect the accuracy of computer-assisted sperm morphology analysis.

[0003] There is a problem of color block impurities in the results of segmentation and recognition of sperm images, and there is no solution to this problem in the prior art. How to identify and filter out color block impurities to improve the accuracy of human sperm morphology analysis is a technical problem that needs to be solved. Summary of the invention

[0004] The present invention aims at the problem of color block impurities in sperm segmentation images in the prior art and provides a method and device for removing color block impurities from human sperm images.

[0005] In order to solve the above technical problems, the present invention is solved by the following technical solutions:

[0006] A method for removing color block impurities from a human sperm image, the method comprising:

[0007] S1. Collect sperm images after being treated with staining reagents;

[0008] S2. Use the pre-trained instance segmentation model to segment the sperm head, acrosome region, and post-acrosomal region in the sperm image, and post-process and match the segmentation results to obtain the information of all independent sperm in the sperm image. The information of a single independent sperm includes three part contours and the confidence level of the corresponding contours; the three parts include the sperm head contour, acrosome region contour, and post-acrosomal region contour.

[0009] S3. Use the sperm and color block impurity classification algorithm to find the color block impurities in the independent sperm in the sperm image and delete them, so as to obtain a sperm segmentation result without color block impurities.

[0010] The method for removing color block impurities from human sperm images is based on two prior knowledge. The two prior knowledge are explained as follows.

[0011] Prior knowledge 1: The pixel distribution of color block impurities is obviously different from the normal sperm pixel distribution.

[0012] Prior knowledge 2: The number of color block impurities in all independent sperm obtained after sperm image segmentation, post-processing, matching and assembly is far less than the normal number of sperm.

[0013] The sperm and color block impurity classification algorithm in the sperm image color block impurity removal method includes the following steps:

[0014] Based on the information of all the individual sperms in the sperm image and the difference measurement between any two sperm heads, a hierarchical clustering algorithm is used to divide all the individual sperms in the sperm image into at most two clusters, one is the sperm cluster and the other is the impurity cluster. These two clusters are the first classification results of sperm and color block impurities.

[0015] Based on the first classification results of sperm and color block impurities and the confidence level in the sperm head segmentation results of sperm images, a comprehensive classification index is calculated. The sperm and color block impurities are classified using this comprehensive index to obtain the second classification results of sperm and color block impurities.

[0016] Preferably, the difference measurement between any two sperm heads in the sperm and color block impurity classification algorithm includes:

[0017] Traverse all the individual sperms that have been segmented, matched, and assembled in a sperm image; for every two independent sperms i and j, calculate the histograms of the three channels B, G, and R within the outline of the sperm head, hist i,B , hist i,G , hist i,R , hist j,B , hist j,G , hist j,R ; Among them, hist i,B is the B channel histogram within the outline of sperm head i, hist i,G is the G channel histogram within the sperm head contour, hist i,R is the R channel histogram within the sperm head contour, hist j,B is the B channel histogram within the sperm j head contour, hist j,G is the G channel histogram within the sperm head contour, hist j,R is the R channel histogram within the outline of the sperm j head;

[0018] Use the histogram comparison index to measure the matching degree d(H1,H2) of the two histograms H1 and H2, and the matching degree match between sperm i and sperm j i,j :

[0019]

[0020] The difference measure of the two sperm heads is calculated;

[0021] If the histogram comparison index is proportional to the degree of histogram matching, then the difference between the heads of two sperm is head_difference ij , head_difference ij =1-norm(match i,j );

[0022] If the histogram comparison index is inversely proportional to the degree of histogram matching, then the difference between the heads of two sperm is head_difference ij , head_difference ij =norm(match i,j );

[0023] Among them, norm is a function that normalizes the degree of histogram matching.

[0024] Preferably, the histogram comparison indicators include correlation coefficient, chi-square comparison, crossover, Bhattacharyya distance, Hellinger distance, alternate chi-square comparison and KL divergence.

[0025] As a preference, the histogram matching degree, the normalization function norm, is defined as follows:

[0026]

[0027] Among them, x min is the minimum value of the matching degree between any two independent sperms in a sperm image; x max is the maximum value of the matching degree between any two independent sperms in a sperm image;

[0028] Preferably, in the method for removing color block impurities from human sperm images, the calculation of the comprehensive classification index and the use of the calculation classification include:

[0029] Calculate the median head_difference_median of the difference measure between the sperm in the impurity cluster and all the sperm in the normal cluster in the first classification result; the difference measure between a sperm in the impurity cluster and all the sperm in the normal cluster constitutes a set of data, which are arranged in order. The number in the middle position is the median head_difference_median of the difference measure between the sperm in the impurity cluster and all the sperm in the normal cluster;

[0030] Calculate the average confidence level of the sperm segmentation results in the impurity cluster in the first classification result, that is, the average head_confidence_avg of the confidence level of the sperm head contour, the confidence level of the acrosome region contour, and the confidence level of the post-acrosomal region contour inferred by the segmentation algorithm; if the acrosome region and post-acrosomal region contour of a sperm do not exist, the corresponding contour is not considered when calculating head_confidence_avg; calculate the average head_confidence_avg of the confidence level of the sperm head contour, the confidence level of the acrosome region contour, and the confidence level of the post-acrosomal region contour inferred by the segmentation algorithm i ;

[0031]

[0032] Wherein, i represents the subscript number of the independent sperm in the impurity cluster in the first classification result;

[0033] Calculate the comprehensive indicator impurity_confidence, which is the comprehensive confidence level of whether the sperm in the impurity cluster is impurity;

[0034]

[0035] For the sperm in the impurity cluster in the first classification result, the preset impurity confidence threshold impurity_threshold is used. When impurity_confidence ≥ impurity_threshold, it is classified as impurity. Otherwise, it is classified as normal sperm, and the second classification result of sperm and color block impurity is obtained.

[0036]

[0037] In order to solve the above technical problems, the present invention also provides a human sperm image processing device, which comprises:

[0038] An image acquisition module, the image acquisition module is used to acquire sperm images after being treated with a staining reagent;

[0039] The segmentation and recognition module uses the pre-trained instance segmentation model to segment the sperm head, acrosome region, and post-acrosomal region in the sperm image, and post-processes and assembles the segmentation results to obtain the information of all single independent sperm in the sperm image. The single independent sperm information includes the three part contours and confidence levels of the single independent sperm; the three parts include the sperm head contour, acrosome region contour, and post-acrosomal region contour;

[0040] The sperm and color block impurity classification module searches for color block impurities in independent sperm in the sperm image and deletes them, thereby obtaining a sperm segmentation result without color block impurities.

[0041] In order to solve the above technical problems, the present invention also provides a computer device, which includes at least one processor and at least one memory, and the at least one memory is used to store at least one computer program; when the at least one computer program is executed by the at least one processor, the at least one processor implements the method for removing color block impurities in a human sperm image.

[0042] In order to solve the above technical problem, the present invention also provides a storage medium, wherein the storage medium stores processor executable instructions, and when the executable instructions are executed by the processor, they are used to execute the method for removing color block impurities from human sperm images.

[0043] The present invention has significant technical effects due to the adoption of the above technical solution:

[0044] The method for removing color block impurities from human sperm images proposed in the present invention improves the accuracy of removing color block impurities from human sperm images, is conducive to realizing automated segmentation and analysis of human sperm morphology, indirectly reduces the mechanical and repetitive workload of doctors, improves the work efficiency of human sperm morphological analysis, and reduces the waste of social resources.

[0045] The method of the present invention has the advantages of being simple and having high operating effects, and is very suitable for use in industrial scenarios. In addition, the method of filtering color block impurities can accurately and effectively delete the color block impurities that are segmented and identified as sperm in the sperm image, and the final sperm image segmentation and recognition result has no color block impurities, which is conducive to improving the accuracy of computer-assisted sperm morphology analysis and ultimately outputting more accurate and reliable human sperm morphology analysis conclusions. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Flow chart of the method of the present invention;

[0047] Figure 2 A schematic diagram of the structure of human sperm of the present invention;

[0048] Figure 3 is the original image of the verification color block impurity filtering of the present invention;

[0049] Figure 4 This is an image of the effect of filtering impurities of the present invention;

[0050] Figure 5 It is a flow chart of sperm image extraction of the present invention. DETAILED DESCRIPTION

[0051] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0052] Example 1

[0053] A method for removing color block impurities from a human sperm image comprises the following steps:

[0054] S1. Collect sperm images after being treated with staining reagents;

[0055] S2. Use the pre-trained instance segmentation model to segment the sperm head, acrosome region, and post-acrosomal region in the sperm image, and post-process and match the segmentation results to obtain the information of all independent sperm in the sperm image. The information of a single independent sperm includes three part contours and the confidence level of the corresponding contours; the three parts include the sperm head contour, acrosome region contour, and post-acrosomal region contour; S3. Find the color block impurities in the segmentation and recognition results and delete them.

[0056] The algorithm for finding color block impurities is based on two prior knowledge. For all the individual sperm obtained, the confidence level of the instance is output based on the difference in pixel distribution and the segmentation model inference output, and it is determined whether a sperm to be judged is impurity. The two prior knowledge are explained as follows:

[0057] Prior knowledge 1: The pixel distribution of color block impurities is obviously different from the normal sperm pixel distribution.

[0058] Prior knowledge 2: The number of color block impurities in sperm images is far less than the normal number of sperm.

[0059] The specific description is as follows:

[0060] Traverse all the matched and assembled single independent sperm in a sperm image. For every two sperm i and j, calculate the histogram of the B, G, and R channels within the sperm head contour, hist i,B , hist i,G , hist i,R , hist j,B , hist k,G , hist j,R ; Among them, hist i,B is the B channel histogram within the outline of sperm head i, hist i,G is the G channel histogram within the sperm head contour, hist i,R is the R channel histogram within the sperm head contour, hist j,B is the B channel histogram within the sperm j head contour, hist j,G is the G channel histogram within the sperm head contour, hist j,R is the R channel histogram within the outline of the sperm j head;

[0061] The degree of differentiation of any two sperm heads is measured as follows:

[0062] Select an indicator to describe the degree of match between the two histograms H1 and H2. To verify the comparison indicator of the degree of match of a histogram in this patent, the Pearson correlation coefficient can be selected. The calculation formula of the degree of match d(H1, H2) between the two histograms using the Pearson correlation coefficient is as follows:

[0063]

[0064] in:

[0065]

[0066] Where N is the total number of histogram bins, is the mean of the first input histogram, is the mean of the second input histogram.

[0067] Calculate the matching degree of the three channel histograms of the two sperm head contours respectively i,j , average the matching degree of the three channel histograms.

[0068]

[0069] In OpenCV, the cv::compareHist function is called, and the correlation coefficient is used as the histogram comparison index. The calculated result ranges from [0,1], so there is no need to normalize the histogram matching degree. Because the correlation coefficient is a histogram comparison index, it is proportional to the histogram matching degree. Therefore, the difference between any two sperm heads is head_difference. ij , calculated using the following formula:

[0070] head_difference ij =1-match i,j

[0071] Hierarchical clustering is a type of clustering algorithm that creates a hierarchical nested clustering tree by calculating the distance between sample data points of different categories to achieve classification of sample data. The aggregate hierarchical clustering algorithm first assumes that each sample is a separate cluster, then merges two clusters with close distances to create a new cluster, and repeats the process until the stopping condition is met (until there is only one cluster).

[0072] The difference measure of any two sperm heads can be compared to the distance between samples in hierarchical clustering. With the distance between samples, the hierarchical clustering algorithm can be applied to classify the segmentation and recognition results.

[0073] Usually, the inference results of the segmentation and recognition algorithm are mostly sperm, and a very small number are impurities. The cluster with fewer objects in the cluster is regarded as the cluster where the impurities are.

[0074] Traverse each sperm in the impurity cluster and classify the impurities and sperm using the following method.

[0075] Calculate the median head_difference_median of the difference between the sperm in the impurity cluster and all the sperm in the normal cluster. The head_difference_median search method is as follows:

[0076] The difference measure between a sperm in the impurity cluster and all the sperm in the normal cluster is a set of data. For this set of data, arranged in order, the number in the middle position is the median of the difference measure between the sperm in the impurity cluster and all the sperm in the normal cluster.

[0077] Calculate the segmentation algorithm inference output of the sperm head contour confidence level, acrosome region contour confidence level, and the average head_confidence_avg of the acrosome post-region contour confidence level i ;

[0078]

[0079] Wherein, i represents the subscript number of the independent sperm in the impurity cluster in the first classification result.

[0080] If the acrosome region or post-acrosomal region contour of a sperm does not exist, the corresponding contour will not be considered when calculating head_confidence_avg.

[0081] The following formula is used to obtain the comprehensive confidence level that the sperm in the impurity cluster is an impurity: impurity_confidence.

[0082]

[0083] A pre-set impurity confidence threshold value impurity_threshold is set. When impurity_confidence ≥ impurity_threshold, the sperm is classified as impurity, otherwise it is normal sperm. As shown in the following formula, the impurity_threshold of the method of the present invention is set to 0.75.

[0084]

[0085] Figure 4In the figure, among all the independent sperms identified by segmentation, the entities classified as sperms by the color block impurity classification algorithm are framed by black rectangles; the entities classified as color block impurities by the color block impurity classification algorithm are framed by blue rectangles.

[0086] Example 2

[0087] Based on Example 1, this embodiment is a human sperm image processing device, which includes:

[0088] An image acquisition module, the image acquisition module is used to acquire sperm images after being treated with a staining reagent;

[0089] The segmentation and recognition module uses the pre-trained instance segmentation model to segment the sperm head, acrosome region, and post-acrosomal region in the sperm image, and then post-processes and assembles the segmentation results to obtain the information of all independent sperm in the sperm image. The information of a single independent sperm includes three part contours and the confidence level of the corresponding contours; the three parts include the sperm head contour, acrosome region contour, and post-acrosomal region contour;

[0090] The sperm and color block impurity classification module searches for color block impurities in the segmentation and recognition results and deletes them, thereby obtaining a segmentation and recognition result without color block impurities.

[0091] Example 3

[0092] A computer device comprises at least one processor and at least one memory, wherein the at least one memory is used to store at least one computer program; when the at least one computer program is executed by the at least one processor, the at least one processor implements the method for removing color block impurities from human sperm images.

[0093] Example 4

[0094] This embodiment provides a storage medium, wherein the storage medium stores processor executable instructions, and when the executable instructions are executed by the processor, they are used to execute the method for removing color block impurities from human sperm images.

[0095] Example 5

[0096] On the basis of the above-mentioned embodiment, this embodiment samples 40 human sperm morphology images stained by Papanicolaou and Diff-Quik to test the method for removing color block impurities in human sperm images proposed by the present invention.

[0097] It is found that the method is relatively fast. In the hardware environment of NVIDIA GeForce GTX 1650 graphics card and Intel(R) Core(TM) i7-9700 CPU@3.00GHz 3.00GHz, for a 1920x1200 three-channel sperm morphology image, the average total running analysis time of the segmentation and analysis method proposed by the present invention is 800ms / sperm image, and the sperm and color block impurity classification algorithm has a running time of 0.5ms / sperm image.

[0098] The independent sperm finally segmented and identified do not contain color block impurities, and the color block impurity filtering effect is very good.

Claims

1. A method for removing color block impurities from a human sperm image, the method comprising: S1. Collect sperm images after being treated with staining reagents; S2. Use the pre-trained instance segmentation model to segment the sperm head, acrosome region, and post-acrosomal region in the sperm image, and post-process and match the segmentation results to obtain the information of all independent sperm in the sperm image. The information of a single independent sperm includes three part contours and the confidence level of the corresponding contours; the three parts include the sperm head contour, acrosome region contour, and post-acrosomal region contour; S3. Use the sperm and color block impurity classification algorithm to find the color block impurities in the independent sperm in the sperm image and delete them, so as to obtain a sperm segmentation result without color block impurities.

2. The method for removing color block impurities from human sperm images according to claim 1, characterized in that: The sperm and color block impurity classification algorithm includes the following steps: Based on the information of all the single independent sperm in the sperm image and the difference measurement of any two sperm heads, a hierarchical clustering algorithm is used to divide all the single independent sperm in the sperm image into at most two clusters, the two clusters include sperm clusters and impurity clusters, and the two clusters are the first classification results of sperm and color block impurities; Based on the first classification results of sperm and color block impurities and the confidence level in the sperm head segmentation results of sperm images, a comprehensive classification index is calculated. The sperm and color block impurities are classified using the comprehensive classification index to obtain the second classification results of sperm and color block impurities.

3. The method for removing color block impurities from human sperm images according to claim 2, characterized in that: Any two sperm heads difference measurement, including: Traverse all the individual sperms in a sperm image that have been segmented, matched, and assembled; for any two sperms i and j, calculate the histograms of the three channels B, G, and R within the outline of the sperm head, hist i,B , hist i,G , hist i,R , hist j,B , hist j,G , hist j,R ; Among them, hist i,B is the B channel histogram within the outline of sperm head i, hist i,G is the G channel histogram within the sperm head contour, hist i,R is the R channel histogram within the sperm head contour, hist j,B is the B channel histogram within the sperm j head contour, hist j,G is the G channel histogram within the sperm head contour, hist j,R is the R channel histogram within the outline of the sperm j head; The histogram comparison index is used to measure the matching degree d(H1,H2) of two histograms H1 and H2, and the matching degree match between sperm i and sperm j i,j : The difference measure of the two sperm heads is calculated; If the histogram comparison index is proportional to the degree of histogram matching, then the difference between the heads of two sperm is head_difference ij , head_difference ij =1-norm(match i,j ); If the histogram comparison index is inversely proportional to the degree of histogram matching, then the difference between the heads of two sperm is head_difference ij , head_difference ij =norm(match i,j ); Among them, norm is a function that normalizes the histogram matching degree.

4. The method for removing color block impurities from human sperm images according to claim 3, characterized in that: Histogram comparison metrics include correlation coefficient, chi-square comparison, crossover, Bhattacharyya distance, Hellinger distance, alternate chi-square comparison, and KL divergence.

5. The method for removing color block impurities from human sperm images according to claim 2, characterized in that: The calculation of the comprehensive classification index and the use of the calculation classification include: Calculate the median head_difference_median of the difference measure between the sperm in the impurity cluster and all the sperm in the normal cluster in the first classification result; the difference measure between a sperm in the impurity cluster and all the sperm in the normal cluster constitutes a set of data, which are arranged in order. The number in the middle position is the median head_difference_median of the difference measure between the sperm in the impurity cluster and all the sperm in the normal cluster; Calculate the average confidence level of the sperm segmentation results in the impurity cluster in the first classification result, that is, the average head_confidence_avg of the confidence level of the sperm head contour, the confidence level of the acrosome region contour, and the confidence level of the post-acrosomal region contour output by the segmentation algorithm inference; if the acrosome region and post-acrosomal region contour of a sperm do not exist, the corresponding contour is not considered when calculating head_confidence_avg; Wherein, i represents the subscript number of the independent sperm in the impurity cluster in the first classification result; Calculate the comprehensive indicator impurity_confidence, that is, the comprehensive confidence level of whether the sperm in the impurity cluster is impurity; For the sperm in the impurity cluster in the first classification result, the preset impurity confidence threshold impurity_threshold is used. When impurity_confidence ≥ impurity_threshold, it is classified as impurity. Otherwise, it is classified as normal sperm, and the second classification result of sperm and color block impurity is obtained.

6. A human sperm image processing device, characterized in that: include: An image acquisition module, the image acquisition module is used to acquire sperm images after being treated with a staining reagent; The segmentation and matching module uses the pre-trained instance segmentation model to segment the sperm head, acrosome region, and post-acrosomal region in the sperm image, and then post-processes and assembles the segmentation results to obtain the information of all single independent sperm in the image. A single independent sperm includes three partial contours and the confidence level of the corresponding contours; the three partial contours include the sperm head contour, acrosome region contour, and post-acrosomal region contour; The sperm and color block impurity classification module searches for color block impurities in independent sperm in the sperm image and deletes them, thereby obtaining a sperm segmentation result without color block impurities.

7. A computer device, characterized in that: The method comprises at least one processor and at least one memory, wherein the at least one memory is used to store at least one computer program; when the at least one computer program is executed by the at least one processor, the at least one processor implements the method for removing color block impurities in a human sperm image as described in any one of claims 1 to 5.

8. A storage medium, characterized in that: The storage medium stores processor executable instructions, and when the executable instructions are executed by the processor, they are used to execute the method for removing color block impurities from a human sperm image according to any one of claims 1-5.

Images