Human sperm image color block impurity removal method, and apparatus

Abstract

The present invention relates to sperm image processing technology. Disclosed are a human sperm image color block impurity removal method, and an apparatus. The method comprises: acquiring a sperm image processed with a staining reagent; using a pre-trained instance segmentation model to perform segmentation processing on sperm heads, acrosome regions, and acrosome posterior regions in the sperm image, and performing post-processing, matching and assembling on segmentation results to obtain segmentation and identification information of all individual independent sperms in the image; and searching for color block impurities in segmentation and identification results, and deleting same, so as to obtain sperm segmentation results having no color block impurities. The method of the present invention has the advantages of simplicity and good operation effect, and is very suitable for practical use in industrial scenarios. In addition, by means of the method for filtering color block impurities, color block impurities of sperms segmented and recognized in a sperm image can be accurately and effectively deleted, and final sperm image segmentation and recognition results have no color block impurities, thereby improving the accuracy of computer-assisted sperm morphology analysis.

Description

A method and apparatus for removing color block impurities from human sperm images Technical Field

[0001] This invention relates to sperm image processing technology, and more particularly to a method and apparatus for removing color block impurities from human sperm images. Background Technology

[0002] The sperm images in this patent refer to microscopic images obtained by scanning human semen using a high-powered microscope after staining with staining reagents in a human sperm morphology analysis scenario. For the sperm images, a segmentation algorithm is used to segment and match the sperm contours, obtaining information on all individual sperm in the image. Colored impurities, which resemble the shape of sperm heads, are often incorrectly identified as individual sperm by the segmentation algorithm. This affects the accuracy of computer-aided sperm morphology segmentation.

[0003] Color block impurities exist in the segmentation and recognition results of sperm images, and there is currently no solution to this problem in existing technologies. Identifying and filtering out these color block impurities to improve the accuracy of human sperm morphology analysis is a technical problem that needs to be solved. Summary of the Invention

[0004] This invention addresses the problem of color block impurities in sperm segmentation images in existing technologies. It provides a method and apparatus for removing color block impurities from human sperm images.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] A method for removing color block impurities from human sperm images, the method comprising:

[0007] S1. Acquire images of sperm after treatment with staining reagent;

[0008] S2. Use a pre-trained instance segmentation model to segment the sperm head, acrosome region, and post-acrosome region in the sperm image, and perform post-processing and matching assembly on the segmentation results to obtain information on all independent sperm in the sperm image. The information on a single independent sperm includes three parts contours and the confidence level of the corresponding contours; the three parts include the contour of the sperm head, the contour of the acrosome region, and the contour of the post-acrosome region.

[0009] S3. Using a sperm and color block impurity classification algorithm, find color block impurities in individual sperm in the sperm image and delete them to obtain sperm segmentation results without color block impurities.

[0010] The method for removing color block impurities from human sperm images is based on two prior knowledge points. These two prior knowledge points are explained below.

[0011] Prior knowledge 1: The pixel distribution of color block impurities is significantly different from that of normal sperm pixels.

[0012] Prior knowledge 2: The number of color block impurities in all the independent sperm obtained after sperm image segmentation, post-processing, matching and assembly is far less than the number of normal sperm.

[0013] The sperm and color patch impurity classification algorithm in the sperm image color patch impurity removal method includes the following steps:

[0014] Based on the information of all individual sperm in the sperm image, and based on the difference measure of any two sperm heads, a hierarchical clustering algorithm is used to divide all individual sperm in the sperm image into at most two clusters: one is the sperm cluster, and the other is the impurity cluster. These two clusters are the first classification results of sperm and color block impurities.

[0015] Based on the confidence levels in the first classification results of sperm and color block impurities and the sperm head segmentation results of sperm images, a comprehensive classification index is calculated. Using this comprehensive index, sperm and color block impurities are classified to obtain the second classification results of sperm and color block impurities.

[0016] As a preferred embodiment, the measure of difference between any two sperm heads in the sperm and color patch impurity classification algorithm includes:

[0017] Iterate through all segmented, matched, and assembled individual sperm cells in a sperm image; for any two individual sperm cells i and j, calculate the histograms of the B, G, and R channels within the sperm head contour, and then... i,B hist i,G hist i,R hist j,B hist j,G hist j,R Among them, hist i,B Histogram of B-channels within the contour of sperm i head. i,G Histogram of the G-channel within the contour of sperm i head. i,R Histogram of the R-channel within the contour of sperm i head. j,B Histogram of B-channels within the contour of the sperm head. j,G Histogram of the G-channel within the contour of the sperm head. j,R Histogram of R channels within the outline of the sperm head;

[0018] Histogram comparison metrics are used to measure the degree of matching between two histograms H1 and H2, d(H1,H2), specifically the degree of matching between sperm i and sperm j. i,j :

[0019]

[0020] The difference between the two sperm heads was calculated.

[0021] If the histogram comparison metric is proportional to the degree of histogram matching, then the difference between the two sperm heads is measured by head_difference. ij head_difference ij =1-norm(match) i,j );

[0022] If the histogram comparison metric is inversely proportional to the degree of histogram matching, then the difference between the two sperm heads is measured by head_difference. ij head_difference ij =norm(match) i,j );

[0023] Here, norm is a function that normalizes the histogram matching degree.

[0024] As preferred methods, histogram comparison metrics include correlation coefficient, chi-square comparison, cross-crossing, Bartholomew's distance, Hellinger distance, alternative chi-square comparison, and KL divergence.

[0025] As a preferred option, the function norm for histogram matching and normalization is defined as follows:

[0026]

[0027] Where, x min x represents the minimum matching degree between any two independent sperm cells in a sperm image; max This represents the maximum matching degree between any two independent sperm cells in a sperm image;

[0028] Preferably, in the method for removing color block impurities from human sperm images, the calculation and classification of the comprehensive classification index includes:

[0029] Calculate the median head_difference_median of the difference measure between sperm in the impurity cluster and all sperm in the normal cluster in the first classification result; the difference measure between a sperm in the impurity cluster and all sperm in the normal cluster is a set of data, arranged in order, and the number in the middle position is the median head_difference_median of the difference measure between this sperm in the impurity cluster and all sperm in the normal cluster.

[0030] Calculate the average confidence level of sperm segmentation results in the impurity clusters of the first classification result, i.e., the average confidence level of the sperm's head contour, acrosome contour, and post-acrosome contour output by the segmentation algorithm, head_confidence_avg; if the acrosome or post-acrosome contour of a sperm does not exist, the corresponding contour is not considered when calculating head_confidence_avg; calculate the average confidence level of the sperm's head contour, acrosome contour, and post-acrosome contour output by the segmentation algorithm, head_confidence_avg. i ;

[0031]

[0032] Where i represents the index number of the independent sperm in the impurity cluster in the first classification result;

[0033] The comprehensive index impurity_confidence is calculated, which is the overall confidence level of whether sperm in the impurity cluster are impurities;

[0034]

[0035] For sperm in the impurity cluster in the first classification result, a preset impurity confidence threshold (impurity_threshold) is used. If impurity_confidence ≥ impurity_threshold, it is classified as impurity; otherwise, it is classified as normal sperm, thus obtaining the second classification result of sperm and color block impurities.

[0036]

[0037] To address the aforementioned technical problems, the present invention also provides a human sperm image processing apparatus, comprising:

[0038] Image acquisition module, used to acquire images of sperm after treatment with staining reagent;

[0039] The segmentation and recognition module uses a pre-trained instance segmentation model to segment the sperm head, acrosome region, and post-acrosome region in the sperm image. It then processes and matches the segmentation results to obtain information on all individual sperm in the sperm image. The information on an individual sperm includes the contours of the three parts of the sperm and the confidence level. The three parts include the contours of the sperm head, acrosome region, and post-acrosome region.

[0040] The sperm and color block impurity classification module finds and deletes color block impurities in individual sperm in the sperm image, thus obtaining sperm segmentation results without color block impurities.

[0041] To address the aforementioned technical problems, the present invention also provides a computer device comprising at least one processor and at least one memory, wherein the at least one memory is used to store at least one computer program; when the at least one computer program is executed by the at least one processor, the at least one processor implements the method for removing color block impurities from human sperm images.

[0042] To address the aforementioned technical problems, the present invention also provides a storage medium storing processor-executable instructions, which, when executed by a processor, are used to perform the aforementioned method for removing color block impurities from human sperm images.

[0043] This invention, by adopting the above technical solutions, has significant technical effects:

[0044] This invention proposes a method for removing color block impurities from human sperm images, which improves the accuracy of color block impurity removal in human sperm images, facilitates automated segmentation and analysis of human sperm morphology, indirectly reduces the repetitive workload of doctors, improves the efficiency of human sperm morphology analysis, and reduces the waste of social resources.

[0045] The method of this invention has the advantages of simplicity and high efficiency, making it very suitable for industrial applications. Furthermore, the method of filtering out color block impurities can accurately and effectively remove color block impurities that are segmented and identified as sperm in sperm images. The final sperm image segmentation and identification results are free of color block impurities, which improves the accuracy of computer-aided sperm morphology analysis and helps to output more accurate and reliable human sperm morphology analysis conclusions. Attached Figure Description

[0046] Figure 1. Flowchart of the method of the present invention;

[0047] Figure 2 is a schematic diagram of the human sperm structure of the present invention;

[0048] Figure 3 is the original image of the verification color block impurity filtering of the present invention;

[0049] Figure 4 is an image showing the effect of the present invention in filtering impurities;

[0050] Figure 5 is a flowchart of the sperm image extraction process of the present invention. Detailed Implementation

[0051] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0052] Example 1

[0053] A method for removing color block impurities from human sperm images includes the following steps:

[0054] S1. Acquire images of sperm after treatment with staining reagent;

[0055] S2. Use a pre-trained instance segmentation model to segment the sperm head, acrosome region, and post-acrosome region in the sperm image. Post-process and match the segmentation results to obtain information on all independent sperm in the sperm image. The information of a single independent sperm includes three parts contours and the confidence level of the corresponding contours. The three parts include the contour of the sperm head, the contour of the acrosome region, and the contour of the post-acrosome region. S3. Find and delete color block impurities in the segmentation recognition results.

[0056] The algorithm for finding impurities in color patches is based on two prior knowledge points. For all individual sperm samples obtained, the confidence level of the instance is output based on the differences in pixel distribution and the segmentation model, determining whether a sperm sample is an impurity. The two prior knowledge points are explained below:

[0057] Prior knowledge 1: The pixel distribution of color block impurities is significantly different from that of normal sperm pixels.

[0058] Prior knowledge 2: Colored impurities in sperm images are far fewer than the number of normal sperm.

[0059] The specific description is as follows:

[0060] Iterate through all matched and assembled individual sperm cells in a sperm image. For any two sperm cells i and j, calculate the histograms of the B, G, and R channels within the sperm head contour. i,B hist i,G hist i,R hist j,B hist k,G hist j,R Among them, hist i,B Histogram of B-channels within the contour of sperm i head. i,G Histogram of the G-channel within the contour of sperm i head. i,R Histogram of the R-channel within the contour of sperm i head. j,B Histogram of B-channels within the contour of the sperm head. j,G Histogram of the G-channel within the contour of the sperm head. j,R Histogram of R channels within the outline of the sperm head;

[0061] The degree of difference in the heads of any two sperm cells is measured as follows:

[0062] Choose an index to describe the degree of matching between two histograms H1 and H2. To verify the degree of matching of the histograms in this patent, the Pearson correlation coefficient can be selected. The formula for calculating the degree of matching d(H1,H2) between the two histograms using the Pearson correlation coefficient is as follows:

[0063]

[0064] in:

[0065]

[0066] Where N is the total number of histogram intervals, It is the mean of the first input histogram. It is the mean of the second input histogram.

[0067] Calculate the three-channel histogram matching degree of the two sperm head contours respectively. i,j The average value of the matching degree of the histograms of the three channels is calculated.

[0068]

[0069] In OpenCV, the `cv::compareHist` function uses the correlation coefficient as a histogram comparison metric. The calculated result ranges from [0,1], therefore, normalization of the histogram matching degree is unnecessary. Furthermore, since the correlation coefficient histogram comparison metric is proportional to the histogram matching degree, the difference between any two sperm heads is measured by `head_difference`. ij Calculate using the following formula:

[0070] head_difference ij =1-match i,j

[0071] Hierarchical clustering is a type of clustering algorithm that creates a hierarchical nested clustering tree by calculating the distances between sample data points of different categories, thereby classifying sample data. The aggregate hierarchical clustering algorithm first assumes that each sample is a separate cluster, then merges the two closest clusters to create a new cluster, repeating this process until a stopping condition is met (when only one cluster remains).

[0072] The measure of difference between any two sperm heads can be analogous to the distance between samples in hierarchical clustering. With the distances between samples, hierarchical clustering algorithms can be applied to classify the segmentation and identification results.

[0073] Typically, the inference results of segmentation and recognition algorithms mostly represent sperm, with a very small number representing impurities. The cluster with the fewest objects within it is considered the cluster containing the impurities.

[0074] Iterate through each sperm cell in the impurity cluster and classify the impurities from the sperm cells using the following method.

[0075] Calculate the median head_difference_median, the measure of difference between sperm in the impurity cluster and all sperm in the normal cluster. The head_difference_median is found as follows:

[0076] The measure of difference between a sperm in the impurity cluster and all sperm in the normal cluster is a set of data. The middle number in this set of data, arranged in order, is the median of the measure of difference between the sperm in the impurity cluster and all sperm in the normal cluster.

[0077] The segmentation algorithm calculates the average confidence levels of the sperm's head contour, acrosome contour, and post-acrosome contour, outputting the confidence level as head_confidence_avg. i ;

[0078]

[0079] Where i represents the index number of the independent sperm in the impurity cluster in the first classification result.

[0080] If the acrosome region and postacrosome region contours of a certain sperm do not exist, the corresponding contours are not considered when calculating head_confidence_avg.

[0081] The overall confidence level that a sperm in an impurity cluster is an impurity is obtained using the following formula: impurity_confidence.

[0082]

[0083] A preset impurity confidence threshold, `impurity_threshold`, is used. If `impurity_confidence` ≥ `impurity_threshold`, the sperm is classified as impurity; otherwise, it is considered normal sperm. As shown in the formula below, the `impurity_threshold` is set to 0.75 to verify the method of this invention.

[0084]

[0085] In Figure 4, among all the individual sperm identified by the segmentation algorithm, the entity classified as sperm by the color block impurity classification algorithm is enclosed in a black rectangle; the entity classified as color block impurity by the color block impurity classification algorithm is enclosed in a blue rectangle.

[0086] Example 2

[0087] Based on Example 1, this example is a human sperm image processing device, which includes:

[0088] Image acquisition module, used to acquire images of sperm after treatment with staining reagent;

[0089] The segmentation and recognition module uses a pre-trained instance segmentation model to segment the sperm head, acrosome region, and post-acrosome region in the sperm image. It then processes and matches the segmentation results to obtain information on all individual sperm in the sperm image. The information on a single individual sperm includes three parts of the contour and the confidence level of the corresponding contour. The three parts include the contour of the sperm head, the contour of the acrosome region, and the contour of the post-acrosome region.

[0090] The sperm and color block impurity classification module searches for and deletes color block impurities in the segmentation and recognition results, thereby obtaining segmentation and recognition results without color block impurities.

[0091] Example 3

[0092] A computer device includes at least one processor and at least one memory, the at least one memory being used to store at least one computer program; when the at least one computer program is executed by the at least one processor, the at least one processor causes the at least one processor to implement the human sperm image color block impurity removal method.

[0093] Example 4

[0094] This embodiment provides a storage medium storing processor-executable instructions, which, when executed by the processor, are used to perform the human sperm image color block impurity removal method.

[0095] Example 5

[0096] Based on the above embodiments, this embodiment samples 40 human sperm morphology images stained with Papanicolaou and Diff-Quik to test the method for removing color block impurities from human sperm images proposed in this invention.

[0097] Actual testing revealed that this method is relatively fast. With an NVIDIA GeForce GTX 1650 graphics card and an Intel(R) Core(TM) i7-9700 CPU@3.00GHz, the average total runtime of the segmentation and analysis method proposed in this invention for 1920x1200 three-channel sperm morphology images is 800ms per sperm image, and the sperm and color patch impurity classification algorithm has a runtime of 0.5ms per sperm image.

[0098] The individual sperm that were finally segmented and identified did not contain any color block impurities, indicating that the filtering effect for color block impurities was excellent.

Claims

1. A method for removing color block impurities from human sperm images, the method comprising: S1. collecting sperm images after treatment with a staining reagent; S2. using a pre-trained instance segmentation model to segment the sperm head, acrosomal region, and post-acrosomal region in the sperm images, and post-processing and matching assembly of the segmentation results to obtain information of all independent sperm in the sperm images, the information of a single independent sperm including three part outlines and the confidence level of the corresponding outlines; the three part outlines including the outline of the sperm head, the outline of the acrosomal region, and the outline of the post-acrosomal region; S3. using a sperm and color block impurity classification algorithm to find and delete color block impurities in the independent sperm in the sperm images, thereby obtaining sperm segmentation results without color block impurities.

2. The method of claim 1, wherein the method is characterized by: The sperm and color block impurity classification algorithm comprises the following steps: Based on all single independent sperm information in the sperm images, based on the difference metric of any two sperm heads, using a hierarchical clustering algorithm, all single independent sperm in the sperm images are divided into at most two clusters, the two clusters including a sperm cluster and an impurity cluster, the two clusters being the first classification result of sperm and color block impurities; Based on the confidence level in the sperm head segmentation result of the sperm image, a comprehensive classification index is calculated, and the comprehensive classification index is used to classify sperm and color block impurities to obtain a second classification result of sperm and color block impurities.

3. The method of claim 2, wherein the method is characterized by, The difference metric of any two sperm heads includes: Traverse all the single independent sperm which have been segmented, matched and assembled in a sperm image; for any two sperms i, j, calculate the histograms of B, G and R channels within the head contour of the sperms, respectively, hist i,B , hist i,G , hist i,R , hist j,B , hist j,G , hist j,R ; wherein, hist i,B is the B channel histogram within the head contour of the sperm i, hist i,G is the G channel histogram within the head contour of the sperm i, hist i,R is the R channel histogram within the head contour of the sperm i, hist j,B is the B channel histogram within the head contour of the sperm j, hist j,G is the G channel histogram within the head contour of the sperm j, and hist j,R is the R channel histogram within the head contour of the sperm j; The matching degree d(H1, H2) of two histograms H1, H2 is measured using the histogram comparison indicator, the matching degree match of sperm i with sperm j i,j : The difference metric of the two sperm heads is calculated; If the histogram comparison metric measures the degree to which the histograms match, then the difference between the two sperm heads is measured by head_difference = 1 - norm(match ij ) ij ); where norm(match i,j ) is the norm of the difference between the two histograms. If the histogram comparison metric measures the degree of match of the histograms inversely, then the difference of the two sperm heads is measured by head_difference ij , head_difference ij = norm(match i,j ); wherein norm is a function for normalizing the degree of histogram matching.

4. The method of claim 3, wherein the method is characterized by, The histogram comparison index includes the correlation coefficient, chi-square comparison, cross-intersection, Bhattacharyya distance, Hellinger distance, reserve chi-square comparison, and KL divergence.

5. The method of removing color patch impurities from human sperm images as claimed in claim 2, wherein, The comprehensive classification index calculation and use of the calculation classification include: The median of the difference metric of the sperm in the impurity cluster in the first classification result and all sperm in the normal cluster is calculated as head_difference_median; the difference metric of a sperm in the impurity cluster and all sperm in the normal cluster forms a set of data, which is arranged in order, and the number in the middle position is the median of the difference metric of the sperm in the impurity cluster and all sperm in the normal cluster, which is head_difference_median; Calculate the average confidence level of sperm segmentation results in the impurity cluster in the first classification result, that is, the average head_confidence_avg of the confidence level of the head contour, the acrosome contour, and the post-acrosome contour of the sperm output by the segmentation algorithm; if the acrosome and post-acrosome contours of a sperm do not exist, the corresponding contours are not considered when calculating head_confidence_avg. wherein i represents the subscript number of the independent sperm in the impurity cluster in the first classification result; calculating an integrated indicator impurity_confidence, i.e. an integrated confidence level that the sperm in the impurity cluster is an impurity; For the sperm in the impurity cluster in the first classification result, a preset impurity confidence threshold impurity threshold is used. When impurity_confidence ≥ impurity_threshold, the sperm is classified as an impurity, otherwise, the sperm is classified as a normal sperm, to obtain a sperm and color block impurity second classification result; 6. A processing apparatus of a human sperm image, characterized by, It includes: An image acquisition module for acquiring sperm images after treatment with a staining reagent; A segmentation and matching module using a pre-trained instance segmentation model to segment the sperm head, acrosomal region, and post-acrosomal region in the sperm images, and post-processing and matching assembly of the segmentation results to obtain information of all single independent sperm in the images, the single independent sperm including three part outlines and the confidence level of the corresponding outlines; the three part outlines including the outline of the sperm head, the outline of the acrosomal region, and the outline of the post-acrosomal region; A sperm and color block impurity classification module for finding and deleting color block impurities in the independent sperm in the sperm images, thereby obtaining sperm segmentation results without color block impurities.

7. A computer device, comprising: A computer program product, comprising at least one memory and at least one processor, said at least one memory for storing at least one computer program; which computer program, when executed by the at least one processor, causes the at least one processor to carry out the method of any one of claims 1-5.

8. A storage medium, characterized by The computer program product, wherein the storage medium stores processor-executable instructions for execution by a processor to perform the method of any one of claims 1-5.

Images