Sperm image segmentation processing method and device

Abstract

The invention discloses a sperm image segmentation processing method and device. The sperm image segmentation processing method comprises the following steps: acquiring a sperm image processed by a staining reagent; segmenting the sperm image by using a pre-trained instance segmentation model to obtain a sperm contour set and a sperm contour category set in the sperm image, and reasoning and outputting a sperm confidence level set of an instance category; performing post-processing on a result output by instance segmentation; obtaining a post-processed sperm contour set and a sperm contour category set, and reasoning and outputting a sperm confidence level set of the instance category; and assembling and matching the sperm contours based on the post-processed sperm contour set, the sperm contour category set and a sperm confidence level set of the instance category output by reasoning to obtain all independent sperm information. According to the invention, the contour segmentation of each constituent part of the sperms output through segmentation is accurate, the contour omission ratio is low, the integrity is good, and the sperms after final matching and combination have the advantages of high accuracy and low omission recognition rate.

Description

Technical Field

[0001] The invention relates to sperm image processing technology, and in particular to a method and device for sperm image segmentation processing. Background Art

[0002] The sperm image in this patent refers to the microscopic image obtained by using a high-power microscope to collect and scan human semen after staining it with a staining reagent in the human sperm morphology analysis scenario. In the hospital, doctors sample, dilute, and stain semen to make slide specimens, and then observe, count, and analyze them under a microscope with the naked eye. Manual measurement and statistics methods are labor-intensive and inefficient for human sperm morphology analysis, resulting in unnecessary waste of resources.

[0003] In the fifth edition of the Laboratory Manual for the Examination and Processing of Human Semen, "sperm consists of a head, neck, midpiece, principal segment and terminal segment. Since it is difficult to observe the terminal segment of sperm through an optical microscope, it can be considered that sperm consists of a head (and neck) and a tail (midpiece and principal segment). Only sperm with normal heads and tails are considered normal. All sperm with critical morphology should be considered abnormal." The sperm head contains the acrosome region and the post-acrosomal region, and the tail contains the midpiece and principal segment. The neck is very short and can be combined with the midpiece when the neck is segmented. In the fifth edition of the Normal Sperm Morphology Classification of the Laboratory Manual for the Examination and Processing of Human Semen, a sperm is judged to be normal, and there are specific requirements for the acrosome region, post-acrosomal region, midpiece and principal segment of the sperm. Therefore, the segmentation of the contour boundaries of the acrosome region, post-acrosomal region, neck, midpiece and principal segment is a very important component for computer-assisted sperm morphology analysis, which is related to the accuracy of subsequent sperm morphology analysis.

[0004] Prior art 1: CN118196406A proposes using a segmentation model to iteratively identify the overall contour of a single sperm in a stained image, and using a clustering algorithm to segment the boundaries of each sperm component.

[0005] Prior art 2: Patent CN116091381A proposes using a segmentation algorithm to obtain the head contour, and then segmenting the middle section and tail of the sperm based on an adaptive threshold segmentation algorithm + a conditional random field algorithm.

[0006] Prior art 3: Patent CN116030458A proposes using a segmentation model to obtain the segmentation outline of the sperm head, and then merges and segments the tail based on regional growth.

[0007] Prior art 4: In the specific implementation of the patent CN111563550A, the whole sperm, head and tail boundaries are segmented by a preset sperm outline template.

[0008] Prior art 5: Patent CN104268515A uses the Snake model and the gradient vector flow field external force to detect the overall contour, head contour, and mid-section boundary of a single sperm.

[0009] Prior art 6: Patent CN112801967A uses a spot detection algorithm to determine the sperm head area, and uses a semantic segmentation algorithm to segment the sperm image into four parts: acrosome, nucleus, midsection and background. Patent WO2022CN142159 proposes segmenting a single sperm into head, midsection and tail.

[0010] The above patents do not provide a complete technical solution for dividing the boundaries of the sperm head, acrosome region, post-acrosomal region, neck, middle section, and main section.

[0011] In summary, the prior art lacks a complete technical solution that simultaneously solves the technical problems of sperm head, acrosome region, post-acrosomal region, neck and middle segment, and main segment boundary detection and segmentation.

[0012] In human sperm images, when a segmentation algorithm is used to solve the problem of segmenting the various components of sperm, the instance categories of instance segmentation are the acrosome region, the post-acrosomal region, the middle segment, and the main segment. After matching and combining the contours of the acrosome region and the post-acrosomal region, the complete sperm head contour is obtained. There will be a crack in the sperm head contour at the junction of the acrosome region and the post-acrosomal region, and it cannot be perfectly merged into a sperm head. The closing operation in the image morphological operation cannot solve this problem well. The sperm head contour after the closing operation will have a small inward concave notch at the junction, making the head contour irregular and not smooth. A normal sperm head will therefore be mistakenly identified as an abnormal sperm. This sub-technical problem will affect the judgment of whether the sperm is normal or abnormal.

[0013] In addition, in the output results of instance segmentation prediction, there may be some omissions in the segmentation of the sperm acrosome region, the contour of the post-acrosomal region, and the contour of the middle section. In sperm images with a large number of sperm, using segmentation algorithms for segmentation may result in sperm omissions.

[0014] After solving the sub-technical problem of detecting and segmenting the boundaries of the acrosome region, post-acrosomal region, neck and middle segment, and main segment of human sperm in sperm images, the sub-technical problem of how to accurately and efficiently match and combine the acrosome region, post-acrosomal region, neck and middle segment, and main segment of sperm belonging to the same sperm will be encountered. This matching and combining step is prone to incorrect matching, resulting in incorrect matching of the head, neck, middle segment, and main segment between adjacent sperm. In the prior art, there is no technical solution for this sub-problem.

[0015] In summary, the technical problem of achieving sperm segmentation in a complete, comprehensive and low missed detection rate will encounter the following sub-technical problems:

[0016] 1. Completely and accurately segment the sperm head, acrosome region, post-acrosomal region, neck and middle segment, and main segment outline.

[0017] 2. There are cracks in the contour boundary between the acrosome region and the post-acrosomal region, which causes the sperm head to be irregular after matching and combination, and reduces the contour accuracy of the sperm components after matching and combination.

[0018] 3. Sperm images with a large number of sperm may have missed detection of segmentation instance contours. The sperm acrosome region, post-acrosomal region, and mid-section contours may be missed in some test images. These missed detection and missed recognition problems will reduce the integrity and accuracy of the final segmentation effect.

[0019] 4. The contours of the various sperm components segmented by the example are scattered in space, and there is no information about the entire sperm. In the prior art, there is a lack of an algorithm for matching and combining the various sperm components segmented with high accuracy. Summary of the invention

[0020] The present invention aims at the technical problem that the prior art lacks a method that can completely and fully solve the technical problem of the sperm head, acrosome region, post-acrosomal region, neck and middle section, and main section contour segmentation and matching of the sperm components after segmentation. A sperm image segmentation processing method and device are provided.

[0021] In order to solve the above technical problems, the present invention is solved by the following technical solutions:

[0022] A human sperm image segmentation processing method, comprising:

[0023] S1. Collect sperm images after being treated with staining reagents;

[0024] S2. Use the pre-trained instance segmentation model to segment the sperm head, acrosome region, post-acrosomal region, neck and middle segment, and main segment in the sperm image to obtain a sperm contour set, a sperm contour category set, and an inference output sperm confidence level set of the instance category in the sperm image;

[0025] S3. Post-processing the output result of the instance segmentation; obtaining a post-processed sperm contour set, a sperm contour category set, and a sperm confidence level set of the instance category output by reasoning;

[0026] S4. Based on the post-processed sperm contour set, sperm contour category set and the inferred output sperm confidence level set of the instance category, the contour sets of each part of the sperm are assembled and matched to obtain all the single independent sperm in the sperm image. A single independent sperm includes its head contour, acrosome region contour, post-acrosomal region contour, neck and mid-segment contour, main segment contour, and the inferred output confidence levels of each of these five parts.

[0027] Preferably, based on the post-processed sperm contour set, the sperm contour category set and the sperm confidence level set of the instance category output by reasoning, the contour set of each part of the sperm is assembled and matched, including:

[0028] Using the sperm head contour, acrosome region contour, and post-acrosomal region contour matching and assembly algorithm, the sperm head contour, acrosome region contour, and post-acrosomal region contour are matched and assembled to obtain the first matching assembly result of all sperms in the sperm image;

[0029] For the first matching assembly result of all sperms in the sperm image, the contour correction and supplementation algorithm of the acrosome region and the post-acrosomal region is used to obtain the second matching assembly result of all sperms in the sperm image;

[0030] For the second matching assembly result of all sperms in the sperm image, the sperm head contour is matched and assembled with the neck and middle section contour using the sperm head, neck and middle section contour matching assembly model to obtain the third matching assembly result of all sperms in the sperm image;

[0031] For the third matching assembly result of all sperms in the sperm image, the third matching assembly result of all sperms in the sperm image is used with the main segment contour matching assembly model to realize the third matching assembly result of all sperms in the sperm image and the main segment contour matching assembly, so as to obtain the fourth matching assembly result of all sperms in the sperm image;

[0032] For the fourth matching assembly result of all sperms in the sperm image, a middle segment contour supplementation algorithm is used to supplement the possible middle segment missing in the fourth matching assembly result, so as to obtain the fifth matching assembly result of all sperms in the sperm image;

[0033] The fifth matching assembly result includes all the individual sperms in the sperm image. The individual sperms contain five-part sperm contours, which are the sperm head contour, the sperm acrosome region contour, the sperm post-acrosomal region contour, the sperm neck and mid-segment contour, the sperm main segment contour, and the confidence levels of the inferred output sperm five-part contours.

[0034] Preferably, post-processing the instance segmentation output result includes:

[0035] Set the instance segmentation post-processing parameters, set the instance segmentation post-processing parameters test_topk_per_image greater than 100;

[0036] Enlarge the contour image output by the instance segmentation model to the size of the input image;

[0037] The sperm contour is processed by deleting the predicted contours below the preset contour confidence threshold, deleting the predicted contours with an area below the preset area threshold, and using the non-maximum suppression algorithm to delete the repeated contours, thereby obtaining the contour set of all sperm components in the sperm image after post-processing, the contour category set, and the confidence level set of the inference output instance category.

[0038] As a preferred embodiment, the sperm head contour, the acrosome region contour, and the post-acrosomal region contour matching and assembly algorithm include:

[0039] The results of sperm image instance segmentation and post-processing are stored in their respective category lists according to instance contour categories;

[0040] Traverse the list of all sperm head contours, segment each sperm head contour, and find the matching acrosome region and post-acrosomal region contours, including:

[0041] The intersection and union ratio IoU1 of the bounding rectangular box of all the acrosome region contours obtained by sperm image segmentation and post-processing and the bounding rectangular box of the head contour is calculated. The acrosome region with the largest IoU1 is the acrosome region of the current sperm head; the intersection and union ratio IoU2 of the bounding rectangular box of all the post-acrosomal region contours obtained by sperm image segmentation and post-processing and the bounding rectangular box of the sperm head contour is calculated. The post-acrosomal region with the largest IoU2 is the post-acrosomal region of the current sperm head.

[0042] The calculation formula of the intersection over union (IoU) of two rectangular boxes is:

[0043] The bounding rectangle of a contour is defined as the smallest rectangle that can completely contain the contour, and the boundary of the rectangle is parallel to the coordinate axis, that is, the sides of the rectangle are horizontal or vertical.

[0044] As a preferred embodiment, the contour correction and supplementation algorithm of the acrosome region and the post-acrosome region includes:

[0045] Based on the first matching assembly results of all sperm in the sperm image, traverse the first matching assembly results of all sperm, and for each of the first matching assembly results:

[0046] When both the acrosome region and the post-acrosomal region exist, the contour correction algorithm of the acrosome region and the post-acrosomal region is executed. The contour correction includes:

[0047] The union of the acrosome region outline, the post-acrosomal region outline and the head outline is taken as the final merged head outline;

[0048] For the acrosome region and the post-acrosomal region belonging to the same head, the contour with the highest confidence level inferred by the instance segmentation model is retained, which is the contour of the first part of the sperm head, and its contour category remains unchanged; for the contour with a low confidence level, the contour area of ​​the sperm head after the final merger is used, and the contour corresponding to the first part of the sperm head is subtracted, and the contour corresponding to the remaining area is the corrected contour of the other part of the sperm head. The corrected contour category is the category of the contour with a low confidence level; contour correction is completed;

[0049] If the head contour exists and one of the acrosome region or the post-acrosomal region is missing, the acrosome region and the post-acrosomal region contour supplementation algorithm is executed. The acrosome region and the post-acrosomal region contour supplementation algorithm includes:

[0050] The contour area of ​​the sperm head is used to subtract the contour area of ​​the intact sperm part to obtain the contour corresponding to the missing part area, which is the contour of the other missing part of the sperm head.

[0051] Preferably, the sperm head contour matching and assembly algorithm with the neck and midsection contours includes:

[0052] Constant c i,j It represents the measure of the matching degree between the head contour in the i-th second matching assembly result and the j-th sperm mid-segment contour obtained by sperm image segmentation and post-processing. The specific calculation method is:

[0053] Where i represents the number subscript of the second matching assembly result, j represents the number subscript of the sperm mid-segment contour obtained by sperm image segmentation and post-processing; IoU i,j The intersection-and-union ratio of the rectangular bounding box of the head contour in the i-th second matching assembly result and the rectangular bounding box of the j-th sperm midsection contour obtained by sperm image segmentation and post-processing; when calculating the intersection-and-union ratio, the rectangular bounding box of the midsection contour is kept unchanged with the center point coordinates unchanged, and the width and height directions of the sides are enlarged by 1.3 times. This processing ensures that the elongated midsection rectangular bounding box will intersect with the adjacent sperm head after enlargement; distance i,j represents the Euclidean distance between the two nearest points between the head contour in the i-th second matching assembly result and the j-th sperm midsection contour obtained by sperm image segmentation and post-processing; tail_middle_s j represents the contour area of ​​the jth sperm middle segment obtained by sperm image segmentation and post-processing; max(tail_middle_s) represents the maximum area of ​​the sperm middle segment contours obtained by all sperm image segmentation and post-processing in the current sperm image;

[0054] I is the indicator function,

[0055]

[0056] Clamp is a function that limits the range of variables.

[0057]

[0058] Decision variable x i,j Indicates whether the head contour in the second matching assembly result of the i-th sperm matches the j-th sperm midsection contour obtained by sperm image segmentation and post-processing; i represents the number subscript of the sperm second matching assembly result, and j represents the number subscript of the sperm midsection contour obtained by sperm image segmentation and post-processing;

[0059]

[0060] The mathematical model for matching the head contour with the midsection contour is described as follows:

[0061] Objective function:

[0062]

[0063] Subject to:

[0064] x i,j =0 or x i,j =1 (2)

[0065]

[0066] Wherein, m represents the total number of second matching assembly results of all sperm in the sperm image, and n represents the total number of sperm mid-segment contours obtained by sperm image segmentation and post-processing;

[0067] Inequality (3) indicates that for each head contour in the second matching assembly result, there is at most one sperm mid-segment contour obtained by sperm image segmentation and post-processing that matches it;

[0068] Inequality (4) indicates that for each sperm mid-segment contour obtained by sperm image segmentation and post-processing, there is at most one head contour in the second matching assembly result that matches it;

[0069] The mathematical model for solving the matching of the head contour and the mid-segment contour is used. The optimal matching solution between the sperm head and the sperm mid-segment is the third matching assembly result of all sperm in the sperm image.

[0070] Preferably, the third matching assembly result and the main segment contour matching assembly algorithm include:

[0071] By matching the middle segment’s flag middle_exist_flag i Describe whether the third matching assembly result has a matching middle segment;

[0072]

[0073] When the third matching assembly result has a matching middle segment, the matching degree between the middle segment contour of the third matching assembly result and the contours of each sperm main segment obtained by sperm image segmentation and post-processing is calculated; the matching degree d between the middle segment of the third matching assembly result and the contours of the sperm main segment obtained by sperm image segmentation and post-processing is calculated. i,j :

[0074]

[0075] Wherein, i represents the number subscript of the third matching assembly result, and j represents the number subscript of the sperm main segment contour obtained by sperm image segmentation and post-processing;

[0076] Clamp is a function that limits the range of variables:

[0077]

[0078] middle_exist_flag i =1, distance i,j represents the Euclidean distance between the two closest points between the middle segment contour of the i-th third matching assembly result and the j-th sperm main segment contour obtained by sperm image segmentation and post-processing; confidence j It indicates the confidence level of the main segment instance in the inference output of the instance segmentation model, where j represents the subscript number of the sperm main segment contour obtained by sperm image segmentation and post-processing;

[0079] When there is no matching middle segment in the third matching assembly result, the matching degree d between the head contour in the third matching assembly result and the contours of each sperm main segment obtained by sperm image segmentation and post-processing is calculated. i,j ;

[0080] For all main segment contours, extract its skeleton line and check whether the extended line of the skeleton line intersects with the head contour in the third matching assembly result. If not, then d i,j = 0, indicating that the matching degree is 0; if they intersect, the matching degree is d i,j The calculation rules are as follows:

[0081]

[0082] middle_exist_flag i = 0, distancei,j represents the Euclidean distance between the two closest points between the head contour of the i-th third matching assembly result and the j-th sperm main segment contour obtained by sperm image segmentation and post-processing; confidence j It indicates the confidence level of the main segment instance output by the segmentation model inference, where j represents the number subscript of the sperm main segment contour obtained by sperm image segmentation and post-processing;

[0083] Decision variable y i,j Indicates whether the i-th third matching assembly result matches the j-th sperm main segment contour obtained by sperm image segmentation and post-processing;

[0084]

[0085] Wherein, i represents the number subscript of the third matching assembly, and j represents the number subscript of the sperm main segment contour obtained by sperm image segmentation and post-processing;

[0086] The mathematical model description of the third matching assembly result and the main segment contour matching assembly,

[0087] Objective function:

[0088]

[0089] Subject to:

[0090] y i,j =0 or y i,j =1 (6);

[0091]

[0092] y i,j =0when(middle_exist_flag i =1)and((distance i,j

[0093] >dist_middle_main_thres)

[0094] or((dist_sperm_main_thres <distance i,j ≤

[0095] dist_middle_main_thres)and(head_tail_distance i,j >

[0096] dist_sperm_main_thres)))(9);

[0097] yi,j =0when(middle_exist_flag i =0)and(distance i,j >

[0098] k*long_axis i ) (10);

[0099] Wherein, m represents the total number of third matching assembly results in the sperm image, and n1 represents the number of sperm main segment contours obtained by sperm image segmentation and post-processing;

[0100] dist_middle_main_thres indicates the preset threshold value of the farthest distance between the middle segment contour and the main segment contour belonging to the same sperm; dist_sperm_main_thres indicates the preset threshold value of the farthest distance between the third matching assembly result belonging to the same sperm and the main segment contour; long_axis i Indicates the major axis length of the head contour in the i-th third matching assembly result; head_tail_distance i,j represents the Euclidean distance between the nearest two points of the head contour in the i-th third matching assembly result and the j-th sperm main segment contour obtained by sperm image segmentation and post-processing;

[0101] Inequality (7) indicates that each third matching assembly result has at most one sperm main segment contour obtained by sperm image segmentation and post-processing that matches it;

[0102] Inequality (8) indicates that each sperm main segment contour obtained by sperm image segmentation and post-processing has at most one third matching assembly result matching it;

[0103] Inequality (9) describes the situation where there is a matching middle segment in the third matching assembly result. Based on prior knowledge, when any of the following two situations is met, the corresponding third matching assembly result should not match the sperm main segment contour obtained by sperm image segmentation and post-processing;

[0104] Case 1: When the distance between the middle contour of the third matching assembly result and the main contour of the sperm obtained by sperm image segmentation and post-processing is greater than the distance threshold dist_middle_main_thres;

[0105] Case 2: When the distance between the middle contour of the third matching assembly result and the sperm main segment contour obtained by sperm image segmentation and post-processing, and the distance between the head contour in the third matching assembly result and the sperm main segment contour obtained by sperm image segmentation and post-processing, are both greater than the distance threshold dist_sperm_main_thres;

[0106] Inequality (10) describes the situation where there is no matching middle segment in the third matching assembly result. The head contour in the third matching assembly result and the sperm main segment contour obtained by sperm image segmentation and post-processing are greater than the distance threshold k*long_axis i When , it should not be matched. The distance threshold here is defined as k times the long axis of the head contour in the i-th third matching assembly result;

[0107] A mathematical model for matching and assembling the third matching assembly result with the sperm main segment contour obtained by sperm image segmentation and post-processing is solved. The optimal matching and assembly solution of the third matching assembly result and the sperm main segment is the fourth matching assembly result of all sperm in the sperm image.

[0108] Preferably: a middle section contour supplement algorithm in a human sperm image segmentation processing method comprises: if the fourth matching assembly result of all sperm in the sperm image has a missing middle section, a line segment with a width equal to the width of the main section contour is drawn between the two nearest points between the head contour and the main section contour in the fourth matching assembly result, and the outer contour of the line segment is regarded as the missing middle section contour, so as to obtain the fifth matching assembly result of all sperm in the sperm image.

[0109] In order to solve the above technical problems, the present invention also provides a sperm image segmentation processing device, comprising:

[0110] An image acquisition module is used to acquire sperm morphology images after being treated with a staining reagent;

[0111] Sperm head, acrosome region, post-acrosomal region, neck and middle segment, main segment segmentation module: used for segmentation of various components of sperm;

[0112] Sperm component matching and splicing module: used for matching and splicing of sperm components;

[0113] Use the acrosome region and post-acrosomal region contour correction and supplement module: used for the acrosome region and post-acrosomal region contour correction and supplement;

[0114] Mid-section contour supplement module: used to supplement the missing mid-section contour.

[0115] The present invention provides a computer device, comprising at least one processor and at least one memory, wherein the at least one memory is used to store at least one computer program; when the at least one computer program is executed by the at least one processor, the at least one processor implements the human sperm image segmentation processing method.

[0116] The present invention provides a storage medium, wherein the storage medium stores processor executable instructions, and when the executable instructions are executed by the processor, they are used to execute any one of the human sperm image segmentation processing methods described above.

[0117] The present invention has significant technical effects due to the adoption of the above technical solution:

[0118] In view of the above-mentioned technical problems of sperm segmentation, the beneficial effect of the present invention is to form a human sperm image segmentation processing method and device by combining multiple models and algorithms, which can solve or alleviate the above-mentioned technical problems.

[0119] The present invention can automatically segment sperm images through a device for segmenting and processing human sperm images, and completes the preparatory work of analyzing human sperm morphology, which helps to realize computer-assisted sperm morphology analysis, helps to reduce the mechanical and repetitive workload of doctors, improves the work efficiency of human sperm morphology analysis, and reduces the waste of social resources. The method and device proposed in the present invention can completely and accurately segment the five parts of the sperm head, acrosome region, post-acrosomal region, neck and middle section, and main section contour in the sperm image, and the segmented output of the contours of each component of the sperm is accurate, the contour missed detection rate is low, and the integrity is good. The sperm after the final matching combination has the advantages of high accuracy and low missed recognition rate. These advantages are very important for sperm morphology analysis, have important clinical auxiliary diagnosis value, and are conducive to the final output of more accurate and reliable human sperm morphology analysis conclusions. BRIEF DESCRIPTION OF THE DRAWINGS

[0120] Figure 1 Flow chart of the method of the present invention;

[0121] Figure 2 A schematic diagram of the structure of human sperm of the present invention;

[0122] Figure 3 A test image of the present invention;

[0123] Figure 4 is the segmented image of the present invention;

[0124] Figure 5 This is an image showing the results of matching and assembling all independent sperms according to the present invention. DETAILED DESCRIPTION

[0125] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0126] Example 1

[0127] A human sperm image segmentation processing method, comprising:

[0128] S1. Collect sperm images after being treated with staining reagents;

[0129] S2. Use the pre-trained instance segmentation model to segment the sperm head, acrosome region, post-acrosomal region, neck and middle segment, and main segment in the sperm image to obtain a sperm contour set, a sperm contour category set, and an inference output sperm confidence level set of the instance category in the sperm image;

[0130] S3. Post-processing the output result of the instance segmentation; obtaining a post-processed sperm contour set, a sperm contour category set, and a sperm confidence level set of the instance category output by reasoning;

[0131] S4. Based on the post-processed sperm contour set, sperm contour category set and the inferred output sperm confidence level set of the instance category, the contour sets of each part of the sperm are assembled and matched to obtain all the single independent sperm in the sperm image. A single independent sperm includes its head contour, acrosome region contour, post-acrosomal region contour, neck and mid-segment contour, main segment contour, and the inferred output confidence levels of each of these five parts.

[0132] Based on the post-processed sperm contour set, sperm contour category set and the sperm confidence level set of the instance category output by reasoning, the contour set of each part of the sperm is assembled and matched, including:

[0133] Using the sperm head contour, acrosome region contour, and post-acrosomal region contour matching and assembly algorithm, the sperm head contour, acrosome region contour, and post-acrosomal region contour are matched and assembled to obtain the first matching assembly result of all sperms in the sperm image;

[0134] For the first matching assembly result of all sperms in the sperm image, the contour correction and supplementation algorithm of the acrosome region and the post-acrosomal region is used to obtain the second matching assembly result of all sperms in the sperm image;

[0135] For the second matching assembly result of all sperms in the sperm image, the sperm head contour is matched and assembled with the neck and middle section contour using the sperm head, neck and middle section contour matching assembly model to obtain the third matching assembly result of all sperms in the sperm image;

[0136] For the third matching assembly result of all sperms in the sperm image, the third matching assembly result of all sperms in the sperm image is used with the main segment contour matching assembly model to realize the third matching assembly result of all sperms in the sperm image and the main segment contour matching assembly, so as to obtain the fourth matching assembly result of all sperms in the sperm image;

[0137] For the fourth matching assembly result of all sperms in the sperm image, a middle segment contour supplementation algorithm is used to supplement the possible middle segment missing in the fourth matching assembly result, so as to obtain the fifth matching assembly result of all sperms in the sperm image;

[0138] The fifth matching assembly result includes all the individual sperms in the sperm image. The individual sperms contain five-part sperm contours, which are the sperm head contour, the sperm acrosome region contour, the sperm post-acrosomal region contour, the sperm neck and mid-segment contour, the sperm main segment contour, and the confidence levels of the inferred output sperm five-part contours.

[0139] Post-processing of the instance segmentation output includes:

[0140] Set the instance segmentation post-processing parameters, set the instance segmentation post-processing parameter test_topk_per_image greater than 100; Set the instance segmentation post-processing parameter test_topk_per_image greater than 100. This technical means is used to at least partially solve the problem of missed detection of segmentation instances in sperm images with a large number of sperm. This technical means can at least partially solve the problem of missed detection of segmentation instances in sperm images with a large number of sperm. The principle is explained as follows:

[0141] For the MaskDINO model, the default post-processing parameter value test_topk_per_image=100. This default post-processing parameter may cause missed detection of segmented instances for sperm images with a large number of sperm. Explanation of missed detection reasons: In the MaskDINO model, test_topk_per_image is one of the instance segmentation post-processing parameters. Its function is to retain test_topk_per_image instances per image, that is, it presets how many targets to be segmented in an image. The default value is 100. Using the default value of 100, in the human sperm morphology image segmentation scenario, when the number of instance segmentation categories is 5, it is equivalent to 20 sperm in the preset image, 20=100 / 5. For sperm images with more than 20 sperm, sperm beyond 20 will be missed.

[0142] Enlarge the contour image output by the instance segmentation model to the size of the input image;

[0143] The sperm contour is processed by deleting the predicted contours below the preset contour confidence threshold, deleting the predicted contours with an area below the preset area threshold, and using the non-maximum suppression algorithm to delete the repeated contours, thereby obtaining the contour set of all sperm components in the sperm image after post-processing, the contour category set, and the confidence level set of the inference output instance category.

[0144] The matching and assembly algorithms for sperm head contour, acrosome region contour, and post-acrosomal region contour include:

[0145] The results of sperm image instance segmentation and post-processing are stored in their respective category lists according to instance contour categories;

[0146] Traverse the list of all sperm head contours, segment each sperm head contour, and find the matching acrosome region and post-acrosomal region contours, including:

[0147] The intersection and union ratio IoU1 of the bounding rectangular box of all the acrosome region contours obtained by sperm image segmentation and post-processing and the bounding rectangular box of the head contour is calculated. The acrosome region with the largest IoU1 is the acrosome region of the current sperm head; the intersection and union ratio IoU2 of the bounding rectangular box of all the post-acrosomal region contours obtained by sperm image segmentation and post-processing and the bounding rectangular box of the sperm head contour is calculated. The post-acrosomal region with the largest IoU2 is the post-acrosomal region of the current sperm head.

[0148] The contour correction and supplement algorithms of the acrosome and post-acrosomal regions include:

[0149] Based on the first matching assembly results of all sperm in the sperm image, traverse the first matching assembly results of all sperm, and for each of the first matching assembly results:

[0150] When both the acrosome region and the post-acrosomal region exist, the contour correction algorithm of the acrosome region and the post-acrosomal region is executed. The contour correction includes:

[0151] The union of the acrosome region outline, the post-acrosomal region outline and the head outline is taken as the final merged head outline;

[0152] For the acrosome region and the post-acrosomal region belonging to the same head, the contour with the highest confidence level inferred by the instance segmentation model is retained, which is the contour of the first part of the sperm head, and its contour category remains unchanged; for the contour with a low confidence level, the contour area of ​​the sperm head after the final merger is used, and the contour corresponding to the first part of the sperm head is subtracted, and the contour corresponding to the remaining area is the corrected contour of the other part of the sperm head. The corrected contour category is the category of the contour with a low confidence level; contour correction is completed;

[0153] If the head contour exists and one of the acrosome region or the post-acrosomal region is missing, the acrosome region and the post-acrosomal region contour supplementation algorithm is executed. The acrosome region and the post-acrosomal region contour supplementation algorithm includes:

[0154] The contour area of ​​the sperm head is used to subtract the contour area of ​​the intact sperm part to obtain the contour corresponding to the missing part area, which is the contour of the other missing part of the sperm head.

[0155] The sperm head contour matching and assembly algorithm with the neck and mid-section contours includes:

[0156] Constant c i,jIt represents the measure of the matching degree between the head contour in the i-th second matching assembly result and the j-th sperm mid-segment contour obtained by sperm image segmentation and post-processing. The specific calculation method is:

[0157] Where i represents the number subscript of the second matching assembly result, j represents the number subscript of the sperm mid-segment contour obtained by sperm image segmentation and post-processing; IoU i,j The intersection-and-union ratio of the rectangular bounding box of the head contour in the i-th second matching assembly result and the rectangular bounding box of the j-th sperm midsection contour obtained by sperm image segmentation and post-processing; when calculating the intersection-and-union ratio, the rectangular bounding box of the midsection contour is kept unchanged with the center point coordinates unchanged, and the width and height sides are enlarged by 1.3 times. This processing ensures that the elongated midsection rectangular bounding box will intersect with the adjacent sperm head after enlargement; distance i,j represents the Euclidean distance between the two nearest points of the head contour in the i-th second matching assembly result and the j-th sperm midsegment contour obtained by sperm image segmentation and post-processing; tail_middle_s j represents the contour area of ​​the jth sperm middle segment obtained by sperm image segmentation and post-processing; max(tail_middle_s) represents the maximum area of ​​the sperm middle segment contours obtained by all sperm image segmentation and post-processing in the current sperm image;

[0158] I is the indicator function,

[0159]

[0160] Clamp is a function that limits the range of variables.

[0161]

[0162] Decision variable x i,j Indicates whether the head contour in the second matching assembly result of the i-th sperm matches the j-th sperm midsection contour obtained by sperm image segmentation and post-processing; i represents the number subscript of the sperm second matching assembly result, and j represents the number subscript of the sperm midsection contour obtained by sperm image segmentation and post-processing;

[0163]

[0164] The mathematical model for matching the head contour with the midsection contour is described as follows:

[0165] Objective function:

[0166]

[0167] Subject to: xi,j =0 or x i,j =1 (2)

[0168]

[0169] Wherein, m represents the total number of second matching assembly results of all sperm in the sperm image, and n represents the total number of sperm mid-segment contours obtained by sperm image segmentation and post-processing;

[0170] Inequality (3) indicates that for each head contour in the second matching assembly result, there is at most one sperm mid-segment contour obtained by sperm image segmentation and post-processing that matches it;

[0171] Inequality (4) indicates that for each sperm mid-segment contour obtained by sperm image segmentation and post-processing, there is at most one head contour in the second matching assembly result that matches it;

[0172] The mathematical model for solving the matching of the head contour and the mid-segment contour is used. The optimal matching solution between the sperm head and the sperm mid-segment is the third matching assembly result of all sperm in the sperm image.

[0173] The third matching assembly result and the main segment contour matching assembly algorithm include:

[0174] By matching the middle segment’s flag middle_exist_flag i Describe whether the third matching assembly result has a matching middle segment;

[0175]

[0176] When the third matching assembly result has a matching middle segment, the matching degree between the middle segment contour of the third matching assembly result and the contours of each sperm main segment obtained by sperm image segmentation and post-processing is calculated; the matching degree d between the middle segment of the third matching assembly result and the contours of the sperm main segment obtained by sperm image segmentation and post-processing is calculated. i,j :

[0177]

[0178] Wherein, i represents the number subscript of the third matching assembly result, and j represents the number subscript of the sperm main segment contour obtained by sperm image segmentation and post-processing;

[0179] Clamp is a function that limits the range of variables:

[0180]

[0181] middle_exist_flag i =1, distance i,jrepresents the Euclidean distance between the two closest points between the middle segment contour of the i-th third matching assembly result and the j-th sperm main segment contour obtained by sperm image segmentation and post-processing; confidence j It indicates the confidence level of the main segment instance in the inference output of the instance segmentation model, where j represents the subscript number of the sperm main segment contour obtained by sperm image segmentation and post-processing;

[0182] When there is no matching middle segment in the third matching assembly result, the matching degree d between the head contour in the third matching assembly result and the contours of each sperm main segment obtained by sperm image segmentation and post-processing is calculated. i,j ;

[0183] For all main segment contours, extract its skeleton line and check whether the extended line of the skeleton line intersects with the head contour in the third matching assembly result. If not, then d i,j = 0, indicating that the matching degree is 0; if they intersect, the matching degree is d i,j The calculation rules are as follows:

[0184]

[0185] middle_exist_flag i = 0, distance i,j represents the Euclidean distance between the two closest points between the head contour of the i-th third matching assembly result and the j-th sperm main segment contour obtained by sperm image segmentation and post-processing; confidence j It indicates the confidence level of the main segment instance output by the segmentation model inference, where j represents the subscript number of the sperm main segment contour obtained by sperm image segmentation and post-processing;

[0186] Decision variable y i,j Indicates whether the i-th third matching assembly result matches the j-th sperm main segment contour obtained by sperm image segmentation and post-processing;

[0187]

[0188] Wherein, i represents the number subscript of the third matching assembly, and j represents the number subscript of the sperm main segment contour obtained by sperm image segmentation and post-processing;

[0189] The mathematical model description of the third matching assembly result and the main segment contour matching assembly,

[0190] Objective function:

[0191]

[0192] Subject to:

[0193] y i,j =0 or y i,j =1 (6);

[0194]

[0195] y i,j =0when(middle_exist_flag i =1)and((distance i,j

[0196] >dist_middle_main_thres)

[0197] or((dist_sperm_main_thres <distance i,j ≤

[0198] dist_middle_main_thres)and(head_tail_distance i,j >

[0199] dist_sperm_main_thres)))(9);

[0200] y i,j =0when(middle_exist_flag i =0)and(distance i,j >

[0201] k*long_axis i ) (10);

[0202] Wherein, m represents the total number of third matching assembly results in the sperm image, and n1 represents the number of sperm main segment contours obtained by sperm image segmentation and post-processing;

[0203] dist_middle_main_thres indicates the preset threshold value of the farthest distance between the middle segment contour and the main segment contour belonging to the same sperm; dist_sperm_main_thres indicates the preset threshold value of the farthest distance between the third matching assembly result belonging to the same sperm and the main segment contour; long_axis i Indicates the major axis length of the head contour in the i-th third matching assembly result; head_tail_distance i,j represents the Euclidean distance between the nearest two points of the head contour in the i-th third matching assembly result and the j-th sperm main segment contour obtained by sperm image segmentation and post-processing;

[0204] Inequality (7) indicates that each third matching assembly result has at most one sperm main segment contour obtained by sperm image segmentation and post-processing that matches it;

[0205] Inequality (8) indicates that each sperm main segment contour obtained by sperm image segmentation and post-processing has at most one third matching assembly result matching it;

[0206] Inequality (9) describes the situation where there is a matching middle segment in the third matching assembly result. Based on prior knowledge, when any of the following two situations is met, the corresponding third matching assembly result should not match the sperm main segment contour obtained by sperm image segmentation and post-processing;

[0207] Case 1: When the distance between the middle contour of the third matching assembly result and the sperm main contour obtained by sperm image segmentation and post-processing is greater than the distance threshold dist_middle_main_thres;

[0208] Case 2: When the distance between the middle contour of the third matching assembly result and the sperm main segment contour obtained by sperm image segmentation and post-processing, and the distance between the head contour in the third matching assembly result and the sperm main segment contour obtained by sperm image segmentation and post-processing, are both greater than the distance threshold dist_sperm_main_thres;

[0209] Inequality (10) describes the situation where there is no matching middle segment in the third matching assembly result. The head contour in the third matching assembly result and the sperm main segment contour obtained by sperm image segmentation and post-processing are greater than the distance threshold k*long_axis i When , it should not be matched. The distance threshold here is defined as k times the long axis of the head contour in the i-th third matching assembly result;

[0210] A mathematical model for matching and assembling the third matching assembly result with the sperm main segment contour obtained by sperm image segmentation and post-processing is solved. The optimal matching and assembly solution of the third matching assembly result and the sperm main segment is the fourth matching assembly result of all sperm in the sperm image.

[0211] A mid-segment contour supplementation algorithm in a human sperm image segmentation processing method, comprising:

[0212] If the fourth matching assembly result of all sperm in the sperm image has a missing middle segment, a line segment with a width equal to the width of the main segment outline is drawn between the two nearest points between the head outline and the main segment outline in the fourth matching assembly result, and the outer contour of the line segment is regarded as the missing middle segment outline, thereby obtaining the fifth matching assembly result of all sperm in the sperm image.

[0213] Example 2

[0214] Based on Example 1, this embodiment is a sperm image segmentation processing device, comprising:

[0215] An image acquisition module is used to acquire sperm morphology images after being treated with a staining reagent;

[0216] Sperm head, acrosome region, post-acrosomal region, neck and middle segment, main segment segmentation module: used for segmentation of various components of sperm;

[0217] Sperm component matching and splicing module: used for matching and splicing of sperm components;

[0218] Use the acrosome region and post-acrosomal region contour correction and supplement module: used for the acrosome region and post-acrosomal region contour correction and supplement;

[0219] Mid-section contour supplement module: used to supplement the missing mid-section contour.

[0220] Example 3

[0221] A computer device comprises at least one processor and at least one memory, wherein the at least one memory is used to store at least one computer program; when the at least one computer program is executed by the at least one processor, the at least one processor implements the human sperm image segmentation processing method.

[0222] Example 4

[0223] A storage medium, wherein the storage medium stores processor executable instructions, and when the executable instructions are executed by the processor, they are used to execute any of the human sperm image segmentation processing methods described above.

[0224] Example 5

[0225] Based on the above embodiment, this embodiment samples 40 human sperm morphology images stained by Papanicolaou and Diff-Quik, and tests show that the method, system and device for segmenting and analyzing human sperm images proposed in the present invention have almost no missed detection results, and the segmentation contour is very close to the realized sperm boundary, and the image segmentation and analysis effects of different batches of samples stained with two different reagents are very good. It has the advantages of strong robustness, high degree of automation and strong generalization.

[0226] The method is relatively fast. In the hardware environment of NVIDIA GeForce GTX 1650 graphics card and Intel(R) Core(TM) i7-9700 CPU@3.00GHz 3.00GHz, for a 1920x1200 three-channel sperm morphology image, the average total running analysis time of the segmentation and analysis method proposed by the present invention is 800ms / sperm image, and the running time of the sperm parts matching combination algorithm proposed by the present invention is within 50ms / sperm image.

Claims

1. A human sperm image segmentation processing method, include: S1. Collect sperm images after being treated with staining reagents; S2. Use the pre-trained instance segmentation model to segment the sperm head, acrosome region, post-acrosomal region, neck and middle segment, and main segment in the sperm image to obtain a sperm contour set, a sperm contour category set, and an inference output sperm confidence level set of the instance category in the sperm image; S3. Post-processing the output result of the instance segmentation; obtaining a set of contours of each part of the post-processed sperm, a set of sperm contour categories, and a set of sperm confidence levels of the instance category output by reasoning; S4. Based on the post-processed contour set of each sperm part, the sperm contour category and the inference output of the sperm confidence level of the instance category, the contour set of each sperm part is assembled and matched to obtain all the single independent sperm in the sperm image, wherein the single independent sperm includes its head contour, acrosome region contour, post-acrosomal region contour, neck and middle segment contour, main segment contour, and the inference output of the confidence level of each of these five parts; Based on the post-processed sperm contour set, sperm contour category set and the sperm confidence level set of the instance category output by reasoning, the contour set of each part of the sperm is assembled and matched, including: Using the sperm head contour, acrosome region contour, and post-acrosomal region contour matching and assembly algorithm, the sperm head contour, acrosome region contour, and post-acrosomal region contour are matched and assembled to obtain the first matching assembly result of all sperms in the sperm image; The first matching assembly result of all sperms in the sperm image is processed by using the contour correction of the acrosome region and the post-acrosomal region and the supplementary algorithm to obtain the second matching assembly result of all sperms in the sperm image; For the second matching assembly result of all sperms in the sperm image, the sperm head contour is matched and assembled with the neck and middle section contour using the sperm head, neck and middle section contour matching assembly model to obtain the third matching assembly result of all sperms in the sperm image; For the third matching assembly result of all sperms in the sperm image, the third matching assembly result of all sperms in the sperm image is used with the main segment contour matching assembly model to achieve the third matching assembly result of all sperms in the sperm image and the main segment contour matching assembly, so as to obtain the fourth matching assembly result of all sperms in the sperm image; For the fourth matching assembly result of all sperms in the sperm image, a middle segment contour supplementation algorithm is used to supplement the possible middle segment missing in the fourth matching assembly result, so as to obtain the fifth matching assembly result of all sperms in the sperm image; The fifth matching assembly result is all the single independent sperm information in the sperm image, and the single independent sperm information includes the five-part outline of the sperm. The five-part outline of the sperm are the sperm head outline, the sperm acrosome region outline, the sperm post-acrosomal region outline, the sperm neck and mid-piece outline, the sperm main segment outline, and the confidence level of each of the five-part outlines of the sperm that are inferred and output.

2. A human sperm image segmentation processing method according to claim 1, Features: Post-processing of the instance segmentation output includes: Set the instance segmentation post-processing parameters, set the instance segmentation post-processing parameters test_topk_per_image greater than 100; Enlarge the contour image output by the instance segmentation model to the size of the input image; The sperm contour is processed by deleting the predicted contours below the preset contour confidence threshold, deleting the predicted contours with an area below the preset area threshold, and using the non-maximum suppression algorithm to delete the repeated contours, thereby obtaining the contour set of all sperm components in the sperm image after post-processing, the contour category set, and the confidence level set of the inference output instance category.

3. A human sperm image segmentation processing method according to claim 1, It is characterized in that The sperm head contour, acrosome region contour, and post-acrosomal region contour matching and assembly algorithm comprises: The results of sperm image instance segmentation and post-processing are stored in their respective category lists according to instance contour categories; Traverse the list of all sperm head contours, segment each sperm head contour, and find the matching acrosome region and post-acrosomal region contours, including: The intersection and union ratio IoU1 of the bounding rectangular box of all the acrosome region contours obtained by sperm image segmentation and post-processing and the bounding rectangular box of the head contour is calculated. The acrosome region with the largest IoU1 is the acrosome region of the current sperm head; the intersection and union ratio IoU2 of the bounding rectangular box of all the post-acrosomal region contours obtained by sperm image segmentation and post-processing and the bounding rectangular box of the sperm head contour is calculated. The post-acrosomal region with the largest IoU2 is the post-acrosomal region of the current sperm head.

4. A human sperm image segmentation processing method according to claim 1, Features: The contour correction and supplement algorithms of the acrosome and post-acrosomal regions include: Based on the first matching assembly results of all sperm in the sperm image, traverse the first matching assembly results of all sperm, and for each of the first matching assembly results: When both the acrosome region and the post-acrosomal region exist, the contour correction algorithm of the acrosome region and the post-acrosomal region is executed. The contour correction includes: The union of the acrosome region outline, the post-acrosomal region outline and the head outline is taken as the final merged head outline; For the acrosome region and the post-acrosomal region belonging to the same head, the contour with the highest confidence level inferred by the instance segmentation model is retained, which is the contour of the first part of the sperm head, and its contour category remains unchanged; for the contour with a low confidence level, the contour area of ​​the sperm head after the final merger is used, and the contour corresponding to the first part of the sperm head is subtracted, and the contour corresponding to the remaining area is the corrected contour of the other part of the sperm head. The corrected contour category is the category of the contour with a low confidence level; contour correction is completed; If the head contour exists and one of the acrosome region or the post-acrosomal region is missing, the acrosome region and the post-acrosomal region contour supplementation algorithm is executed. The acrosome region and the post-acrosomal region contour supplementation algorithm includes: The contour area of ​​the sperm head is used to subtract the contour area of ​​the intact sperm part to obtain the contour corresponding to the missing part area, which is the contour of the other missing part of the sperm head.

5. The human sperm image segmentation processing method according to claim 1, Features: The sperm head contour matching and assembly algorithm with the neck and mid-section contours includes: Constant c i,j It represents the measure of the matching degree between the head contour in the i-th second matching assembly result and the j-th sperm mid-segment contour obtained by sperm image segmentation and post-processing. The specific calculation method is: Where i represents the number subscript of the second matching assembly result, j represents the number subscript of the sperm mid-segment contour obtained by sperm image segmentation and post-processing; IoU i,j The intersection-and-union ratio of the rectangular bounding box of the head contour in the i-th second matching assembly result and the rectangular bounding box of the j-th sperm midsection contour obtained by sperm image segmentation and post-processing; when calculating the intersection-and-union ratio, the rectangular bounding box of the midsection contour is kept unchanged with the center point coordinates unchanged, and the width and height sides are enlarged by 1.3 times. This processing ensures that the elongated midsection rectangular bounding box will intersect with the adjacent sperm head after enlargement; distance i,j represents the Euclidean distance between the two nearest points of the head contour in the i-th second matching assembly result and the j-th sperm midsection contour obtained by sperm image segmentation and post-processing; tail_middle_s j represents the contour area of ​​the jth sperm middle segment obtained by sperm image segmentation and post-processing; max(tail_middle_s) represents the maximum area of ​​the sperm middle segment contours obtained by all sperm image segmentation and post-processing in the current sperm image; I is the indicator function, Clamp is a function that limits the range of variables. Decision variable x i,j Indicates whether the head contour in the second matching assembly result of the i-th sperm matches the j-th sperm midsection contour obtained by sperm image segmentation and post-processing; i represents the number subscript of the sperm second matching assembly result, and j represents the number subscript of the sperm midsection contour obtained by sperm image segmentation and post-processing; The mathematical model for matching the head contour with the midsection contour is described as follows: Objective function: Subject to:x i,j =0 or x i,j =1 (2) Wherein, m represents the total number of second matching assembly results of all sperm in the sperm image, and n represents the total number of sperm mid-segment contours obtained by sperm image segmentation and post-processing; Inequality (3) indicates that for each head contour in the second matching assembly result, there is at most one sperm mid-segment contour obtained by sperm image segmentation and post-processing that matches it; Inequality (4) indicates that for each sperm mid-segment contour obtained by sperm image segmentation and post-processing, there is at most one head contour in the second matching assembly result that matches it; The mathematical model for solving the matching of the head contour and the mid-segment contour is used. The optimal matching solution between the sperm head and the sperm mid-segment is the third matching assembly result of all sperm in the sperm image.

6. A human sperm image segmentation processing method according to claim 1, Features: The third matching assembly result and the main segment contour matching assembly algorithm include: By matching the middle segment’s flag middle_exist_flag i Describe whether the third matching assembly result has a matching middle segment; When the third matching assembly result has a matching middle segment, the matching degree between the middle segment contour of the third matching assembly result and the contours of each sperm main segment obtained by sperm image segmentation and post-processing is calculated; the matching degree d between the middle segment of the third matching assembly result and the contours of the sperm main segment obtained by sperm image segmentation and post-processing is calculated. i,j : Wherein, i represents the number subscript of the third matching assembly result, and j represents the number subscript of the sperm main segment contour obtained by sperm image segmentation and post-processing; Clamp is a function that limits the range of variables: middle_exist_flag i =1, distance i,j represents the Euclidean distance between the two closest points between the middle segment contour of the i-th third matching assembly result and the j-th sperm main segment contour obtained by sperm image segmentation and post-processing; confidence j It indicates the confidence level of the main segment instance in the inference output of the instance segmentation model, where j represents the subscript number of the sperm main segment contour obtained by sperm image segmentation and post-processing; When there is no matching middle segment in the third matching assembly result, the matching degree d between the head contour in the third matching assembly result and the contours of each sperm main segment obtained by sperm image segmentation and post-processing is calculated. i,j ; For all main segment contours, extract its skeleton line and check whether the extended line of the skeleton line intersects with the head contour in the third matching assembly result. If not, then d i,j = 0, indicating that the matching degree is 0; if they intersect, the matching degree is d i,j The calculation rules are as follows: middle_exist_flag i = 0, distance i,j represents the Euclidean distance between the two closest points of the head contour of the i-th third matching assembly result and the j-th sperm main segment contour obtained by sperm image segmentation and post-processing; confidence j It indicates the confidence level of the main segment instance output by the segmentation model inference, where j represents the subscript number of the sperm main segment contour obtained by sperm image segmentation and post-processing; Decision variable y i,j Indicates whether the i-th third matching assembly result matches the j-th sperm main segment contour obtained by sperm image segmentation and post-processing; Wherein, i represents the number subscript of the third matching assembly, and j represents the number subscript of the sperm main segment contour obtained by sperm image segmentation and post-processing; The mathematical model of the third matching assembly result and the main segment contour matching assembly is described as follows: Objective function: Wherein, m represents the total number of third matching assembly results in the sperm image, and n1 represents the number of sperm main segment contours obtained by sperm image segmentation and post-processing; dist_middle_main_thres indicates the preset threshold value of the farthest distance between the middle segment contour and the main segment contour belonging to the same sperm; dist_sperm_main_thres indicates the preset threshold value of the farthest distance between the third matching assembly result belonging to the same sperm and the main segment contour; long_axis i Indicates the major axis length of the head contour in the i-th third matching assembly result; head_tail_distance i,j represents the Euclidean distance between the nearest two points of the head contour in the i-th third matching assembly result and the j-th sperm main segment contour obtained by sperm image segmentation and post-processing; Inequality (7) indicates that each third matching assembly result has at most one sperm main segment contour obtained by sperm image segmentation and post-processing that matches it; Inequality (8) indicates that each sperm main segment contour obtained by sperm image segmentation and post-processing has at most one third matching assembly result matching it; Inequality (9) describes the situation where there is a matching middle segment in the third matching assembly result. Based on prior knowledge, when any of the following two situations is met, the corresponding third matching assembly result should not match the sperm main segment contour obtained by sperm image segmentation and post-processing; Case 1: When the distance between the middle contour of the third matching assembly result and the sperm main contour obtained by sperm image segmentation and post-processing is greater than the distance threshold dist_middle_main_thres; Case 2: When the distance between the middle contour of the third matching assembly result and the sperm main segment contour obtained by sperm image segmentation and post-processing, and the distance between the head contour in the third matching assembly result and the sperm main segment contour obtained by sperm image segmentation and post-processing, are both greater than the distance threshold dist_sperm_main_thres; Inequality (10) describes the situation where there is no matching middle segment in the third matching assembly result. The head contour in the third matching assembly result and the sperm main segment contour obtained by sperm image segmentation and post-processing are greater than the distance threshold k*long_axis i When , it should not be matched. The distance threshold here is defined as k times the long axis of the head contour in the i-th third matching assembly result; A mathematical model for matching and assembling the third matching assembly result with the sperm main segment contour obtained by sperm image segmentation and post-processing is solved. The optimal matching and assembly solution of the third matching assembly result and the sperm main segment is the fourth matching assembly result of all sperm in the sperm image.

7. A human sperm image segmentation processing method according to claim 1, It is characterized in that The mid-segment contour supplementation algorithm comprises: If the fourth matching assembly result of all sperm in the sperm image has a missing middle segment, a line segment with a width equal to the width of the main segment outline is drawn between the two nearest points between the head outline and the main segment outline in the fourth matching assembly result, and the outer contour of the line segment is regarded as the missing middle segment outline, thereby obtaining the fifth matching assembly result of all sperm in the sperm image.

8. A human sperm image segmentation processing device, It is characterized in that include: An image acquisition module is used to acquire sperm morphology images after being treated with a staining reagent; Sperm head, acrosome region, post-acrosomal region, neck and middle segment, main segment segmentation module: used for segmentation of various components of sperm; Sperm component matching and splicing module: used for matching and splicing of sperm components; Acrosome and post-acrosomal region contour correction and supplement module: used for acrosome and post-acrosomal region contour correction and supplement; Mid-section contour supplement module: used to supplement the missing mid-section contour.

9. A computer device, It is characterized in that It comprises at least one processor and at least one memory, wherein the at least one memory is used to store at least one computer program; when the at least one computer program is executed by the at least one processor, the at least one processor implements the human sperm image segmentation processing method according to any one of claims 1 to 7.

10. A storage medium, It is characterized in that The storage medium stores processor executable instructions, and when the executable instructions are executed by the processor, they are used to execute the human sperm image segmentation processing method according to any one of claims 1 to 7.

Images