Sperm morphology training prediction method and system based on YOLO-OBB

Abstract

The invention relates to the technical field of sperm morphology detection, in particular to a sperm morphology training prediction method and system based on YOLO-OBB, and the method comprises the steps: collecting a sperm image, and carrying out the preprocessing of the sperm image, and obtaining a target image; frame selection is carried out on sperms in the target image by using a plurality of directional bounding boxes to obtain bounding boxes, sperm forms in the bounding boxes are labeled to obtain a labeled data set, and labeled labels are sperm counts, normal sperms, normal sperm heads, normal sperm middle segments and abnormal sperm main segments. According to the method, after the sperm part in the target image is selected through the OBB, the noise influence degree of the image in the bounding box can be calculated, and when the noise influence degree is larger than the preset threshold value, the image in the bounding box is deleted. The use of the images in the bounding box with noise influence to train the YOLO detection model is avoided, and the training efficiency and performance of the YOLO detection model are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of sperm morphology detection, and more specifically, to a sperm morphology training prediction method and system based on YOLO-OBB. Background Art

[0002] Semen analysis is an important way to determine male fertility, and one of the more commonly used tests is sperm morphology. Sperm morphology detection is an important method for detecting sperm deformity rate. The basic process of common morphology detection is that medical workers treat semen with dyes, then observe it under a 100x microscope, and manually classify and count the morphology. Conventional detection methods are greatly affected by conditions such as the detection environment, the experience and level of the inspectors, and are time-consuming, labor-intensive, and have low accuracy, which can easily cause errors in the test results and bring difficulties to clinical treatment and scientific research. Computer-assisted sperm morphology analysis is to stain sperm with dyes to make sperm inactive, then collect static sperm images through an image acquisition system, and finally use algorithms to detect sperm morphology. The detection parameters of this method mainly include sperm morphology, deformity classification, and other cell categories.

[0003] Taking the deep learning method as an example, the target detection model can be used to identify and locate individual sperm in the sperm morphology image, and extract the individual sperm image in the sperm morphology image; then the classification model can be used to classify and identify the morphology of the individual sperm images to obtain the sperm morphology detection results. However, when training the target detection model, it is necessary to select and annotate the various parts of the sperm in the sperm image. However, the rectangular frame currently used is larger than the actual area of ​​the sperm, and other parts of the sperm may be selected, which will affect the elements of model training, increase the difficulty of training, and reduce the accuracy of sperm head morphology analysis. Summary of the invention

[0004] The present invention provides a sperm morphology training prediction method and method based on YOLO-OBB, aiming to solve the problem that the range marked by the rectangular frame currently used in the related art is larger than the actual area of ​​the sperm, which may frame other parts of the sperm, thereby affecting the elements of model training, increasing the difficulty of training, and reducing the accuracy of sperm head morphology analysis.

[0005] In a first aspect, the present invention provides a sperm morphology training and prediction method based on YOLO-OBB, comprising: collecting a sperm image, preprocessing the sperm image to obtain a target image; selecting sperm in the target image using multiple directional bounding box methods to obtain each bounding box, and annotating the sperm morphology within the bounding box to obtain an annotated data set, wherein the annotated labels are sperm count, normal sperm, normal sperm head, normal sperm mid-segment and abnormal sperm main segment; training a YOLO detection model using the annotated data set to obtain a trained YOLO detection model; inputting the target image into the trained YOLO detection model to predict the sperm morphology label.

[0006] Furthermore, the sperm image is preprocessed, wherein the preprocessing includes denoising, graying and image enhancement.

[0007] Furthermore, it also includes: calculating the degree of noise influence of the image in the boundary box corresponding to the sperm counting label, the degree of noise influence indicates that the difference between the mean of the grayscale value of the pixel points of the image in the boundary box and the mean of the grayscale value of the pixel points of all images in the boundary boxes is positively correlated; based on the size of the noise influence degree of the image in the boundary box, adjusting the boundary box.

[0008] Further, the bounding box is adjusted, including: normalizing the degree of noise influence on the image in the bounding box; in response to the degree of noise influence on the image in the bounding box after the normalization process being greater than a preset threshold, deleting the image in the bounding box; in response to the degree of noise influence on the image in the bounding box after the normalization process being less than or equal to the preset threshold, retaining the image in the bounding box.

[0009] Furthermore, it also includes: obtaining the size of the sperm count according to the attributes of the bounding box, wherein the attributes include the length and width of the bounding box, wherein the sperm count indicates that the bounding box selects sperm and the sperm head is selected and marked; analyzing the sperm morphology based on the size of the sperm count to determine whether the sperm head meets the standard.

[0010] Furthermore, the sperm morphology is analyzed based on the size of the sperm count to determine whether the sperm head meets the standards, including: when the length, width and aspect ratio of the sperm head all meet the standards, the sperm head meets the standards for subsequent reproductive health analysis; when any of the length, width and aspect ratio of the sperm head does not meet the standards, the sperm head does not meet the standards.

[0011] Furthermore, the empirical value of the preset threshold is 0.7.

[0012] Further, in response to the length, width and aspect ratio of the sperm head all meeting the standards, including: in response to the length of the sperm head being within a first credible limit interval, the length of the sperm head meets the standards; in response to the width of the sperm head being within a second credible limit interval, the width of the sperm head meets the standards; in response to the aspect ratio of the sperm head being within a third credible limit interval, the aspect ratio of the sperm head meets the standards.

[0013] In a second aspect of the present invention, a sperm morphology training prediction system based on YOLO-OBB is also provided, comprising a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement any of the above-described sperm morphology training prediction methods based on YOLO-OBB.

[0014] Beneficial effect: After selecting the sperm part in the target image through OBB, the noise impact degree of the image in the bounding box can be calculated. When the noise impact degree is greater than the preset threshold, the image in the bounding box is deleted. Avoid using these images in the bounding box with noise impact to train the YOLO detection model, and improve the training efficiency and performance of the YOLO detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The following detailed description is read with reference to the accompanying drawings, which illustrate several embodiments of the present invention in an exemplary and non-limiting manner, and in which like or corresponding reference numerals represent like or corresponding parts, wherein: Figure 1 is a flowchart schematically illustrating sperm morphology prediction according to an embodiment of the present invention; Figure 2 FIG. 1 is a flowchart schematically showing whether sperm is qualified according to an embodiment of the present invention. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0017] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0018] Taking the deep learning method as an example, the target detection model can be used to identify and locate a single sperm in the sperm morphology image. The target detection model can be a YOLOv8 target detection model to extract a single sperm image from the sperm morphology image; then the classification model is used to classify and identify the morphology of the individual sperm images to obtain the sperm morphology detection results. However, when training the target detection model, it is necessary to select and annotate the various parts of the sperm in the sperm image. However, the range of the rectangular frame currently used is larger than the actual area of ​​the sperm, and other parts of the sperm may be selected, which will affect the elements of model training, increase the difficulty of training, and reduce the accuracy of sperm head morphology analysis.

[0019] like Figure 1 and Figure 2 As shown, S101: collecting a sperm image, and preprocessing the sperm image to obtain a target image.

[0020] In one embodiment, when collecting a sperm sample, the male discharges semen directly into a dedicated clean container by masturbation. The container should be provided by a medical institution and must be sterile and dry. The sperm sample is then placed on a microscope slide and photographed using a microscope to obtain a sperm image. Different staining methods can be used to enhance image contrast as needed. Common stains include: Giemsa staining: After staining, the structure of the sperm head and tail is clearer. Caprylic acid staining: Staining increases the contrast of the cell membrane, making the sperm morphology easier to identify.

[0021] It should be noted that the microscope equipment can be an optical microscope, a digital microscope, or a confocal microscope. The appropriate microscope should be selected according to the needs. The functions of each microscope are as follows: Optical microscope: For general sperm analysis, an optical microscope is sufficient to provide sufficient image quality. The microscope needs to have a higher magnification (such as 400x or 1000x) to clearly observe the morphology of sperm. Digital microscope: Modern digital microscopes are usually equipped with high-resolution cameras that can directly transfer images to a computer. Digital microscopes can provide higher precision and facilitate subsequent analysis. Confocal microscope: For analysis requiring higher image quality and precision, a confocal microscope can be used. It can provide higher resolution and clarity, and is suitable for fine morphological studies.

[0022] In one embodiment, after obtaining the sperm image, the sperm image is preprocessed, wherein the preprocessing includes denoising, graying and image enhancement. The reason is that the sperm image may be interfered by various noises during the acquisition process, and these noises may come from the equipment itself, environmental factors or the transmission process. The presence of noise will seriously affect the quality of the image and reduce the accuracy of subsequent processing. Therefore, denoising is the first step in sperm image preprocessing. Graying is the process of converting a color image into a grayscale image. In sperm images, color information may not be necessary for subsequent analysis and recognition, while grayscale images can simplify the processing process and reduce the amount of calculation. Image enhancement is to improve the visual effect of the image and enhance the useful information in the image. In sperm images, it may be necessary to enhance the contrast, brightness and other features of the target for better recognition and analysis. Data enhancement operations are performed on the data set of the target image, wherein the data enhancement operations include random cropping, random rotation, scaling and translation operations to increase the amount and diversity of data, which helps to improve the generalization ability of the model.

[0023] S102: Train the YOLO detection model.

[0024] In one embodiment, after obtaining the target image, different parts of the sperm in the target image are framed using multiple directional bounding box methods. Compared with the axis-aligned bounding box, the directional bounding box has less redundant content framed by the directional bounding box, which reduces the difficulty of model training. Then the sperm morphology in the bounding box is annotated, wherein the annotated labels are sperm count, normal sperm, normal sperm head, normal sperm midsection and abnormal sperm main section, respectively, to obtain a labeled data set. The data set is divided into a training set and a validation set in proportion. The training set is input into the YOLO detection model for training, and the labeled target image is output. The YOLO detection model is verified using the validation set, and the YOLO detection model is obtained after the evaluation performance is achieved.

[0025] OBB (Oriented Bounding Box) is a different box selection method from the traditional Axis-Aligned Bounding Box (AABB). The bounding box of AABB selection is always aligned with the coordinate axis, while OBB can be rotated to align with the actual direction of the object, which can more accurately describe the shape and position of the object, especially when the object is rotated.

[0026] It should be noted that "sperm count" means that the selected content is sperm, and the sperm head is manually selected through OBB for annotation; "normal sperm" means normal sperm morphology, which is manually determined and selected, and then the other parts except the sperm tail are selected through OBB for annotation; "normal sperm head" means normal sperm head morphology, which is manually determined and selected, and then the normal sperm head is selected through OBB for annotation; "normal sperm midsection" means normal sperm midsection morphology, which is manually determined and selected, and then the normal sperm midsection and part of the sperm head are selected through OBB for annotation; "abnormal sperm main segment" means abnormal sperm main segment morphology, which is manually determined and selected, and then the abnormal sperm main segment is selected through OBB for annotation; because the normal head and midsection morphology of sperm are highly consistent, the normal morphology is marked; because the consistency of the normal main segment of sperm is low and the abnormal morphology is obvious, the obvious difference is easier to help YOLO training, so the abnormal sperm main segment morphology is marked.

[0027] In one embodiment, for the image in the bounding box, the image in the bounding box may be affected by noise, which may affect the training effect of the YOLO detection model. At this point, the degree of noise influence in each image in the bounding box can be calculated, and the bounding box can be adjusted according to the degree of noise influence of each image in the bounding box, wherein the adjustment is to delete the image in the bounding box, reduce the images in the bounding box with a larger degree of noise influence, and improve the accuracy of the training YOLO detection model.

[0028] In one embodiment, the noise influence degree of the image in the boundary box corresponding to the sperm counting label is calculated, and the noise influence degree indicates that the difference between the mean grayscale value of the pixel points of the image in the boundary box and the mean grayscale value of the pixel points of all images in the boundary boxes is positively correlated.

[0029] In one embodiment, the noise influence degree of the image within the bounding box corresponding to the sperm counting label is calculated, and the calculation formula is: ; In the formula, Indicates the degree of noise influence on the image within the bounding box corresponding to the i-th sperm counting label, represents the mean grayscale value of all pixels in the image within the bounding box corresponding to the i-th sperm counting label, represents the mean gray value of all pixels in the image within the bounding box corresponding to the qth sperm counting label, N represents the total number of bounding boxes corresponding to all sperm counting labels, Represents the standard normalization function.

[0030] The greater the difference between the mean grayscale value of the pixel points in the image within the i-th bounding box and the mean grayscale value of the pixel points in all bounding boxes, the greater the difference, indicating that the image within the i-th bounding box has a greater noise impact. And according to the noise impact degree of the image within the bounding box corresponding to the i-th sperm counting label Determine whether to adjust the bounding box. When the i-th sperm count label corresponds to the noise influence of the image in the bounding box When the noise level of the image in the boundary box corresponding to the i-th sperm counting label is greater than the preset threshold, the image in the boundary box is deleted. When it is less than or equal to the preset threshold, the image in the boundary box is retained. In this embodiment, the empirical value of the preset threshold is 0.7, and in other embodiments, the empirical value of the preset threshold may be 0.75 or 0.8, etc., which may be adjusted according to specific implementation conditions.

[0031] Through the above steps, after selecting the sperm part in the target image through OBB, the noise influence degree of the image in the bounding box can be calculated. When the noise influence degree is greater than the preset threshold, the image in the bounding box is deleted. Avoid using these images in the bounding box with noise influence to train the YOLO detection model, and improve the training efficiency and performance of the YOLO detection model.

[0032] S103: Input the target image into the trained YOLO detection model to predict the morphological label of sperm.

[0033] In one embodiment, the target image is input into a trained YOLO detection model, and the labeled target image is input to obtain the sperm count, normal sperm, normal sperm head, normal sperm mid-piece and abnormal sperm main section labeled on the target image.

[0034] S201: Obtain the size of the sperm head.

[0035] In one embodiment, after obtaining each sperm morphology, the sperm morphology can be analyzed, and the analysis process is to determine whether the sperm head meets the standard. Thus, the size of each sperm head and whether it meets the standard are obtained, so as to perform subsequent fertility health analysis, wherein the length and width of the bounding box are the size of the sperm head.

[0036] S202: Determine whether the sperm head meets the standards.

[0037] In one embodiment, the process of determining whether the sperm head meets the standard is as follows: according to the attributes of the bounding box, wherein the attributes include the length and width of the bounding box, the size of the sperm count is obtained, wherein the length and width of the bounding box are the length and width of the sperm head; based on the size of the sperm count, the sperm morphology is analyzed to determine whether the sperm head meets the standard. Determining whether the sperm head meets the standard includes: if the length, width and aspect ratio of the sperm head all meet the standard, then the sperm head meets the standard; if any of the length, width and aspect ratio of the sperm head does not meet the standard, then the sperm head does not meet the standard.

[0038] In one embodiment, in response to the length, width and aspect ratio of the sperm head meeting the standard, the method includes: if the length of the sperm head is within a first confidence interval, the length of the sperm head meets the standard. In this embodiment, the first confidence interval is 3.7 μm to 4.7 μm. If the width of the sperm head is within a second confidence interval, the width of the sperm head meets the standard. In this embodiment, the second confidence interval is 2.5 μm to 3.2 μm. If the aspect ratio of the sperm head is within a third confidence interval, the aspect ratio of the sperm head meets the standard. In this embodiment, the second confidence interval is between 1.3 and 1.8.

[0039] The present invention also provides a sperm morphology training prediction system based on YOLO-OBB. The system includes a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, a sperm morphology training prediction method based on YOLO-OBB according to the first aspect of the present invention is implemented.

[0040] The system also includes other components familiar to those skilled in the art, such as a communication bus and a communication interface, whose configuration and functions are known in the art and thus will not be described in detail here.

[0041] In the present invention, the aforementioned memory may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus or device. For example, a computer-readable storage medium may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc., or any other medium that can be used to store the required information and can be accessed by an application, a module, or both. Any such computer storage medium may be part of a device or accessible or connectable to a device. Any application or module described in the present invention may be implemented using computer-readable / executable instructions that may be stored or otherwise maintained by such a computer-readable medium.

[0042] In the description of this specification, "plurality" or "several" means at least two, such as two, three or more, etc., unless otherwise clearly and specifically defined.

[0043] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0044] The above-mentioned embodiments only express several implementation methods of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent application. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A sperm morphology training prediction method based on YOLO-OBB, characterized in that: include: Collecting sperm images, preprocessing the sperm images, and obtaining target images; The sperm in the target image is framed by using a directional bounding box method to obtain each bounding box, and the sperm morphology in the bounding box is annotated to obtain an annotated data set, wherein the annotated labels are sperm count, normal sperm, normal sperm head, normal sperm mid-segment and abnormal sperm main segment; Use the labeled data set to train the YOLO detection model to obtain a trained YOLO detection model; The target image is input into the trained YOLO detection model to predict the morphological label of sperm.

2. The sperm morphology training prediction method based on YOLO-OBB according to claim 1, characterized in that: The sperm image is preprocessed, wherein the preprocessing includes denoising, graying and image enhancement.

3. The sperm morphology training prediction method based on YOLO-OBB according to claim 1, characterized in that: Also includes: Calculating the noise influence degree of the image in the boundary box corresponding to the sperm counting label, the noise influence degree indicates that the difference between the mean grayscale value of the pixel points of the image in the boundary box and the mean grayscale value of the pixel points of all the images in the boundary boxes is positively correlated; The bounding box is adjusted based on the degree of noise influence on the image within the bounding box.

4. The sperm morphology training prediction method based on YOLO-OBB according to claim 3, characterized in that: Adjustments are made to this bounding box, including; Normalizing the noise impact degree of the image within the boundary box; In response to the noise impact degree of the image within the boundary box after normalization being greater than a preset threshold, deleting the image within the boundary box; In response to the noise influence degree of the image within the boundary box after the normalization processing being less than or equal to a preset threshold, the image within the boundary box is retained.

5. The sperm morphology training prediction method based on YOLO-OBB according to claim 1, characterized in that: Also includes: According to the attributes of the bounding box, wherein the attributes include the length and width of the bounding box, the size of the sperm count is obtained, wherein the sperm count indicates that the bounding box selects sperm and the sperm head is selected and marked; The sperm morphology is analyzed based on the size of the sperm count to determine whether the sperm head meets the standard.

6. The sperm morphology training prediction method based on YOLO-OBB according to claim 5, characterized in that: Analyzing the sperm morphology based on the size of the sperm count to determine whether the sperm head meets the standard includes: In response to the length, width and aspect ratio of the sperm head all meeting the standards, the sperm head meets the standards for subsequent fertility health analysis; In response to any one of the length, width and aspect ratio of the sperm head not meeting the standard, the sperm head does not meet the standard.

7. The sperm morphology training prediction method based on YOLO-OBB according to claim 4, characterized in that: The empirical value of the preset threshold is 0.

7.

8. The sperm morphology training prediction method based on YOLO-OBB according to claim 1, characterized in that: In response to the length, width and aspect ratio of the sperm head meeting the standards, including: In response to the length of the sperm head being within the first confidence limit interval, the length of the sperm head meets the standard; In response to the width of the sperm head being within the second confidence limit interval, the width of the sperm head meets the standard; In response to the length-to-width ratio of the sperm head being within the third confidence limit interval, the length-to-width ratio of the sperm head meets the standard.

9. A sperm morphology training prediction system based on YOLO-OBB, comprising a processor and a memory, characterized in that: The memory stores a computer program, and the processor executes the computer program to implement the sperm morphology training prediction method based on YOLO-OBB as described in any one of claims 1 to 8.

Images