Mark-free sperm quality evaluation method, system and equipment

Abstract

The invention discloses an unmarked sperm quality evaluation method, system and equipment. On the basis of a high-throughput and multi-dimensional flow cytometry imaging system, intensity, phase, bright field and dark field images of sperm cells are obtained, a mass sperm cell image database is established, and a sperm cell multi-dimensional image analysis model based on a multi-model fusion deep learning network is constructed, trained and optimized. On the basis, a trained sperm cell multi-dimensional image analysis network model is adopted to analyze sperm cell images obtained by detecting a semen sample by a multi-dimensional flow cytometry imaging system, and sperm structure and function parameters such as the proportion of sperms in the semen sample and the sperm DNA fragmentation rate are analyzed; and high-speed, high-precision and label-free sperm quality evaluation is realized.

Description

Technical Field

[0001] The present invention relates to a label-free sperm quality assessment method, system and device, and in particular to a sperm quality detection method based on an optofluidic time-domain stretching microscopy imaging system, quantitative phase imaging, optical polarization imaging and a multi-model fusion algorithm. Background Art

[0002] Clinically, it is generally believed that reduced sperm count and reduced sperm quality are the main signs of male infertility. Semen examination is an important means to evaluate male fertility. Among them, routine semen examination is the most common in clinical practice, mainly used to evaluate the number of sperm in semen, sperm density, sperm motility, liquefaction time and sperm morphology. Among them, sperm morphological analysis, as an indispensable part of semen examination, can qualitatively characterize the structural characteristics of sperm. The analysis results can reflect the deformity rate and fertilization ability of sperm, providing a basis for clinical diagnosis and treatment. Similarly, the DNA fragmentation rate reflects the integrity and stability of sperm DNA. A high level of DNA fragmentation rate usually means that the sperm genome has a higher damage rate. The normal sperm morphology rate and DNA fragmentation rate are positively correlated with the success rate of assisted reproductive technology. Selecting sperm with normal morphology and DNA fragmentation rate for assisted reproductive technology can improve the in vitro fertilization rate. However, sperm morphology is diverse and the classification criteria are complex. In particular, sperm with abnormal morphology is limited by existing detection technology, which brings great challenges to achieving high-precision sperm morphological analysis while maintaining cell activity. At the same time, DNA fragmentation rate detection also has similar problems as morphological detection. Therefore, once high-precision label-free detection is achieved, it will not only be of great help in evaluating sperm quality, but will also provide important information references for sperm formation, assisted reproductive technology, biological evolution and other aspects.

[0003] At present, the most traditional and widely used method for sperm morphology detection in clinical practice is smear microscopy, which stains the sperm on the smear to observe whether the sperm morphology is abnormal under an optical microscope. During the test, the doctor needs to evaluate and analyze the head, neck, middle section and tail of each sperm to confirm whether it is a normal sperm. This places high demands on the doctor's professional knowledge, professional level and work experience. It is easy for doctors to give different judgment results due to subjective differences in the judgment criteria. In addition, the workload is huge, time-consuming and labor-intensive, which brings great burden to the doctor's work. In order to solve the problems of strong subjectivity and labor-intensiveness in smear microscopy and improve the automation of sperm morphology analysis, computer-assisted sperm morphometric analysis (CASMA) is increasingly used in clinical practice. At present, CASMA usually analyzes the images or videos of sperm cells obtained by a combination of optical microscope and CCD or CMOS, and then extracts sperm cells through image enhancement, threshold segmentation, edge detection and other technologies, and segments the sperm head, neck and tail. However, due to methodological differences, such as sample staining methods, focus and illumination of optical microscopes, and the technical difficulty of correctly distinguishing sperm heads from cell fragments and dye particles in semen, the accuracy and reproducibility of CASMA analysis results are affected.

[0004] At present, several commonly used detection methods for sperm DNA fragmentation rate each have different technical characteristics and application scenarios. The sperm chromatin structure analysis (SCSA) method is the most widely used. It uses flow cytometry to detect the stability of chromatin to evaluate DNA integrity, and the results are expressed as DNA fragmentation index (DFI). The advantages of SCSA are high throughput and relatively high accuracy. The TUNEL method directly detects DNA damage by marking the ends of DNA breaks. It can be analyzed using a fluorescence microscope or flow cytometer and is suitable for detection that requires high sensitivity. The comet assay (single cell gel electrophoresis) identifies DNA damage by detecting the tail shape formed by the dragging of damaged DNA in an electric field. It is suitable for occasions where in-depth analysis of individual cells is required. Another method is the SCD method, namely the Halosperm test, which determines the integrity of DNA by detecting the halo formed by sperm DNA without nucleosomes. This method is known for its simplicity and speed. However, these methods inevitably require complex staining operations and damage the original sperm, which affects the subsequent selection of suitable sperm for assisted reproduction.

[0005] The Chinese patent application (application number: CN202111130886.8) published on January 11, 2022 provides a sperm head detection method and system. The invention uses a convolutional neural network to automatically detect the sperm head, which improves the efficiency and accuracy of sperm detection and effectively helps doctors perform sperm morphology analysis. The Chinese patent application (application number: CN202210673228.1) published on June 15, 2022 provides an automatic semen analysis method based on multi-frame information fusion and occlusion perception. The present invention proposes for the first time a combination of an edge-sensitive U-Net model and an occlusion-aware tracker based on joint probability data association to achieve accurate and stable sperm detection and tracking, and can be easily integrated into existing computer-assisted sperm analysis systems to provide semen analysis data with better accuracy and robustness. The Chinese patent application (application number: CN202110789070.X) published on October 8, 2021 discloses the application and staining method of Shorr staining reagent. The kit uses fresh testicular puncture biopsy tissue frozen sections as the staining object, which can dye different parts of sperm into different colors and effectively distinguish sperm from background cells, improve the recognition of sperm, and shorten the diagnosis time of sperm pathological morphology. The Chinese patent application (application number: CN202111677213.4) published on April 8, 2022 proposes a human sperm morphology staining reagent and staining method. The human sperm staining solution SVS includes: 0.1-1M sodium chloride solution, 20-100mM eosin solution, 100-200g / L nigrosine solution, 0.01-1% formaldehyde solution, 1M sodium hydroxide solution, and hydrochloric acid solution to adjust the pH value to 7.3-9.5. This invention is a one-step method, simple and efficient. Only one bottle of reagent can meet all staining needs. Staining can be completed in only 30 to 60 seconds. After air drying, human sperm staining agent sperm morphology can be observed under a microscope. The Chinese patent application (application number: CN202210275918.1) published on May 24, 2022 discloses a sperm analysis slide and detection system, which can observe multiple samples at a time to improve detection efficiency; the depth of the upper surface of the sample groove from the electrostatic membrane is no more than 8 mm, which is convenient for single-layer diffusion imaging of semen, which is conducive to directly observing the morphological information of live sperm and improving detection accuracy.

[0006] The Chinese patent (application number: CN202111662563.3) published on December 30, 2021 discloses a kit and method for detecting DNA fragmentation of active sperm. By using carbon quantum dots as Pi detection reagents, based on flow cytometry, and in combination with a sperm chromatin structure detection method based on acridine orange dye, it is possible to detect sperm motility and DNA fragmentation rate of active sperm. The Chinese patent application (application number: CN202211019378.7) published on November 15, 2022 discloses a DNA fragmentation analysis method based on deep learning and related equipment. The DNA fragmentation analysis method based on deep learning and related equipment obtains the target image and predicts the sperm DNA fragmentation category for each pixel in the image. The ratio of the halo size data of the halo non-fragment to the approximate short diameter of the sperm is used to judge the suspected fragments in the halo non-fragment, and finally the halo-free fragments and the suspected fragments are used to estimate the sperm DNA fragmentation rate.

[0007] The above patents use machine learning to analyze sperm microscopic images or develop new sperm staining methods, but there are still problems with the small number of sperm evaluated and the need for complex staining operations. Therefore, comprehensive, label-free sperm cell testing is urgently needed to achieve accurate evaluation of sperm morphology and DNA fragmentation rate. Summary of the invention

[0008] In order to solve the problem that the number of sperm cells observed in the existing sperm morphology and DNA fragmentation rate detection is small and complex staining operations are required, the present invention provides a label-free sperm quality assessment method, system and equipment, which adopts a trained sperm cell multidimensional image analysis network model to analyze the sperm cell images obtained by detecting semen samples by a multidimensional flow cytometer imaging system, analyzes the proportion of each sperm morphology category in the semen sample, sperm DNA fragmentation rate and other sperm structure and function parameters, and realizes high-speed, high-precision and label-free sperm quality assessment.

[0009] According to one aspect of the present invention, there is provided a method for assessing sperm quality without labeling, comprising: The sperm cells in the semen samples were detected using a multi-dimensional flow cytometric imaging system to obtain the intensity, phase, bright field and dark field images of the sperm cells; Inputting the intensity, phase, bright field and dark field images of the acquired sperm cells into the trained sperm cell morphology classification and DNA fragmentation rate prediction network model, outputting sperm cell morphology classification and DNA fragmentation rate prediction results, and evaluating sperm quality according to the sperm cell morphology classification and DNA fragmentation rate prediction results; The training of the sperm cell morphology classification and DNA fragmentation rate prediction network model includes: Construct a sperm cell image database containing all sperm morphology categories and sperm DNA fragmentation rates, and distinguish sperm cells of different morphology categories and different DNA fragmentation rates; Construct a sperm cell multidimensional image analysis framework based on a multi-model fusion deep learning network to extract and analyze the multidimensional features of sperm cell images with known morphological categories and DNA fragmentation rates, and establish a sperm cell morphology classification and DNA fragmentation rate prediction network model; The sperm cell image database is used to train the established sperm cell morphology classification and DNA fragmentation rate prediction network model, and the trained sperm cell morphology classification and DNA fragmentation rate prediction network model is output.

[0010] As a further technical solution, the training of the sperm cell morphology classification and DNA fragmentation rate prediction network model also includes: verifying the prediction results of the model through clinical sperm morphology and DNA fragmentation rate index detection results, and optimizing the deep learning algorithm based on the verification results.

[0011] As a further technical solution, a multi-dimensional flow cytometric imaging system is used to detect sperm cells in a semen sample, including: Based on the optofluidic time-domain stretch imaging system, a quantitative phase imaging system is integrated to obtain the intensity and phase images of sperm cells; Based on the previous step, an optical polarization imaging system was integrated to obtain bright-field and dark-field images of sperm cells.

[0012] As a further technical solution, a sperm cell image database containing all sperm morphology categories and sperm DNA fragmentation rates is constructed, and sperm cells of different morphology categories and different DNA fragmentation rates are distinguished, including: Collect semen samples, and obtain a number of sperm cell images containing different morphological categories and different DNA fragmentation rate indices after detection by a multi-dimensional flow cytometric imaging system; The image clustering method is used to preliminarily classify the acquired sperm cell images of different categories, and the sperm cell images that are misclassified in image clustering are selected and correctly classified through manual screening to establish a sperm cell image database.

[0013] As a further technical solution, the established sperm cell morphology classification and DNA fragmentation rate prediction network model includes: a feature extraction module, which is used to extract the intensity and phase image features of sperm cells through the Yolo-V3 network, and to extract the structural contour features of sperm cells through the ShumleNet-V2 network; a feature stacking module, which is used to stack all the extracted features using the Concate method; a feature classification module, which is used to classify the images of sperm cells in various dimensions using the VGG-16 network to obtain sperm category classification and DNA fragmentation index prediction results.

[0014] According to one aspect of the present invention, there is provided a marker-free sperm quality assessment system, comprising: An image acquisition module is used to detect sperm cells in a semen sample using a multi-dimensional flow cytometric imaging system to obtain intensity, phase, bright field and dark field images of the sperm cells; An image processing module is used to input the intensity, phase, bright field and dark field images of the acquired sperm cells into the trained sperm cell morphology classification and DNA fragmentation rate prediction network model, output sperm cell morphology classification and DNA fragmentation rate prediction results, and evaluate sperm quality according to the sperm cell morphology classification and DNA fragmentation rate prediction results; The training of the sperm cell morphology classification and DNA fragmentation rate prediction network model includes: Construct a sperm cell image database containing all sperm morphology categories and sperm DNA fragmentation rates, and distinguish sperm cells of different morphology categories and different DNA fragmentation rates; Construct a sperm cell multidimensional image analysis framework based on a multi-model fusion deep learning network to extract and analyze the multidimensional features of sperm cell images with known morphological categories and DNA fragmentation rates, and establish a sperm cell morphology classification and DNA fragmentation rate prediction network model; The sperm cell image database is used to train the established sperm cell morphology classification and DNA fragmentation rate prediction network model, and the trained sperm cell morphology classification and DNA fragmentation rate prediction network model is output.

[0015] According to one aspect of the present invention, there is provided a label-free sperm quality assessment system, comprising: a multidimensional flow cytometric imaging system and a computing device; the multidimensional flow cytometric imaging system is used to detect sperm cells in a semen sample to obtain intensity, phase, bright field and dark field images of the sperm cells; the computing device is used to input the acquired intensity, phase, bright field and dark field images of the sperm cells into a trained sperm cell morphology classification and DNA fragmentation rate prediction network model, output sperm cell morphology classification and DNA fragmentation rate prediction results, and assess sperm quality based on the sperm cell morphology classification and DNA fragmentation rate prediction results.

[0016] According to one aspect of the present invention, a marker-free sperm quality assessment device is provided, comprising a memory and a processor, wherein the memory stores program instructions executed by the processor, and the processor calls the program instructions to execute the steps of a marker-free sperm quality assessment method.

[0017] According to one aspect of the present invention, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions enable the computer to execute the steps of the marker-free sperm quality assessment method.

[0018] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention can obtain a large amount of sperm cell image data by detecting sperm through a multi-dimensional flow cytometric imaging system, which is conducive to the comprehensive evaluation of sperm cells in semen; by analyzing the sperm cell image database through the established multi-model fusion deep learning algorithm, the sperm cell image features are extracted and classified, which can realize fast and accurate sperm morphology and DNA fragmentation rate recognition, and then evaluate sperm quality.

[0019] 2. The method of the present invention adopts a label-free detection method, which simplifies the operation steps and improves the detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, a brief introduction is given below to the drawings used in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A schematic diagram of a flow chart of a marker-free sperm quality assessment method provided by an embodiment of the present invention; Figure 2 A schematic diagram of the structure of a multi-dimensional flow cytometer imaging system provided by an embodiment of the present invention; Figure 3 A schematic diagram of the principle of a sperm cell morphology classification and DNA fragmentation rate prediction network model provided in an embodiment of the present invention; Figure 4 A schematic diagram of a flow chart of marker-free sperm quality assessment provided by an embodiment of the present invention; Figure 5 A schematic diagram of the structure of a marker-free sperm quality assessment system provided by an embodiment of the present invention; Figure 6 A schematic diagram of the structure of another marker-free sperm quality assessment system provided in an embodiment of the present invention.

[0022] In the figure: 101 broadband femtosecond laser; 102 single-mode optical fiber; 103 beam splitter; 104 polarizer; 105 diffraction grating; 106 objective lens; 107 microfluidic chip; 108 objective lens; 109 diffraction grating; 110 beam splitter; 111 1 / 2 wave plate; 112 reflector; 113 delay module; 114 beam splitter; 115 polarization beam splitter; 116, 117, 118 photodetectors; 119 high-speed oscilloscope; 120 computer. DETAILED DESCRIPTION

[0023] It should be noted that: The terms "including" and "having" and any variations thereof in the specification and claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions, for example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to the steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0024] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices. The flowcharts shown in the accompanying drawings are merely exemplary and do not necessarily include all the contents and operations / steps, nor do they necessarily have to be executed in the order described. For example, some operations / steps may be further decomposed, while some operations / steps may be combined or partially combined, so the actual execution order may change according to the actual situation.

[0025] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In addition, the technical features in the various embodiments or single embodiments provided by the present invention are arbitrarily combined with each other to form a new technical solution. This combination is not restricted by the sequence of steps and / or the structural composition mode, but must be based on the ability of ordinary technicians in this field to achieve. When the combination of technical solutions is contradictory or cannot be achieved, it should be considered that this combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0026] See also Figure 1A multi-dimensional sperm cell detection method provided by an embodiment of the present invention comprises the following steps: first, a multi-dimensional flow cytometric imaging system is used to detect sperm cells in a semen sample to obtain intensity, phase, bright field and dark field images of the sperm cells; then, the intensity, phase, bright field and dark field images of the sperm cells obtained are input into a trained sperm cell morphology classification and DNA fragmentation rate prediction network model, sperm cell morphology classification and DNA fragmentation rate prediction results are output, and sperm quality is evaluated based on the sperm cell morphology classification and DNA fragmentation rate prediction results.

[0027] The embodiment of the present invention collects multi-dimensional sperm information through a label-free multi-dimensional flow cytometric imaging system, and uses the sperm information in combination with a trained sperm cell morphology classification and DNA fragmentation rate prediction network model to achieve rapid and high-accuracy classification of the patient's sperm morphology and DNA fragmentation rate.

[0028] Specifically, an embodiment of the present invention provides a multi-dimensional sperm cell detection method, comprising the following steps: Step 1: Collect unknown semen samples, wash the semen samples with PBS, and prepare sperm cell samples suitable for detection by multidimensional flow cytometric imaging system; Step 2: Pass the washed sperm and PBS solution into the inlet of the microfluidic chip, so that the sperm cells can flow in the detection channel of the microfluidic chip at a high speed and stably; Step 3: The multidimensional flow cytometric imaging system collects sperm cells flowing through the detection channel and acquires intensity, phase, bright field, and dark field images, and transmits the collected images to a computer; Step 4: The obtained sperm cell intensity, phase, bright field and dark field images are sent to the trained sperm cell quality analysis network model based on deep learning to determine the sperm morphology category and DNA fragmentation rate index of the sperm cell, and analyze the sperm quality in the sample.

[0029] In the embodiment of the present invention, step 1 further includes: Step 1.1: Integrate a quantitative phase imaging system as a cell intensity and phase image acquisition part based on the optofluidic time-domain stretch imaging system to obtain the intensity and phase images of sperm cells; Step 1.2: Based on step 1.1, an integrated optical polarization imaging system is used as the cell bright field and dark field image acquisition part to obtain bright field and dark field images of sperm cells.

[0030] The embodiment of the present invention develops a high-throughput, multi-dimensional flow imaging system based on the optofluidic time-domain stretch imaging system, and studies imaging technology that adapts to sperm morphology. Specifically, by introducing quantitative phase detection technology and optical polarization detection technology on the optofluidic time-domain stretch imaging system, the imaging system can simultaneously measure the intensity, phase, bright field and dark field information of the light pulse. The light pulse signal is used to invert information of different dimensions such as the size, structure, dry mass, roughness, etc. of the cell to provide complete cell parameters. The computer extracts and analyzes the intensity, phase, bright field, and dark field images of the collected sperm cells, establishes and trains a high-precision sperm morphology classification and DNA fragmentation rate recognition network model, and realizes high-precision analysis of the patient's sperm quality.

[0031] Specifically, the multi-dimensional flow cytometry imaging system includes a broadband femtosecond laser, a single-mode optical fiber, a diffraction grating, a microscope objective, a microfluidic chip, a photodetector, a digitizer, a computer, and a cell intensity and phase image acquisition part and a cell bright field and dark field image acquisition part; The cell intensity and phase image acquisition part includes a beam splitter and a delay device. A beam splitter is introduced after the dispersion optical fiber to divide the light pulse into a detection light and a reference light to construct a quantitative phase imaging system based on the interference principle. The optical path of the reference light is adjusted by the delay device so that the detection light and the reference light reach the photodetector at the same time to achieve spatial interference. After digital signal processing, the intensity and phase images of the cell are restored from the collected time domain interference signal.

[0032] The cell bright field and dark field image acquisition part includes a linear polarizer, a beam splitter, a 1 / 2 wave plate and a polarization beam splitter. By adding a linear polarizer in front of the diffraction grating, the light pulse passing through the cell is linearly polarized light, and then the linear polarized light passing through the cell passes through the 1 / 2 wave plate and the polarization beam splitter and reaches different photodetectors respectively, so as to construct an optical polarization imaging system. By adjusting the 1 / 2 wave plate and the polarization beam splitter, the bright field and dark field images of the cell can be obtained respectively.

[0033] See also Figure 4 The training of the sperm cell morphology classification and DNA fragmentation rate prediction network model provided by the embodiment of the present invention comprises the following steps: Step S1: establishing a sperm cell image database containing all sperm morphology categories, and distinguishing sperm cells of different morphology categories and different DNA fragmentation rate indices by clustering or manual screening; Step S2: Construct a sperm cell image analysis network framework based on a multi-model fusion deep learning network, extract and analyze the multidimensional features of sperm cell images with known morphological categories and DNA fragmentation rate index, and establish and train a high-precision sperm cell morphology classification and DNA fragmentation rate index recognition network model.

[0034] Furthermore, the training of the sperm cell morphology classification and DNA fragmentation rate prediction network model provided by the embodiment of the present invention also includes: Step S3: Verify the classification results of the model through clinical sperm morphology and DNA fragmentation index detection results, and optimize the deep learning algorithm based on the results.

[0035] In the embodiment of the present invention, step S1 further includes: Step S1.1: Collecting male semen samples, and obtaining a large number of sperm cell images containing different morphological categories and different DNA fragmentation rate indices after detection by a multi-dimensional flow cytometric imaging system; Step S1.2: using an image clustering method to preliminarily classify the acquired sperm cell images of different categories, and manually selecting and correctly classifying the sperm cell images that are misclassified in the image clustering to establish a sperm cell image database; Step S1.3: Combined with clinical sperm chromatin structure analysis, the sperm DNA fragmentation rate index is obtained and verified with the DNA fragmentation rate extracted from multidimensional sperm images.

[0036] In the embodiment of the present invention, step S2 further includes: S2.1: Construct a multi-model fusion neural network including an image feature extraction module and a feature classification module to extract and analyze the multidimensional features of sperm cell intensity, phase, bright field and dark field images.

[0037] S2.2: Introduce multi-scale into the network to improve the network’s receptive field and enhance the contextual connection of the image; for example, use the Yolo-V3 network to extract information such as size, dry mass, texture, refractive index, etc. of sperm cell intensity and phase images.

[0038] S2.3: Introduce an attention neural network into the network. By adding spatial channel attention, the model can quickly locate the sperm cell area and effectively encode and analyze the sperm features; for example, use the ShumleNet-V2 network with grouped convolution to extract the birefringence, structure, contour and other feature information of sperm cells in a lightweight and efficient manner.

[0039] Furthermore, the Concate method is used to stack the features extracted from various images. The VGG-16 module is used to analyze the image data of sperm cells in various dimensions. The global information such as sperm outline and size and the local information such as sperm structure, texture, and birefringence are jointly analyzed through convolution operations to classify sperm types and predict the DNA fragmentation rate index.

[0040] S2.4: The sperm cell morphology classification and DNA fragmentation rate recognition model based on the multi-model fusion network was trained using the established sperm cell image database to evaluate sperm quality.

[0041] Based on the same inventive concept as the above method embodiment, the present invention also provides a marker-free sperm quality assessment system, see Figure 6 , including: a multidimensional flow cytometric imaging system and a computing device; the multidimensional flow cytometric imaging system is used to detect sperm cells in semen samples and obtain intensity, phase, bright field and dark field images of sperm cells; the computing device is used to input the acquired intensity, phase, bright field and dark field images of sperm cells into a trained sperm cell morphology classification and DNA fragmentation rate prediction network model, output sperm cell morphology classification and DNA fragmentation rate prediction results, and evaluate sperm quality based on the sperm cell morphology classification and DNA fragmentation rate prediction results.

[0042] The multidimensional flow cytometric imaging system is used to obtain the intensity, phase, bright field and dark field images of sperm cells at high speed and without labeling. The microfluidic chip is used to make the cells flow through the channel stably and quickly, so that the multidimensional flow cytometric imaging system can stably image. The computer uses a deep learning method to adaptively analyze the massive cell data and multidimensional information obtained based on high-throughput, multidimensional flow cytometric detection. By analyzing the intensity, phase, bright field and dark field images, the morphological classification of sperm cells and the proportion of sperm of various morphologies are obtained, and the sperm DNA fragmentation rate index is obtained at the same time.

[0043] See also Figure 2 The multidimensional flow cytometric imaging system provided in an embodiment of the present invention includes a cell intensity and phase image acquisition part, a cell bright field and dark field image acquisition part, a broadband femtosecond laser and a computer, and the computer is used for processing the sperm quality evaluation part.

[0044] The optofluidic time-domain stretch imaging system uses femtosecond pulses as detection light. First, a section of dispersive optical fiber is used to separate the different wavelength components of the ultrashort pulse in the time domain to achieve frequency-time mapping. Then, a diffraction grating is used to project the different wavelength components of the ultrashort pulse spectrum to different positions in space to achieve frequency-space mapping. Subsequently, a microscope objective is used to focus the spatially dispersed light pulses on the cells in the logistics channel, thereby achieving the mapping of the cell spatial information to the pulse time domain waveform. Finally, a single-pixel photodetector is used to collect signals. As the cells flow in the microfluidic channel, each light pulse detects the information of a cross section of the cell, and the pulses are spliced ​​together at the back end through digital signal processing to obtain a complete two-dimensional image of the flowing cell, which can achieve a detection flux of more than 1,000,000 cells / second and a spatial resolution better than 780nm.

[0045] The cell intensity and phase acquisition part introduces a beam splitter after the dispersion optical fiber to divide the light pulse into detection light and reference light to construct a quantitative phase imaging system based on the interference principle. The optical path of the reference light is adjusted by a delay device so that the detection light and the reference light reach the photodetector at the same time to achieve spatial interference. After digital signal processing, the intensity and phase image of the cell is restored from the collected time domain interference signal.

[0046] The cell bright field and dark field image acquisition part is configured to add a linear polarizer in front of the diffraction grating so that the light pulse passing through the cell is linearly polarized light. The linear polarized light passing through the cell then passes through a 1 / 2 wave plate and a polarization beam splitter and reaches different photodetectors respectively, so as to construct an optical polarization imaging system. By adjusting the 1 / 2 wave plate and the polarization beam splitter, the bright field and dark field images of the cell can be obtained respectively.

[0047] The sperm quality assessment part based on deep learning is constructed to include an image feature extraction module, a feature stacking part and a feature classification module. Figure 3 The feature extraction module is mainly composed of Yolo-V3 and ShumleNet-V2. Yolo-V3 is used to extract the size, dry mass, texture, refractive index and other information of sperm cell intensity and phase images. The ShumleNet-V2 network with grouped convolution extracts the birefringence, structure, contour and other feature information of sperm cells in a lightweight and efficient way. The Concate method is used to stack the features extracted from various images together. The VGG-16 module is used to analyze the image data of sperm cells in various dimensions. The convolution operation is used to jointly analyze the global information such as the contour and size of sperm and the local information such as the structure, texture, and birefringence of sperm, classify the sperm category and predict the DNA fragmentation rate index.

[0048] The broadband femtosecond laser 101 is a Vitara-P laser from Coherent, USA, with a central wavelength / repetition rate / spectral width / pulse width of 800nm / 80MHz / 40nm / 20fs. The single-mode optical fiber 102 with a length of 1.5 kilometers is a single-mode optical fiber of model YOFC-780-1.5 from Changfei. The beam splitter 103 with a beam splitting ratio of 90:10 is a beam splitter BS041 from Thorlabs. The reflective diffraction gratings 105 and 109 with a line density of 1200 lines / mm are GR26-0608 from Thorlabs. The microscope objective lenses 106 and 108 with a magnification of 50X and a numerical aperture of 0.65 are LCPLN-IR 50X from Olympus. The beam splitter 110 with a beam splitting ratio of 50:50 is BS005 from Thorlabs. The photodetector is Newport's 1544-B-50 with a bandwidth of 12.5 GHz. The high-speed oscilloscope 119 with a sampling rate of 40 GSa / s is DSA91304A from Keysight Technologies, USA. The parameters of the microfluidic chip 107 are a channel width of 80 µm and a channel height of 40 µm. The polarization beam splitter is Lubang's PBS-612. The linear polarizer 104 is Lubang's FLP25-NIR-M. The 1 / 2 wave plate 111 is Lubang's AHWP20-SNIR. The beam splitter 114 is Thorlabs BS020. The polarization beam splitter 115 is Thorlabs PBS255. The photodetectors 116 / 117 / 118 are Newport 1544-B. The specific models of the reflector 112 and the delay module 113 are not limited.

[0049] On the basis of the above embodiments, an embodiment of the present invention further provides a label-free sperm quality assessment system, which is used to execute a label-free sperm quality assessment method in the above method embodiment.

[0050] See also Figure 5The system includes: an image acquisition module, which is used to detect sperm cells in semen samples using a multidimensional flow cytometric imaging system to obtain intensity, phase, bright field and dark field images of sperm cells; an image processing module, which is used to input the intensity, phase, bright field and dark field images of sperm cells obtained into a trained sperm cell morphology classification and DNA fragmentation rate prediction network model, output sperm cell morphology classification and DNA fragmentation rate prediction results, and evaluate sperm quality based on the sperm cell morphology classification and DNA fragmentation rate prediction results; wherein the training of the sperm cell morphology classification and DNA fragmentation rate prediction network model includes: constructing a network model including the A sperm cell image database with sperm morphology categories and sperm DNA fragmentation rates is provided, and sperm cells of different morphology categories and different DNA fragmentation rates are distinguished; a sperm cell multidimensional image analysis framework based on a multi-model fusion deep learning network is constructed to extract and analyze the multidimensional features of sperm cell images with known morphology categories and DNA fragmentation rates, and to establish a sperm cell morphology classification and DNA fragmentation rate prediction network model; the sperm cell image database is used to train the established sperm cell morphology classification and DNA fragmentation rate prediction network model, and the trained sperm cell morphology classification and DNA fragmentation rate prediction network model is output.

[0051] The present invention provides a marker-free sperm quality assessment system, which is aimed at the problem that the number of sperm cells observed in the existing sperm morphology and DNA fragmentation rate detection is small and requires complex staining operations. Figure 5 Several modules in it are based on a high-throughput, multi-dimensional flow cytometer imaging system to obtain intensity, phase, bright field and dark field images of sperm cells, establish a massive sperm cell image database, and construct, train and optimize a sperm cell multi-dimensional image analysis model based on a multi-model fusion deep learning network to achieve high-speed, high-precision, label-free sperm quality assessment.

[0052] It should be noted that the system embodiment provided by the present invention is used to implement the method in the above method embodiment as well as the method in other method embodiments provided by the present invention. The only difference is that the corresponding functional modules are set. The principle is basically the same as the principle of the above system embodiment provided by the present invention. As long as the technical personnel in this field refer to the specific technical solutions in other method embodiments on the basis of the above system embodiment, obtain the corresponding technical means and the technical solutions composed of these technical means by combining technical features, and on the premise of ensuring the practicality of the technical solutions, they can improve the equipment in the above system embodiment to obtain the corresponding system embodiment, which is used to implement the methods in other method embodiments. For example: Based on the contents of the above system embodiment, as a preferred embodiment, a marker-free sperm quality assessment system provided in the embodiment of the present invention further includes: The verification module is used to verify the prediction results of the model through clinical sperm morphology and DNA fragmentation index test results, and optimize the deep learning algorithm based on the verification results.

[0053] Based on the contents of the above system embodiment, as a preferred embodiment, a marker-free sperm quality assessment system provided in the embodiment of the present invention further includes: The integrated module is used to integrate a quantitative phase imaging system based on the optofluidic time-domain stretching imaging system to obtain intensity and phase images of sperm cells; and to integrate an optical polarization imaging system based on the previous step to obtain bright field and dark field images of sperm cells.

[0054] Based on the contents of the above system embodiment, as a preferred embodiment, a marker-free sperm quality assessment system provided in the embodiment of the present invention further includes: The database construction module is used to collect semen samples and obtain a number of sperm cell images containing different morphological categories and different DNA fragmentation rate indices after detection by a multidimensional flow cytometer imaging system; the image clustering method is used to preliminarily classify the obtained sperm cell images of different categories, and the sperm cell images that are misclassified in image clustering are selected and correctly classified through manual screening to establish a sperm cell image database.

[0055] Based on the content of the above system embodiment, as a preferred embodiment, in a label-free sperm quality assessment system provided in the embodiment of the present invention, the sperm cell morphology classification and DNA fragmentation rate prediction network model established includes: a feature extraction module, which is used to extract sperm cell intensity and phase image features through the Yolo-V3 network, and to extract sperm cell structural contour features through the ShumleNet-V2 network; a feature stacking module, which is used to stack all the extracted features using the Concate method; a feature classification module, which is used to classify the images of sperm cells in various dimensions using the VGG-16 network to obtain sperm category classification and DNA fragmentation index prediction results.

[0056] Based on the same inventive concept as the above embodiment, an embodiment of the present invention also provides a marker-free sperm quality assessment device, including a memory and a processor, the memory stores program instructions executed by the processor, and the processor calls the program instructions to execute the steps of the marker-free sperm quality assessment method.

[0057] In an embodiment of the present invention, the memory may be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), etc., or a volatile memory (volatile memory), such as a random-access memory (RAM). The memory is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory in an embodiment of the present invention may also be a circuit or any other device that can implement a storage function, for storing program instructions and / or data.

[0058] In the embodiments of the present invention, the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of the present invention may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0059] Based on the same inventive concept as the above embodiment, an embodiment of the present invention further provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the steps of the marker-free sperm quality assessment method.

[0060] In summary, the present invention is based on a high-throughput, multi-dimensional flow cytometric imaging system to obtain the intensity, phase, bright field and dark field images of sperm cells, establish a massive sperm cell image database, and construct, train and optimize a sperm cell multi-dimensional image analysis model based on a multi-model fusion deep learning network. On this basis, the trained sperm cell multi-dimensional image analysis network model is used to analyze the sperm cell images obtained by the multi-dimensional flow cytometric imaging system to detect semen samples, analyze the proportion of each morphological category of sperm in the semen sample, sperm DNA fragmentation rate and other sperm structure and function parameters, and achieve high-speed, high-precision, label-free sperm quality assessment.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A marker-free sperm quality assessment method, characterized in that: include: The sperm cells in the semen samples were detected using a multi-dimensional flow cytometric imaging system to obtain the intensity, phase, bright field and dark field images of the sperm cells; Inputting the intensity, phase, bright field and dark field images of the acquired sperm cells into the trained sperm cell morphology classification and DNA fragmentation rate prediction network model, outputting sperm cell morphology classification and DNA fragmentation rate prediction results, and evaluating sperm quality according to the sperm cell morphology classification and DNA fragmentation rate prediction results; The training of the sperm cell morphology classification and DNA fragmentation rate prediction network model includes: Construct a sperm cell image database containing all sperm morphology categories and sperm DNA fragmentation rates, and distinguish sperm cells of different morphology categories and different DNA fragmentation rates; Construct a sperm cell multidimensional image analysis framework based on a multi-model fusion deep learning network to extract and analyze the multidimensional features of sperm cell images with known morphological categories and DNA fragmentation rates, and establish a sperm cell morphology classification and DNA fragmentation rate prediction network model; The sperm cell image database is used to train the established sperm cell morphology classification and DNA fragmentation rate prediction network model, and the trained sperm cell morphology classification and DNA fragmentation rate prediction network model is output.

2. A marker-free sperm quality assessment method according to claim 1, characterized in that: The training of the sperm cell morphology classification and DNA fragmentation rate prediction network model also includes: verifying the prediction results of the model through clinical sperm morphology and DNA fragmentation rate index detection results, and optimizing the deep learning algorithm based on the verification results.

3. A label-free sperm quality assessment method according to claim 1, characterized in that: Detect sperm cells in semen samples using a multi-dimensional flow cytometric imaging system, including: Based on the optofluidic time-domain stretch imaging system, a quantitative phase imaging system is integrated to obtain the intensity and phase images of sperm cells; Based on the previous step, an optical polarization imaging system was integrated to obtain bright-field and dark-field images of sperm cells.

4. A label-free sperm quality assessment method according to claim 1, characterized in that: Construct a sperm cell image database containing all sperm morphology categories and sperm DNA fragmentation rates, and distinguish sperm cells of different morphology categories and different DNA fragmentation rates, including: Collect semen samples, and obtain a number of sperm cell images containing different morphological categories and different DNA fragmentation rate indices after detection by a multi-dimensional flow cytometric imaging system; The image clustering method is used to preliminarily classify the acquired sperm cell images of different categories, and the sperm cell images that are misclassified in image clustering are selected and correctly classified through manual screening to establish a sperm cell image database.

5. A marker-free sperm quality assessment method according to claim 1, characterized in that: The established sperm cell morphology classification and DNA fragmentation rate prediction network model includes: a feature extraction module, which is used to extract the intensity and phase image features of sperm cells through the Yolo-V3 network, and to extract the structural contour features of sperm cells through the ShumleNet-V2 network; a feature stacking module, which is used to stack all the extracted features using the Concate method; a feature classification module, which is used to classify the images of sperm cells in various dimensions using the VGG-16 network to obtain the sperm category classification and DNA fragmentation index prediction results.

6. A marker-free sperm quality assessment system, characterized in that: include: An image acquisition module is used to detect sperm cells in a semen sample using a multi-dimensional flow cytometric imaging system to obtain intensity, phase, bright field and dark field images of the sperm cells; An image processing module is used to input the intensity, phase, bright field and dark field images of the acquired sperm cells into the trained sperm cell morphology classification and DNA fragmentation rate prediction network model, output sperm cell morphology classification and DNA fragmentation rate prediction results, and evaluate sperm quality according to the sperm cell morphology classification and DNA fragmentation rate prediction results; The training of the sperm cell morphology classification and DNA fragmentation rate prediction network model includes: Construct a sperm cell image database containing all sperm morphology categories and sperm DNA fragmentation rates, and distinguish sperm cells of different morphology categories and different DNA fragmentation rates; Construct a sperm cell multidimensional image analysis framework based on a multi-model fusion deep learning network to extract and analyze the multidimensional features of sperm cell images with known morphological categories and DNA fragmentation rates, and establish a sperm cell morphology classification and DNA fragmentation rate prediction network model; The sperm cell image database is used to train the established sperm cell morphology classification and DNA fragmentation rate prediction network model, and the trained sperm cell morphology classification and DNA fragmentation rate prediction network model is output.

7. A marker-free sperm quality assessment system, characterized in that: include: Multidimensional flow cytometry imaging systems and computing devices; The multidimensional flow cytometric imaging system is used to detect sperm cells in semen samples and obtain intensity, phase, bright field and dark field images of sperm cells; the computing device is used to input the acquired intensity, phase, bright field and dark field images of sperm cells into the trained sperm cell morphology classification and DNA fragmentation rate prediction network model, output sperm cell morphology classification and DNA fragmentation rate prediction results, and evaluate sperm quality based on the sperm cell morphology classification and DNA fragmentation rate prediction results.

8. A marker-free sperm quality assessment device, characterized in that: The invention comprises a memory and a processor, wherein the memory stores program instructions executed by the processor, and the processor calls the program instructions to execute the steps of a marker-free sperm quality assessment method according to any one of claims 1 to 5.

9. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the steps of a marker-free sperm quality assessment method as claimed in any one of claims 1 to 5.

Images