A method, system and device for label-free sperm quality assessment

Abstract

The application discloses a label-free sperm quality evaluation method, system and device. The application is based on a high-throughput multi-dimensional flow cytometry imaging system, obtains intensity, phase, bright field and dark field images of sperm cells, establishes a massive sperm cell image database, and constructs, trains and optimizes a sperm cell multi-dimensional image analysis model based on a multi-model fusion deep learning network. On this basis, the trained sperm cell multi-dimensional image analysis network model is used to analyze sperm cell images obtained by the multi-dimensional flow cytometry imaging system in the detection of a semen sample, analyze sperm structure and function parameters such as the proportion of each morphological category of sperm in the semen sample and the sperm DNA fragmentation rate, and realize high-speed, high-precision and label-free sperm quality evaluation.

Description

TECHNICAL FIELD

[0001] The present application relates to a label-free sperm quality evaluation method, system and device, in particular to a sperm quality detection method based on an optical flow time domain stretching microscopic imaging system, quantitative phase imaging, optical polarization imaging and a multi-model fusion algorithm. BACKGROUND

[0002] Clinically, it is generally believed that sperm quantity reduction and sperm quality reduction are the main signs of male infertility. Semen examination is an important means to evaluate the male fertility, among which the routine semen examination is the most common in the clinic, mainly used to evaluate the sperm quantity, sperm density, sperm motility, liquefaction time and sperm morphology in the semen. Among them, the sperm morphology analysis as an indispensable part of the semen examination can qualitatively analyze the structural characteristics of the sperm, and the analysis results can reflect the sperm deformity rate and the fertilization ability, thereby providing a basis for the clinical diagnosis and treatment. Similarly, the DNA fragmentation rate reflects the integrity and stability of the sperm DNA, and a high level of DNA fragmentation rate usually means that the genome of the sperm has a higher damage rate. The normal sperm morphology rate and the DNA fragmentation rate are positively correlated with the success rate of assisted reproductive technology, and selecting sperm with normal morphology and DNA fragmentation rate for assisted reproductive technology can improve the in vitro fertilization rate. However, the sperm morphology is diverse and the classification standard is complex, especially the abnormal morphological sperm, which is limited by the existing detection technology, and it brings great challenges to realize high-precision sperm morphology analysis under the premise of maintaining cell viability. At the same time, the DNA fragmentation rate detection also has similar problems as the morphology detection. Therefore, once the high-precision label-free detection is realized, it will not only greatly help the evaluation of sperm quality, but also provide important information reference for sperm formation, assisted reproductive technology, biological evolution and other aspects.

[0003] The most traditional and widely used method for sperm morphology detection in clinic is smear microscopy, which stains the sperm on the smear to observe the sperm morphology under an optical microscope. In the detection, the doctor needs to evaluate and analyze the head, neck, middle segment and tail of each sperm to confirm whether it is a normal morphology sperm, which puts high requirements on the professional knowledge, professional level and work experience of the doctor. It is easy to give different results due to subjective differences in the understanding of the judgment standard of the doctor, and the workload is huge, time-consuming and labor-intensive, which brings great burden to the work of the doctor. In order to solve the problems of strong subjectivity and labor-intensive of smear microscopy, and improve the automation degree of sperm morphology analysis, computer-aided sperm morphology measurement (CASMA) is more and more used in clinic. The current CASMA usually analyzes the images or videos of sperm cells obtained by combining optical microscopes and CCD or CMOS, and then extracts sperm cells through image enhancement, threshold segmentation, edge detection and other technologies, and segments the head, neck and tail of the sperm. However, due to the differences in methodology, such as sample staining method, focusing and illumination of optical microscope, and the difficulty in correctly distinguishing the sperm head from the cell fragments, dye particles and other cells in the semen, the accuracy and repeatability of the CASMA analysis results are affected.

[0004] Currently, several commonly used methods for sperm DNA fragmentation rate each have different technical characteristics and application scenarios. Sperm chromatin structure assay (SCSA) is the most widely used method, which detects the stability of chromatin by flow cytometry to evaluate DNA integrity, and the result is expressed as DNA fragmentation index (DFI). The advantage of SCSA is high throughput and relatively high accuracy. TUNEL method directly detects DNA damage by labeling DNA break ends, which can be analyzed using fluorescence microscope or flow cytometry, and is suitable for detection requiring high sensitivity. Comet assay (single cell gel electrophoresis) detects the tail shape formed by damaged DNA dragging in an electric field to identify DNA damage, which is suitable for situations requiring in-depth analysis of individual cells. Another method is SCD method, i.e. Halosperm test, which detects the halo formed by removing the sperm DNA of nucleosome to judge the integrity of DNA, and this method is known for its simplicity and speed. However, these methods inevitably require complex staining operations and damage the original sperm, which affects the subsequent selection of suitable sperm for assisted reproduction.

[0005] A sperm head detection method and system are provided in Chinese patent application (application number: CN202111130886.8) published on January 11, 2022. The invention uses a convolutional neural network to automatically detect sperm heads, improving the efficiency and accuracy of sperm detection and effectively assisting doctors in sperm morphology analysis. A method for automatic semen analysis based on multi-frame information fusion with occlusion perception is provided in Chinese patent application (application number: CN202210673228.1) published on June 15, 2022. The invention combines an edge-sensitive U-Net model with an occlusion perception tracker based on joint probability data association, achieving accurate and stable sperm detection and tracking, and can be easily integrated into existing computer-aided sperm analysis systems to provide more accurate and robust semen analysis data. Chinese patent application (application number: CN202110789070.X) published on October 8, 2021 discloses the application and staining method of Shorr staining reagent. The kit uses fresh testicular biopsy tissue frozen sections as the staining object, can dye different parts of sperm into different colors and effectively distinguish sperm from background cells, improving the recognition of sperm and shortening the diagnosis time of sperm pathological morphology. Chinese patent application (application number: CN202111677213.4) published on April 8, 2022 proposes a human sperm morphology staining reagent and staining method. The human sperm staining solution SVS includes: 0.1-1M sodium chloride solution, 20-100mM eosin solution, 100-200g / L aniline black solution, 0.01-1% formaldehyde solution, 1M sodium hydroxide solution, and hydrochloric acid solution to adjust the pH value to 7.3-9.5. The invention is a one-step method that is simple and efficient, and only one bottle of reagent is needed to meet all staining needs. The staining can be completed in 30-60 seconds, and the sperm morphology can be observed under a microscope after air drying. Chinese patent application (application number: CN202210275918.1) published on May 24, 2022 discloses a sperm analysis slide and detection system. The system can observe multiple samples at a time, improving detection efficiency; the distance between the upper surface of the sample groove and the depth of the electrostatic membrane is not more than 8 millimeters, facilitating single-layer diffusion imaging of semen, facilitating direct observation of the morphology of living sperm, and improving detection accuracy.

[0006] A sperm DNA fragmentation detection kit and detection method are disclosed in Chinese patent (application number: CN202111662563.3) published on December 30, 2021. The kit uses carbon quantum dots as a Pi detection reagent, and based on a flow cytometer, it cooperates with a sperm chromatin structure detection method based on acridine orange dye to achieve the detection of sperm motility and DNA fragmentation rate of motile sperm. Chinese patent application (application number: CN202211019378.7) published on November 15, 2022 discloses a DNA fragmentation analysis method based on deep learning and related equipment. The DNA fragmentation analysis method based on deep learning acquires target images and predicts the sperm DNA fragmentation category of each pixel point in the images. The ratio of the halo size data of the halo non-fragment to the approximate short diameter of the sperm is used to determine the suspected fragments in the halo non-fragment. Finally, the non-halo fragments and the suspected fragments are used to estimate the sperm DNA fragmentation rate.

[0007] The above patents use machine learning to analyze sperm microscopic images or develop new sperm staining methods, but there are still problems such as a small number of evaluated sperm and the need for complex staining operations. Therefore, comprehensive and label-free sperm cell detection is urgently needed to accurately evaluate sperm morphology and DNA fragmentation rate. SUMMARY

[0008] To solve the problem of a small number of observed sperm cells and the need for complex staining operations in the detection of sperm morphology and DNA fragmentation rate, the present application provides a label-free sperm quality evaluation method, system and device. The trained sperm cell multi-dimensional image analysis network model is used to analyze the sperm cell images obtained by the multi-dimensional flow cytometry imaging system for detecting semen samples, analyze the proportion of each morphological category of sperm in the semen sample, sperm DNA fragmentation rate and other sperm structure and function parameters, and realize high-speed, high-precision and label-free sperm quality evaluation.

[0009] According to an aspect of the present application, a label-free sperm quality evaluation method is provided, comprising:

[0010] The sperm cells in the semen sample are detected by a multi-dimensional flow cytometry imaging system to obtain intensity, phase, bright field and dark field images of the sperm cells;

[0011] The obtained intensity, phase, bright field and dark field images of the sperm cells are input into the trained sperm cell morphology classification and DNA fragmentation rate prediction network model, and the sperm cell morphology classification and DNA fragmentation rate prediction results are output. The sperm quality is evaluated according to the sperm cell morphology classification and DNA fragmentation rate prediction results;

[0012] The training of the sperm cell morphology classification and DNA fragmentation rate prediction network model includes:

[0013] constructing a sperm cell image database containing all sperm morphology categories and sperm DNA fragmentation rates, and distinguishing sperm cells of different morphology categories and different DNA fragmentation rates;

[0014] constructing a sperm cell multi-dimensional image analysis framework based on a multi-model fusion deep learning network, for extracting and analyzing multi-dimensional features of sperm cell images of known morphology categories and DNA fragmentation rates, and establishing sperm cell morphology classification and DNA fragmentation rate prediction network models;

[0015] training the established sperm cell morphology classification and DNA fragmentation rate prediction network models using the sperm cell image database, and outputting the trained sperm cell morphology classification and DNA fragmentation rate prediction network models.

[0016] As a further technical solution, the training of the sperm cell morphology classification and DNA fragmentation rate prediction network model further comprises: verifying the prediction results of the model by clinical sperm morphology and DNA fragmentation rate index detection results, and optimizing the deep learning algorithm according to the verification results.

[0017] As a further technical solution, the sperm cells in the semen sample are detected using a multi-dimensional flow cytometry imaging system, comprising:

[0018] On the basis of the optical flow time domain stretching imaging system, a quantitative phase imaging system is integrated to obtain intensity and phase images of sperm cells;

[0019] On the basis of the previous step, an optical polarization imaging system is integrated to obtain bright field and dark field images of sperm cells.

[0020] As a further technical solution, a sperm cell image database containing all sperm morphology categories and sperm DNA fragmentation rates is constructed, and sperm cells of different morphology categories and different DNA fragmentation rates are distinguished, comprising:

[0021] After collecting the semen sample, a number of sperm cell images containing different morphology categories and different DNA fragmentation rate indexes are obtained by detecting the semen sample using a multi-dimensional flow cytometry imaging system;

[0022] Using an image clustering method, the obtained sperm cell images of different categories are initially classified, and the image clustering of the sperm cell images that are incorrectly classified is selected and correctly classified by manual screening, so as to establish a sperm cell image database.

[0023] As a further technical solution, the sperm cell morphology classification and DNA fragmentation rate prediction network model established includes: a feature extraction module, which is used to extract sperm cell intensity and phase image features through a Yolo-V3 network and extract sperm cell structure contour features through a ShumleNet-V2 network; a feature stacking module, which is used to stack all the extracted features by using a Concate method; and a feature classification module, which is used to classify each dimension image of the sperm cell by using a VGG-16 network to obtain sperm category classification and DNA fragmentation index prediction results.

[0024] According to an aspect of the present application, a label-free sperm quality evaluation system is provided, comprising:

[0025] An image acquisition module is configured to detect sperm cells in a semen sample by using a multi-dimensional flow cytometry imaging system to obtain intensity, phase, bright field and dark field images of the sperm cells.

[0026] An image processing module is configured to input the acquired intensity, phase, bright field and dark field images of the sperm cells into a trained sperm cell morphology classification and DNA fragmentation rate prediction network model, output sperm cell morphology classification and DNA fragmentation rate prediction results, and evaluate sperm quality according to the sperm cell morphology classification and DNA fragmentation rate prediction results.

[0027] The training of the sperm cell morphology classification and DNA fragmentation rate prediction network model includes:

[0028] A sperm cell image database containing all sperm morphology categories and sperm DNA fragmentation rates is constructed, and sperm cells of different morphology categories and different DNA fragmentation rates are distinguished.

[0029] A sperm cell multi-dimensional image analysis framework based on a multi-model fusion deep learning network is constructed, which is used to extract and analyze multi-dimensional features of sperm cell images of known morphology categories and DNA fragmentation rates, and establish a sperm cell morphology classification and DNA fragmentation rate prediction network model.

[0030] The sperm cell morphology classification and DNA fragmentation rate prediction network model established is trained by using the sperm cell image database, and a trained sperm cell morphology classification and DNA fragmentation rate prediction network model is output.

[0031] According to an aspect of the present application, a label-free sperm quality evaluation system is provided, comprising: a multi-dimensional flow cytometry imaging system and a computing device; the multi-dimensional flow cytometry imaging system is used to detect sperm cells in a semen sample, and obtain intensity, phase, bright field and dark field images of the sperm cells; the computing device is used to input the obtained intensity, phase, bright field and dark field images of the sperm cells into a trained sperm cell morphology classification and DNA fragmentation rate prediction network model, output sperm cell morphology classification and DNA fragmentation rate prediction results, and evaluate sperm quality according to the sperm cell morphology classification and DNA fragmentation rate prediction results.

[0032] According to an aspect of the present application, a label-free sperm quality evaluation device is provided, comprising a memory and a processor, the memory stores program instructions executed by the processor, and the processor calls the program instructions to execute the steps of the label-free sperm quality evaluation method.

[0033] According to an aspect of the present application, a non-transitory computer readable storage medium is provided, which stores computer instructions, and the computer instructions make the computer execute the steps of the label-free sperm quality evaluation method.

[0034] Compared with the prior art, the beneficial effects of the present application are:

[0035] 1、The present application can obtain a large amount of sperm cell image data by detecting sperm through a multi-dimensional flow cytometry imaging system, which is conducive to realizing comprehensive evaluation of sperm cells in semen; by analyzing a sperm cell image database through a multi-model fusion deep learning algorithm established by the present application, the sperm cell image features are extracted and classified, rapid and accurate sperm morphology and DNA fragmentation rate identification can be realized, and then sperm quality can be evaluated.

[0036] 2、The method of the present application adopts a label-free detection method, simplifies the operation steps, and improves the detection efficiency. DETAILED DESCRIPTION

[0037] To make the technical solutions of the embodiments of the present application or the prior art clearer, the drawings used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0038] Figure 1 A flowchart of a label-free sperm quality evaluation method provided by an embodiment of the present application is shown in the figure;

[0039] Figure 2A structural schematic diagram of a multi-dimensional flow cytometry imaging system provided by an embodiment of the present application is shown in the figure;

[0040] Figure 3 A principle schematic diagram of a sperm cell morphology classification and DNA fragmentation rate prediction network model provided by an embodiment of the present application is shown in the figure;

[0041] Figure 4 A label-free sperm quality evaluation flowchart provided by an embodiment of the present application is shown in the figure;

[0042] Figure 5 A structural schematic diagram of a label-free sperm quality evaluation system provided by an embodiment of the present application is shown in the figure;

[0043] Figure 6 A structural schematic diagram of another label-free sperm quality evaluation system provided by an embodiment of the present application is shown in the figure.

[0044] In the figure: 101, a broadband femtosecond laser; 102, a single-mode optical fiber; 103, a beam splitter; 104, a polarizer; 105, a diffraction grating; 106, an objective lens; 107, a microfluidic chip; 108, an objective lens; 109, a diffraction grating; 110, a beam splitter; 111, a 1 / 2 wave plate; 112, a mirror; 113, a delay module; 114, a beam splitter; 115, a polarization beam splitter; 116, 117, and 118, photodetectors; 119, a high-speed oscilloscope; and 120, a computer. DETAILED DESCRIPTION

[0045] It should be noted that:

[0046] The terms "include" and "have" and any variations thereof in the specification and claims of the present application and the above-described figures are intended to cover the non-exclusive inclusion, for example, a process, method, system, product, or device including a series of steps or units does not have to be limited to the steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product, or device.

[0047] The block diagrams shown in the figures are only functional entities, which do not necessarily have to correspond to physically independent entities. That is, these functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices. The flowcharts shown in the figures are only exemplary illustrations, which do not necessarily include all contents and operations / steps, and do not necessarily have to be executed in the order described. For example, some operations / steps can be further divided, and some operations / steps can be combined or partially combined, so that the actual execution order can be changed according to the actual situation.

[0048] For the purposes of the embodiments of the present application, the technical solutions and advantages, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts are within the scope of protection of the present application. In addition, the technical features in each embodiment or in a single embodiment provided by the present application can be combined with each other at will to form new technical solutions, and such combination is not subject to the order of steps and / or structure composition mode, but must be based on the implementation by those of ordinary skill in the art. When the combination of technical solutions contradicts each other or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection required by the present application.

[0049] Please refer to Figure 1 The multi-dimensional sperm cell detection method provided by the embodiments of the present application first detects sperm cells in a semen sample by using a multi-dimensional flow cytometry imaging system to obtain intensity, phase, bright field and dark field images of the sperm cells; then inputs the obtained intensity, phase, bright field and dark field images of the sperm cells into a trained sperm cell morphology classification and DNA fragmentation rate prediction network model, outputs sperm cell morphology classification and DNA fragmentation rate prediction results, and evaluates sperm quality according to the sperm cell morphology classification and DNA fragmentation rate prediction results.

[0050] The embodiments of the present application collect multi-dimensional sperm information by using a label-free multi-dimensional flow cytometry imaging system, and realize rapid and high-accuracy classification of sperm morphology and DNA fragmentation rate by using the sperm information in combination with a trained sperm cell morphology classification and DNA fragmentation rate prediction network model.

[0051] Specifically, the multi-dimensional sperm cell detection method provided by the embodiments of the present application includes the following steps:

[0052] Step 1: Collect an unknown semen sample, wash the semen sample with PBS, and prepare a sperm cell sample suitable for detection by a multi-dimensional flow cytometry imaging system;

[0053] Step 2: Pass the washed sperm and PBS solution into the inlet of a microfluidic chip, so that the sperm cells can flow at high speed and stability in the detection channel of the microfluidic chip;

[0054] Step 3: The multi-dimensional flow cytometry imaging system collects sperm cells flowing through the detection channel, and obtains intensity, phase, bright field and dark field images, and transmits the collected images to a computer;

[0055] Step 4: the obtained sperm cell intensity, phase, bright field and dark field images are input into the trained deep learning-based sperm cell quality analysis network model to determine the sperm morphological category and DNA fragmentation rate index to which the sperm cell belongs, and the sperm quality in the sample is analyzed.

[0056] In the embodiment of the application, step 1 further comprises:

[0057] Step 1.1: integrating a quantitative phase imaging system as a cell intensity and phase image acquisition part on the basis of the optical flow time domain stretching imaging system can obtain intensity and phase images of sperm cells.

[0058] Step 1.2: on the basis of step 1.1, integrating an optical polarization imaging system as a cell bright field and dark field image acquisition part can obtain bright field and dark field images of sperm cells.

[0059] The embodiment of the application develops a high-throughput, multi-dimensional flow imaging system on the basis of the optical flow time domain stretching imaging system and studies an imaging technology suitable for sperm morphology. Specifically, by introducing quantitative phase detection technology and optical polarization detection technology on the optical flow time domain stretching imaging system, the imaging system can simultaneously measure the intensity, phase, bright field and dark field information of the light pulse. The size, structure, dry mass and roughness of the cell are reversed by using the light pulse signal to provide complete cell parameters. The computer extracts and analyzes the intensity, phase, bright field and dark field images of the collected sperm cells, establishes and trains a high-precision sperm morphological classification and DNA fragmentation rate recognition network model, and realizes high-precision analysis of the sperm quality of the patient.

[0060] Specifically, the multi-dimensional flow cytometry imaging system comprises a wideband femtosecond laser, a single-mode optical fiber, a diffraction grating, a microscope objective, a microfluidic chip, a photodetector, a digitizer, a computer, a cell intensity and phase image acquisition part, and a cell bright field and dark field image acquisition part.

[0061] The cell intensity and phase image acquisition part comprises a beam splitter and a retarder. The beam splitter is introduced after the dispersive optical fiber to divide the light pulse into probe light and reference light to construct a quantitative phase imaging system based on the interference principle. The optical path of the reference light is adjusted by the delay device so that the probe light and the reference light reach the photodetector at the same time to realize spatial interference. After digital signal processing, the intensity and phase images of the cell are recovered from the collected time domain interference signals.

[0062] The cell bright field and dark field image acquisition part comprises a linear polarizer, a beam splitter, a 1 / 2 wave plate and a polarization beam splitter. By adding a linear polarizer in front of a diffraction grating, the light pulse passing through the cell is linearly polarized light, and then the linearly polarized light passing through the cell reaches different photodetectors through the 1 / 2 wave plate and the polarization beam splitter to construct an optical polarization imaging system. By adjusting the 1 / 2 wave plate and the polarization beam splitter, the bright field and dark field images of the cell can be obtained respectively.

[0063] Please refer to Figure 4 The training of the sperm cell morphology classification and DNA fragmentation rate prediction network model provided by the embodiment of the application comprises the following steps:

[0064] Step S1: a sperm cell image database containing all sperm morphology categories is established, and different morphology categories and different DNA fragmentation rate indexes of sperm cells are distinguished by clustering or manual screening;

[0065] Step S2: a sperm cell image analysis network framework based on a multi-model fusion deep learning network is constructed, multi-dimensional features of sperm cell images with known morphology categories and DNA fragmentation rate indexes are extracted and analyzed, and a high-precision sperm cell morphology classification and DNA fragmentation rate index recognition network model is established and trained.

[0066] Further, the training of the sperm cell morphology classification and DNA fragmentation rate prediction network model provided by the embodiment of the application further comprises:

[0067] Step S3: the classification results of the model are verified by the clinical sperm morphology and DNA fragmentation rate index detection results, and the deep learning algorithm is optimized according to the results.

[0068] In the embodiment of the application, step S1 further comprises:

[0069] Step S1.1: collecting a semen sample of a male, obtaining a large number of sperm cell images containing different morphology categories and different DNA fragmentation rate indexes through a multi-dimensional flow cytometry imaging system;

[0070] Step S1.2: using an image clustering method, the obtained sperm cell images of different categories are initially classified, and the sperm cell images with incorrect classification in the image clustering are selected out and correctly classified by manual screening to establish a sperm cell image database;

[0071] Step S1.3: combining clinical sperm chromatin structure analysis, obtaining a sperm DNA fragmentation rate index, and verifying the DNA fragmentation rate extracted from the multi-dimensional sperm image.

[0072] In the embodiment of the application, step S2 further comprises:

[0073] S2.1: Construct a multi-model fusion neural network comprising an image feature extraction module and a feature classification module for extracting and analyzing multi-dimensional features of sperm cell intensity, phase, bright field and dark field images.

[0074] S2.2: Introduce multi-scale in the network to improve the receptive field of the network and enhance the contextual connection of the image; for example, use Yolo-V3 network to extract information such as size, dry mass, texture, refractive index of sperm cell intensity and phase images.

[0075] S2.3: Introduce attention neural network in the network, by adding spatial channel attention, the model can quickly locate to the sperm cell area and effectively encode and analyze the sperm features; for example, use ShumleNet-V2 network with grouped convolution to extract birefringence, structure, contour and other feature information of sperm cells in a light and efficient manner.

[0076] Further, the Concate method is used to stack the features extracted from various images together. VGG-16 module is used to analyze the multi-dimensional image data of sperm cells, and through convolution operation, the global information such as the size and contour of sperm and the local information such as the structure, texture and birefringence of sperm are jointly analyzed to classify the sperm categories and predict the DNA fragmentation rate index.

[0077] S2.4: Use the established sperm cell image database to train the sperm cell morphology classification and DNA fragmentation rate identification model based on the multi-model fusion network, and evaluate the sperm quality.

[0078] Based on the same inventive concept as the above method embodiments, the embodiments of the present application also provide a label-free sperm quality evaluation system, please refer to Figure 6 , comprising: a multi-dimensional flow cytometry imaging system and a computing device; the multi-dimensional flow cytometry imaging system is used for detecting sperm cells in a semen sample to obtain intensity, phase, bright field and dark field images of the sperm cells; the computing device is used for inputting the obtained intensity, phase, bright field and dark field images of the sperm cells into the trained sperm cell morphology classification and DNA fragmentation rate prediction network model, outputting sperm cell morphology classification and DNA fragmentation rate prediction results, and evaluating sperm quality according to the sperm cell morphology classification and DNA fragmentation rate prediction results.

[0079] The multi-dimensional flow cytometry imaging system is used to obtain intensity, phase, bright field and dark field images of sperm cells under the premise of high speed and no labeling.

[0080] Referring to Figure 2 The multi-dimensional flow cytometry imaging system provided by the embodiment of the present application comprises a cell intensity and phase image acquisition part, a cell bright field and dark field image acquisition part, a broadband femtosecond laser and a computer, and the computer is used for processing of a sperm quality evaluation part.

[0081] The optical flow time domain stretching imaging system uses femtosecond pulses as probe light, first separates different wavelength components of the ultrashort pulse in the time domain by using a dispersive optical fiber to realize frequency-time mapping, then projects different wavelength components of the ultrashort pulse spectrum to different positions in space by using a diffraction grating to realize frequency-space mapping, then focuses the spatially dispersed light pulse on the cell in the microflow channel by using a microscope objective, thereby realizing the mapping of the spatial information of the cell to the pulse time domain waveform. Finally, a single-pixel photodetector is used for signal acquisition. As the cell flows in the microflow channel, each light pulse detects the information of a cross section of the cell, and the pulses are spliced together through digital signal processing at the back end to obtain a complete two-dimensional image of the flowing cell, and a detection flux of more than 1,000,000 cells / second and a spatial resolution better than 780 nm can be realized.

[0082] The cell intensity and phase acquisition part separates the light pulse into probe light and reference light by introducing a beam splitter after the dispersive optical fiber to construct a quantitative phase imaging system based on the interference principle, adjusts the optical path of the reference light by using a delay timer so that the probe light and the reference light reach the photodetector at the same time to realize spatial interference, and recovers the intensity and phase images of the cell from the collected time domain interference signal through digital signal processing.

[0083] The cell bright field and dark field image acquisition part adds a linear polarizer in front of the diffraction grating so that the light pulse passing through the cell is linearly polarized light, and then the linearly polarized light passing through the cell reaches different photodetectors after passing through a 1 / 2 wave plate and a polarization beam splitter to construct an optical polarization imaging system. By adjusting the 1 / 2 wave plate and the polarization beam splitter, the bright field and dark field images of the cell can be obtained respectively.

[0084] The deep learning-based sperm quality evaluation part is constructed to include an image feature extraction module, a feature stacking part and a feature classification module. Figure 3 The feature extraction module is mainly composed of Yolo-V3 and ShumleNet-V2, Yolo-V3 is used to extract the information of sperm cell intensity and phase image size, dry mass, texture, refractive index, etc., and ShumleNet-V2 network with grouping convolution is used to extract birefringence, structure, contour and other feature information of sperm cells in a light and efficient manner. The Concate method is used to stack the features extracted from various images together. VGG-16 module is used to analyze the multi-dimensional image data of sperm cells, and through convolution operation, the global information such as the contour and size of sperm and the local information such as the structure, texture and birefringence of sperm are jointly analyzed to classify the sperm and predict the DNA fragmentation rate index.

[0085] The broadband femtosecond laser 101 is Vitara-P laser of American Coherent Company, and the center wavelength / repetition rate / spectrum width / pulse width are 800nm / 80MHz / 40nm / 20fs respectively. The length of the 1.5km single-mode optical fiber 102 is YOFC-780-1.5 single-mode optical fiber of Changfei Company. The beam splitter ratio 90:10 beam splitter 103 is selected as the beam splitter BS041 of Thorlabs Company. The reflective diffraction grating 105 and 109 with a ruling density of 1200 lines / mm are selected as GR26-0608 of Thorlabs Company. The microscope objective 106 and 108 with a magnification of 50X and a numerical aperture of 0.65 are selected as LCPLN-IR 50X of Olympus Company. The beam splitter 110 with a beam splitter ratio of 50:50 is selected as BS005 of Thorlabs Company. The photodetector is selected as 1544-B-50 of Newport Company with a bandwidth of 12.5GHz. The high-speed oscilloscope 119 with a sampling rate of 40GSa / s is selected as DSA91304A of American InstruTech. The microfluidic chip 107 has a channel width of 80µm and a channel height of 40µm. The polarization beam splitter is PBS-612 of Lubo Company. The linear polarizer 104 is FLP25-NIR-M of Lubo Company. The 1 / 2 wave plate 111 is AHWP20-SNIR of Lubo Company. The beam splitter 114 is Thorlabs BS020. The polarization beam splitter 115 is Thorlabs PBS255. The photodetectors 116 / 117 / 118 are Newport 1544-B. The specific models of the mirror 112 and the delay module 113 are not limited.

[0086] On the basis of the above-mentioned embodiments, the embodiments of the present application further provide a label-free sperm quality evaluation system, which is used to execute one of the label-free sperm quality evaluation methods in the above-mentioned method embodiments.

[0087] Please refer to Figure 5 The system comprises: an image acquisition module, which is used to detect sperm cells in a semen sample by using a multi-dimensional flow cytometry imaging system, so as to obtain intensity, phase, bright field and dark field images of the sperm cells; an image processing module, which is used to input the acquired intensity, phase, bright field and dark field images of the sperm cells into a trained sperm cell morphology classification and DNA fragmentation rate prediction network model, output sperm cell morphology classification and DNA fragmentation rate prediction results, and evaluate sperm quality according to the sperm cell morphology classification and DNA fragmentation rate prediction results; wherein the training of the sperm cell morphology classification and DNA fragmentation rate prediction network model comprises: constructing a sperm cell image database containing all sperm morphology categories and sperm DNA fragmentation rates, and distinguishing sperm cells of different morphology categories and different DNA fragmentation rates; constructing a sperm cell multi-dimensional image analysis framework based on a multi-model fusion deep learning network, which is used to extract and analyze multi-dimensional features of sperm cell images of known morphology categories and DNA fragmentation rates, and establish a sperm cell morphology classification and DNA fragmentation rate prediction network model; and training the established sperm cell morphology classification and DNA fragmentation rate prediction network model by using the sperm cell image database, and outputting a trained sperm cell morphology classification and DNA fragmentation rate prediction network model.

[0088] The label-free sperm quality evaluation system provided by the embodiments of the present application is aimed at the problem that the number of observed sperm cells is small and complex staining operations are required when existing sperm morphology and DNA fragmentation rate detection is performed, adopts Figure 5 several modules in the system, and is based on a high-throughput, multi-dimensional flow cytometry imaging system to obtain intensity, phase, bright field and dark field images of sperm cells, establish a massive sperm cell image database, and construct, train and optimize a sperm cell multi-dimensional image analysis model based on a multi-model fusion deep learning network, so as to realize high-speed, high-precision and label-free sperm quality evaluation.

[0089] It should be noted that the system embodiments provided by the present application are used to implement the methods in the above method embodiments, and are also used to implement the methods in other method embodiments provided by the present application, the difference is only that the corresponding function modules are set, and the principle is basically the same as that of the above system embodiments provided by the present application, as long as the person skilled in the art improves the equipment in the above system embodiments on the basis of the above system embodiments, refers to the specific technical solutions in other method embodiments, obtains the corresponding technical means by combining technical features, and the technical solutions composed of these technical means, on the premise of ensuring the practicability of the technical solutions, the corresponding system class embodiments are obtained, which are used to implement the methods in other method class embodiments. For example:

[0090] Based on the content of the above system embodiment, as a preferred embodiment, the sperm quality evaluation system provided in the embodiment of the present application further comprises:

[0091] The verification module is configured to verify the prediction result of the model by the clinical sperm morphology and DNA fragmentation index detection result, and optimize the deep learning algorithm according to the verification result.

[0092] Based on the content of the above system embodiment, as a preferred embodiment, the sperm quality evaluation system provided in the embodiment of the present application further comprises:

[0093] The integration module is configured to integrate a quantitative phase imaging system on the basis of the optical flow time domain stretching imaging system, so as to obtain intensity and phase images of sperm cells; and integrate an optical polarization imaging system on the basis of the previous step, so as to obtain bright field and dark field images of the sperm cells.

[0094] Based on the content of the above system embodiment, as a preferred embodiment, the sperm quality evaluation system provided in the embodiment of the present application further comprises:

[0095] The database construction module is configured to collect semen samples, obtain a plurality of sperm cell images containing different morphological categories and different DNA fragmentation indexes through a multi-dimensional flow cytometry imaging system, preliminarily classify the obtained sperm cell images of different categories by using an image clustering method, and pick out and correctly classify the sperm cell images with wrong classification in the image clustering by using an artificial screening method, so as to establish a sperm cell image database.

[0096] Based on the content of the above system embodiment, as a preferred embodiment, in the sperm quality evaluation system without labeling provided in the embodiment of the application, the sperm cell morphology classification and DNA fragmentation rate prediction network model established includes: a feature extraction module, which is used to extract sperm cell intensity and phase image features through a Yolo-V3 network and extract sperm cell structure contour features through a ShumleNet-V2 network; a feature stacking module, which is used to stack all the extracted features by using a Concate method; and a feature classification module, which is used to classify each dimension image of the sperm cell by using a VGG-16 network to obtain sperm category classification and DNA fragmentation index prediction results.

[0097] Based on the same inventive concept as the above embodiments, the embodiment of the application further provides a sperm quality evaluation device without labeling, which comprises a memory and a processor, the memory stores program instructions executed by the processor, and the processor calls the program instructions to execute the steps of the sperm quality evaluation method without labeling.

[0098] In the embodiment of the application, the memory can be a non-volatile memory such as a hard disk (HDD) or a solid-state drive (SSD), and can also be a volatile memory such as a random-access memory (RAM). The memory can be any other medium capable of carrying or storing desired program codes in the form of instructions or data structures and capable of being accessed by a computer, but is not limited to this. The memory in the embodiment of the application can also be a circuit or other any device capable of realizing a storage function, used for storing program instructions and / or data.

[0099] In the embodiment of the application, the processor can be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can realize or execute the disclosed methods, steps and logic block diagrams in the embodiment of the application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in combination with the embodiment of the application can be directly embodied as a hardware processor for execution, or executed by a combination of hardware and software modules in the processor.

[0100] Based on the same inventive concept as the above embodiments, the embodiment of the application further provides a non-transitory computer readable storage medium, which stores computer instructions, and the computer instructions make the computer execute the steps of the sperm quality evaluation method without labeling.

[0101] In summary of the above embodiments, based on a high-throughput multi-dimensional flow cytometry imaging system, intensity, phase, bright field and dark field images of sperm cells are obtained, a mass sperm cell image database is established, a sperm cell multi-dimensional image analysis model based on a multi-model fusion deep learning network is constructed, trained and optimized. On this basis, the trained sperm cell multi-dimensional image analysis network model is used to analyze sperm cell images obtained by the multi-dimensional flow cytometry imaging system for detecting semen samples, analyze the proportion of each morphological category of sperm in the semen sample, sperm DNA fragmentation rate and other sperm structure and function parameters, and realize high-speed, high-precision and label-free sperm quality evaluation.

[0102] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present application.

Claims

1. A label-free sperm quality assessment method, characterized by, The application relates to a method for evaluating sperm quality, comprising the following steps: detecting sperm cells in a semen sample by using a multi-dimensional flow cytometry imaging system to obtain intensity, phase, bright field and dark field images of the sperm cells; inputting the obtained intensity, phase, bright field and dark field images of the sperm cells into a trained sperm cell morphology classification and DNA fragmentation rate prediction network model to output sperm cell morphology classification and DNA fragmentation rate prediction results, and evaluating sperm quality according to the sperm cell morphology classification and DNA fragmentation rate prediction results; wherein the training of the sperm cell morphology classification and DNA fragmentation rate prediction network model comprises the following steps: constructing a sperm cell image database containing all sperm morphology categories and sperm DNA fragmentation rates, and distinguishing sperm cells of different morphology categories and different DNA fragmentation rates; constructing a sperm cell multi-dimensional image analysis framework based on a multi-model fusion deep learning network for extracting and analyzing multi-dimensional features of sperm cell images of known morphology categories and DNA fragmentation rates, and establishing a sperm cell morphology classification and DNA fragmentation rate prediction network model; training the established sperm cell morphology classification and DNA fragmentation rate prediction network model by using the sperm cell image database, and outputting a trained sperm cell morphology classification and DNA fragmentation rate prediction network model.

2. The method as claimed in claim 1, wherein, The training of the sperm cell morphology classification and DNA fragmentation rate prediction network model further comprises the following steps: verifying the prediction results of the model by using clinical sperm morphology and DNA fragmentation rate index detection results, and optimizing the deep learning algorithm according to the verification results.

3. The method as claimed in claim 1, wherein, detecting sperm cells in a semen sample by using a multi-dimensional flow cytometry imaging system, comprising the following steps: integrating a quantitative phase imaging system on the basis of a photonic time-domain stretch imaging system to obtain intensity and phase images of sperm cells; integrating an optical polarization imaging system on the basis of the previous step to obtain bright field and dark field images of sperm cells.

4. The method as claimed in claim 1, wherein, constructing a sperm cell image database containing all sperm morphology categories and sperm DNA fragmentation rates, and distinguishing sperm cells of different morphology categories and different DNA fragmentation rates, comprising the following steps: collecting semen samples, and obtaining a plurality of sperm cell images containing different morphology categories and different DNA fragmentation rate indexes by detecting the semen samples by using a multi-dimensional flow cytometry imaging system; adopting an image clustering method to preliminarily classify the obtained sperm cell images of different categories, and picking out and correctly classifying sperm cell images with wrong classification in the image clustering by using an artificial screening method, so as to establish a sperm cell image database.

5. The method as claimed in claim 1, wherein, The established sperm cell morphology classification and DNA fragmentation rate prediction network model comprises the following steps: a feature extraction module is used for extracting sperm cell intensity and phase image features by using a Yolo-V3 network, and extracting sperm cell structure contour features by using a ShumleNet-V2 network; a feature stacking module is used for stacking all the extracted features by using a Concate method; and a feature classification module is used for classifying sperm cell images of all dimensions by using a VGG-16 network to obtain sperm category classification and DNA fragmentation index prediction results.

6. A label-free sperm quality assessment system, characterized by, The application relates to a method for evaluating sperm quality, comprising the following steps: An image acquisition module is configured to detect sperm cells in a semen sample using a multi-dimensional flow cytometry imaging system to obtain intensity, phase, bright field and dark field images of the sperm cells; An image processing module is configured to input the obtained intensity, phase, bright field and dark field images of the sperm cells into a trained sperm cell morphology classification and DNA fragmentation rate prediction network model, output sperm cell morphology classification and DNA fragmentation rate prediction results, and evaluate sperm quality according to the sperm cell morphology classification and DNA fragmentation rate prediction results. The training of the sperm cell morphology classification and DNA fragmentation rate prediction network model comprises: constructing a sperm cell image database containing all sperm morphology categories and sperm DNA fragmentation rates, and distinguishing sperm cells of different morphology categories and different DNA fragmentation rates; constructing a sperm cell multi-dimensional image analysis framework based on a multi-model fusion deep learning network for extracting and analyzing multi-dimensional features of sperm cell images of known morphology categories and DNA fragmentation rates, and establishing a sperm cell morphology classification and DNA fragmentation rate prediction network model; training the established sperm cell morphology classification and DNA fragmentation rate prediction network model using the sperm cell image database, and outputting a trained sperm cell morphology classification and DNA fragmentation rate prediction network model.

7. A label-free sperm quality assessment system, characterized by, It comprises: a multi-dimensional flow cytometry imaging system and a computing device; the multi-dimensional flow cytometry imaging system is configured to detect sperm cells in a semen sample to obtain intensity, phase, bright field and dark field images of the sperm cells; and the computing device is configured to input the obtained intensity, phase, bright field and dark field images of the sperm cells into a trained sperm cell morphology classification and DNA fragmentation rate prediction network model, output sperm cell morphology classification and DNA fragmentation rate prediction results, and evaluate sperm quality according to the sperm cell morphology classification and DNA fragmentation rate prediction results.

8. A label-free sperm quality assessment device, characterized by, It comprises a memory and a processor, the memory stores program instructions executed by the processor, and the processor invokes the program instructions to perform the steps of the label-free sperm quality evaluation method according to any one of claims 1 to 5.

9. A non-transitory computer-readable storage medium, comprising: The non-transitory computer readable storage medium stores computer instructions, and the computer instructions cause the computer to perform the steps of the label-free sperm quality evaluation method according to any one of claims 1 to 5.

Images