A method for detecting highly active sperm without chemical staining

Abstract

The present invention discloses a method for detecting highly active sperm without chemical staining, comprising: collecting digital images of healthy sperm without chemical staining by using a sperm activity sorting and observation device based on microfluidics; performing non-linear intelligent color transfer processing on the collected digital images by using a complex deep neural network color transfer model; performing binary processing on the R and B channels of the stained images to respectively determine the head positions and tail positions of each sperm, jointly determining the positions of the corresponding complete sperm by the two and intercepting them; inputting the sperm images into a sperm discrimination convolutional neural network based on deep learning to discriminate whether they are complete and healthy. The present invention realizes chemical-staining-free imaging of sperm cells during the sorting process of the sperm activity sorting and observation device based on microfluidics, enabling cells to achieve the same observation and detection effects as those after staining without chemical staining, and improving the efficiency of sperm observation and detection.

Description

Technical Field

[0001] The present invention belongs to the field of semen microscopic analysis, and particularly relates to a method for detecting highly active sperm without chemical staining. Background Art

[0002] Sperm health is a complex concept, and the main indicators include the physical characteristics of semen, sperm density, sperm motility rate, sperm vitality, acrosome reaction ability, sperm morphology, non-sperm cell content, etc. However, at present, the clinical screening of sperm mainly relies on two indicators: vitality and morphology.

[0003] Sperm vitality is considered an important parameter closely related to sperm quality. A safe and effective method for screening viable sperm has great value for clinical applications. Currently, the commonly used sperm screening methods in clinics include: gradient centrifugation method and sperm upstream method. No matter which method is used, it is inevitable to cause damage to the sperm itself during the process of handling sperm. Various operations during sperm processing, including sperm liquefaction, centrifugation to remove seminal plasma, room temperature incubation, sperm freezing, etc., can cause sperm DNA breakage and functional decline. The use of a microfluidic chip can change this situation. Its stable fluid environment can avoid mechanical damage to sperm to the greatest extent. Currently, there are microchannel screening mechanisms, dielectrophoretic force screening, and laminar flow effect screening for sperm vitality.

[0004] As a key step in conventional in vitro fertilization, the morphological examination of sperm is also a guarantee for subsequent fertilization success. Morphological detection mainly relies on cytochemical staining and microscopy. For example, the patent document with the application publication number CN 110458821A discloses a method for sperm morphology analysis based on a deep neural network model. This method relies on sperm staining images or sperm images under a phase contrast microscope as the observation objects for observation, and the detection effect is greatly affected by the smear. In terms of staining, taking the hematoxylin-eosin (H&E) staining method as an example: the cell genetic material region is reflected as a blue-violet appearance, and the cell matrix and cytoplasm parts are reflected as a pink appearance, that is, the sperm head appears blue and the tail appears red. For this staining method, on the one hand, the final observation effect of the stained slide is greatly affected by the operation errors of medical workers during staining and smearing, and the observation and judgment effect is greatly reduced; on the other hand, the chemical staining solution will damage cell activity, making it difficult for clinicians to directly obtain morphologically intact healthy highly active sperm that can be used for in vitro fertilization in the field of view, greatly reducing the efficiency of screening healthy highly active sperm. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for detecting highly active sperm without chemical staining, which improves the observation and detection efficiency of sperm in pathological examinations.

[0006] To achieve the above purpose, the present invention adopts the following solutions:

[0007] Step 1: Use a microfluidics-based sperm motility sorting and observation device to collect a digital image I0 of healthy sperm without chemical staining as the image to be detected.

[0008] Step 2: Construct an initial complex deep neural network color transfer model, optimize the initial model through a training sample set to obtain a complex deep neural network color transfer model, and use the complex deep neural network color transfer model to perform non-linear intelligent color transfer processing on the digital image I0 of non-chemically stained sperm in Step 1 to obtain a stained image I c containing n sperm, where n is 20 - 30, the staining style is the medical H&E staining standard, the sperm head appears blue, and the tail appears red.

[0009] Step 3: Perform binarization processing on the R and B channels of the stained image I c to respectively determine the head position and tail position of each sperm, jointly determine the position of its corresponding complete sperm from the two, and then extract it from the stained image I c to obtain n sperm images I c_i where 1 ≤ i ≤ n.

[0010] Step 4: Input the sperm image I c_i into a sperm discrimination convolutional neural network based on deep learning, and use the standards in the "WHO Laboratory Manual for the Examination and Processing of Human Semen" to determine whether the sperm image I c_i is a morphologically complete healthy sperm. According to the confidence probability, mark its position and confidence probability in I0 and I c to detect the active healthy sperm in the image I0 and the stained image I c .

[0011] Furthermore, in Step 2, the complex deep neural network color transfer model is used to perform non-linear intelligent color transfer processing on the digital image I0 of non-chemically stained sperm in Step 1 to obtain a staining result simulating traditional histopathology, combined with a microfluidics-based sperm motility sorting and observation device, thereby greatly shortening the semen processing time and improving the observation efficiency.

[0012] Furthermore, the construction process of the complex deep neural network color transfer model in Step 2 has the following steps:

[0013] Step 2-1: Set the first complex generator c_G1 to transform the digital image from the X domain to the Y domain, and the second complex generator c_G2 to transform the digital image from the Y domain to the X domain. The transformation result is a complex image with pixel values as complex numbers, completing the transfer of the color style, where the complex images in the X domain and the Y domain refer to dividing the X domain and the Y domain according to the amplitude part of the complex image, and directly dividing for ordinary digital images;

[0014] Step 2-2: Set the first complex discriminator D Y , D Y Perform confidence discrimination on the amplitude matrix of the complex image in the Y domain and perform normalization discrimination on the phase matrix; set the second complex discriminator D X , D X Perform confidence discrimination on the amplitude matrix of the complex image in the X domain and perform normalization discrimination on the phase matrix;

[0015] Step 2-3: Set the losses, the first adversarial loss L GAN (c_G1, D Y , X, Y), the second adversarial loss L GAN (c_G2, D X , Y, X), the first cycle reconstruction discriminator D cyc1 (c_G1, c_G2), the second cycle reconstruction discriminator D cyc1 (c_G2, c_G1). The cycle reconstruction discriminators and the loss functions are all established separately for complex operations to calculate the losses and update the complex weights of the generators and discriminators through backpropagation.

[0016] The two complex generators have the same structure. The complex generator includes 1 complex convolutional encoding block, 9 layers of complex residual layers, and 1 complex convolutional decoding block. The complex convolutional operations of the image used in the encoding block, decoding block, and residual block are defined as: W·z = (Ax - By) + i(Bx + Ay), and the activation function RELU is defined as RELU(z) = RELU(x) + iRELU(y), where the weight matrix W of the convolutional layer = A + iB, the layer input z = x + iy, i is the imaginary unit, A is the real part of the weight matrix, B is the imaginary part of the weight matrix, (x, y) is the complex plane representation coordinate of the layer input, and the output result of the generator is a complex matrix, and the square of the modulus is used as the output intensity image.

[0017] The two complex discriminators have the same structure. The complex discriminator includes 1 amplitude discrimination block composed of 5 layers of amplitude discrimination complex convolutional layers and 1 phase normalization discrimination block composed of 5 layers of phase normalization discrimination complex convolutional layers. The amplitude discrimination entrance receives the modulus of the complex matrix, and the phase discrimination entrance receives the phase information of the complex matrix. The sum of the weights of each layer of amplitude discrimination complex convolutional layer and phase normalization discrimination complex convolutional layer is used as the discriminator weight. Among them, the weight of the amplitude discrimination block is a complex number with an imaginary part of 0, and there is no such restriction on the weight of the phase normalization discrimination block. The discrimination results of amplitude and phase are multiplied and fused at the corresponding positions to output 1 complex matrix.

[0018] Furthermore, the first cycle reconstruction discriminator D cyc1 (c_G1, c_G2) and the second cycle reconstruction discriminator D cyc2(c_G2, c_G1) both contain three discriminant modules: L1, L2, and L3, which are the color component loss, the pattern texture loss, and the image contrast loss respectively.

[0019] Furthermore, the first adversarial loss is:

[0020] L GAN (c_G1, D Y , X, Y) = E y～Pdata(y) [log D Y (y)] + E x～Pdata(x) [log(1 - D Y (c_G1(x)))]

[0021] Furthermore, the second adversarial loss is:

[0022] L GAN (c_G2, D X , Y, X) = E x～Pdata(x) [log D X (x)] + E y～Pdata(y) [log(1 - D X (c_G2(y)))]

[0023] Where E y～Pdata(y) [log D Y (y)] is the loss of real images in the Y domain, E x～Pdata(x) [log D X (x)] is the loss of real images in the X domain, E x～Pdata(x) [log(1 - D Y (c_G1(x)))] is the loss of transferred images in the X domain, E y～Pdata(y) [log(1 - D X (c_G2(y)))] is the loss of transferred images in the Y domain.

[0024] Furthermore, the training steps of the complex deep neural network color transfer model are as follows:

[0025] Step a: Randomly initialize the network weights of each layer of the complex deep neural network color transfer model;

[0026] Step b: Collect digital images of sperm without chemical staining and digital images of standard medical H&E staining as the training sample set, input them into the initial complex deep neural network color transfer model, and attach a random initial phase, and perform color transfer and phase normalization through multiple training iterations;

[0027] Step c: Wait for the generated results and discriminant results to converge, and save the complex deep neural network color transfer model at this time as the complex deep neural network color transfer model.

[0028] Further, in step 3, after binarizing the R and B channels of the stained image I c , the obtained images respectively highlight the tail and head features of sperm cells. After performing erosion and dilation processing on the images, retrieving the global content of the images, the positioning frames of sperm heads and tails can be obtained. Then, the positions of individual sperm cells are determined by combining the two, and cropped to obtain n sperm images I c_i , where 1 ≤ i ≤ n.

[0029] Preferably, the sperm discrimination convolutional neural network described in step 4 can be a conventional convolutional neural network for pattern recognition, such as Alexnet, etc.

[0030] Compared with the prior art, the advantages of the present invention are:

[0031] (1) By constructing a complex deep neural network color transfer model, the present invention realizes the chemical-free staining imaging of sperm cells during the sorting process of a sperm activity sorting and observation device based on microfluidics, and successfully solves the contradiction between "high cell activity" and "staining observation".

[0032] (2) Compared with the real-number-based color transfer network model, the complex deep neural network color transfer model introduces amplitude and phase discrimination, making the model have stronger convergence ability; the method repeats the positioning of sperm heads and tails in different image channels and inputs them into a sperm discrimination convolutional neural network based on deep learning for discrimination according to the standards in the "WHO Laboratory Manual for the Examination and Processing of Human Semen", greatly improving the detection accuracy, and thus greatly improving the sperm observation and detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a flowchart of a high-activity sperm detection method based on chemical-free staining.

[0034] Figure 2 is a schematic diagram of a deep neural network color transfer model.

[0035] Figure 3 is a schematic diagram of the structure of a complex generator.

[0036] Figure 4 is a schematic diagram of the structure of a complex discriminator. DETAILED DESCRIPTION OF THE INVENTION

[0037] The following will further introduce the specific implementation manner, as well as the technical difficulties and inventive points of the present invention in combination with the design examples.

[0038] As Figure 1 shown, a high-activity sperm detection method based on chemical-free staining includes the following steps:

[0039] Step 1: Use a microfluidics-based sperm motility sorting and observation device to collect a digital image I0 of healthy sperm without chemical staining as the image to be detected.

[0040] Step 2: Use a complex deep neural network color transfer model to perform non-linear intelligent color transfer processing on the digital image I0 of sperm without chemical staining in Step 1 to obtain a stained image I containing n sperm, where n ranges from 20 to 30, the staining style is the medical H&E staining standard, the sperm head appears blue, and the tail appears red. This process realizes chemical-free staining imaging of sperm cells during the sorting process of the microfluidics-based sperm motility sorting and observation device, successfully solves the contradiction between "high cell activity" and "staining observation", enables the cells to achieve the same observation effect as staining without chemical staining, thus greatly improving the sperm observation and detection efficiency; on the other hand, compared with the real-number-based color transfer network model, the complex deep neural network color transfer model introduces the joint constraint discrimination of amplitude and phase, making the model have stronger convergence ability and generalization ability. c

[0041] Step 3: Perform binarization processing on the R and B channels of the stained image I, detect the connected regions of each image target, respectively determine the head position and tail position of each sperm, jointly determine the position of its corresponding complete sperm based on the positioning information of the two, and then crop it from the stained image I to obtain n sperm images I, where 1 ≤ i ≤ n. This process improves the positioning accuracy by repeatedly positioning the sperm head and tail in different image channels, and the previous calculation and staining process also enables a simpler image denoising algorithm to be used to remove noise interference before positioning, reducing the computational pressure on the computer. c c c_i

[0042] Step 4: Input the sperm image I into a deep learning-based sperm discrimination convolutional neural network, such as Alexnet trained with a large number of sperm sample data, and judge whether the sperm image I is a morphologically complete healthy sperm according to the standards in the "WHO Laboratory Manual for the Examination and Processing of Human Semen". Mark its position and confidence probability in I0 and I according to the confidence probability of the judgment result, so as to detect the active healthy sperm in the image I0 and the stained image I. c_i c_i c c

[0043] The following specifically describes the construction method of the deep neural network color transfer model in combination with Figure 2 .

[0044] ​​​​​​​​Step 2-1: Set the first complex generator c_G1 to transform the digital image from the X domain to the Y domain, and the second complex generator c_G2 to transform the digital image from the Y domain to the X domain. The transformation result is a complex image with complex pixel values, completing the transfer of color style. The complex images in the X domain and Y domain refer to dividing the X domain and Y domain according to the amplitude part of the complex image, and directly dividing the ordinary digital image; since the propagation process of light can be described by a complex form, and the two-dimensional convolution calculation of the image corresponds to the frequency spectrum calculation, therefore, compared with the real number network, the complex-based neural network has a stronger constraint ability for the model and is easier to converge to a certain extent. In the construction of the generator, complex numbers are easier to fit into the actual physical process.

[0045] Step 2-2: Set the first complex discriminator D Y , D Y to perform a confidence discrimination on the amplitude matrix of the complex image in the Y domain, aiming to determine the real Y domain image as true and the generated Y domain image as false, and perform a normalization discrimination on the phase matrix; set the second complex discriminator D X , D X to perform a confidence discrimination on the amplitude matrix of the complex image in the X domain, aiming to determine the original X domain image as true and the generated X domain image as false, and perform a normalization discrimination on the phase matrix;

[0046] Step 2-3: Set the first cyclic reconstruction discriminator D cyc1 (c_G1, c_G2) to determine whether the X domain image generated by the real X domain image passing through the first complex generator c_G1 and the second complex generator c_G2 is true; set the second cyclic reconstruction discriminator D cyc2 (c_G2, c_G1) to determine whether the Y domain image generated by the real Y domain image passing through the second complex generator c_G2 and the first complex generator c_G1 is true, thereby constraining the cyclic convergence of the two generators. And set the first adversarial loss L GAN (c_G1, D Y , X, Y), the second adversarial loss L GAN (c_G2, D X , Y, X). The cyclic reconstruction discriminator and the loss function are both established separately for complex number operations to calculate the loss and backpropagate to update the complex weights of the generator and the discriminator.

[0047] Furthermore, the first cyclic reconstruction discriminator D cyc1 (c_G1, c_G2) and the second cyclic reconstruction discriminator D cyc2(c_G2, c_G1) both contain 3 discriminant modules: L1, L2, and L3, which are the color component loss, the pattern texture loss, and the image contrast loss respectively. The set cyclic reconstruction discriminator aims to strictly constrain the quality of the color components, texture details, and contrast of the image after each migration and reconstruction of the image.

[0048] Furthermore, the first adversarial loss is:

[0049] L GAN (c_G1, D Y , X, Y) = E y～Pdata(y) [log D Y (y)] + E x～Pdata(x) [log(1 - D Y (c_G1(x)))]

[0050] The second adversarial loss is:

[0051] L GAN (c_G2, D X , Y, X) = E x～Pdata(x) [log D X (x)] + E y～Pdata(y) [log(1 - D X (c_G2(y)))]

[0052] Among them, E y～Pdata(y) [log D Y (y)] is the loss of the real image in the Y domain, E x～Pdata(x) [log D X (x)] is the loss of the real image in the X domain, E x～Pdata(x) [log(1 - D Y (c_G1(x)))] is the loss of the migrated image in the X domain, E y～Pdata(y) [log(1 - D X (c_G2(y)))] is the loss of the migrated image in the Y domain.

[0053] As Figure 3 shown, the two complex generators have the same structure. The complex generator includes 1 complex convolutional encoding block, 9 layers of complex residual layers, and 1 complex convolutional decoding block. The complex convolutional operations of the image used in the encoding block, decoding block, and residual block are defined as: W·z = (Ax - By) + i(Bx + Ay), and the activation function RELU is defined as RELU(z) = RELU(x) + iRELU(y), where the weight matrix W of the convolutional layer = A + iB, the layer input z = x + iy, i is the imaginary unit, A is the real part of the weight matrix, B is the imaginary part of the weight matrix, (x, y) is the coordinate of the complex plane representation of the layer input, and the output result of the generator is a complex matrix, and the square of the modulus is used as the intensity image of the output.

[0054] As Figure 4 shown, the two complex discriminator structures are the same. The complex discriminator includes an amplitude discrimination block composed of 5 layers of amplitude discrimination complex convolutional layers and a phase normalization discrimination block composed of 5 layers of phase normalization discrimination complex convolutional layers. The amplitude discrimination entrance receives the modulus of the complex matrix, and the phase discrimination entrance receives the phase information of the complex matrix. The sum of the weights of each layer of amplitude discrimination complex convolutional layer and phase normalization discrimination complex convolutional layer is used as the discriminator weight. Among them, the weight of the amplitude discrimination block is a complex number with an imaginary part of 0, and there is no such restriction on the weight of the phase normalization discrimination block. The discrimination results of amplitude and phase are multiplied and fused at the corresponding positions to output a complex matrix.

[0055] The training steps of the complex deep neural network color transfer model are specifically as follows:

[0056] Step a: Randomly initialize the network weights of each layer of the complex deep neural network color transfer model;

[0057] Step b: Collect digital images of sperm without chemical staining and digital images stained with the medical H&E staining standard as the training sample set, input them into the initial complex deep neural network color transfer model, and attach a random initial phase. Perform color transfer and phase normalization through multiple training iterations;

[0058] Step c: Wait for the generated result and the discriminant result to converge, and save the complex deep neural network color transfer model at this time as the complex deep neural network color transfer model.

Claims

1. A method for detecting highly active sperm without chemical staining, characterized in that, The steps are as follows Step 1: Use a microfluidics-based sperm motility sorting and observation device to collect a digital image I0 of healthy sperm without chemical staining as the image to be detected; Step 2: Construct an initial complex deep neural network color transfer model, optimize the initial model through a training sample set to obtain a complex deep neural network color transfer model, and use the complex deep neural network color transfer model to perform non-linear intelligent color transfer processing on the digital image I0 of the sperm without chemical staining in Step 1 to obtain a stained image I containing n spermatozoa c , where n is from 20 to 30, the staining style is the medical H&E staining standard, the sperm head appears blue, and the tail appears red; Among them, an initial complex deep neural network color transfer model is constructed as follows: Step 2-1: Set the first complex generator c_G1 to transform the digital image from the X domain to the Y domain, and the second complex generator c_G2 to transform the digital image from the Y domain to the X domain. The transformation result is a complex image with pixel values being complex numbers, completing the transfer of color style. Among them, the complex images in the X domain and the Y domain refer to dividing the X domain and the Y domain according to the amplitude part of the complex image, and directly dividing for ordinary digital images; Step 2-2: Set the first complex discriminator D Y , D Y Perform confidence discrimination on the amplitude matrix of the complex image in the Y domain and perform normalization discrimination on the phase matrix; Set the second complex discriminator D X , D X Perform confidence discrimination on the amplitude matrix of the complex image in the X domain and perform normalization discrimination on the phase matrix; Step 2-3: Set losses, the first adversarial loss L GAN (c_G1, D Y , X, Y), the second adversarial loss L GAN (c_G2, D X , Y, X), the first cycle reconstruction discriminator D cyc1 (c_G1, c_G2), the second cycle reconstruction discriminator D cyc1 (c_G2, c_G1). The cycle reconstruction discriminator and the loss function are both established separately for complex number operations to calculate losses and backpropagate to update the complex number weights of the generator and the discriminator; Step 3. Perform binarization processing on the R and B channels of the stained image I c to respectively determine the head positions and tail positions of each sperm, jointly determine the positions of the corresponding complete sperm from the two, and then extract them from the stained image I c to obtain n sperm images I c_i , where 1 ≤ i ≤ n; Step 4: Input the sperm image I c_i into the sperm discrimination convolutional neural network based on deep learning, and judge the sperm image I c_i according to the standards in the "WHO Laboratory Manual for the Examination and Processing of Human Semen" to determine whether it is a morphologically complete and healthy sperm. Based on the confidence probability, mark its position and confidence probability in I0 and I c to detect the active and healthy sperm in the images I0 and the stained image I c ​ 2. The highly active sperm detection method without chemical staining according to claim 1, wherein: The two complex generators have the same structure. The complex generator includes 1 complex convolutional encoding block, 9 layers of complex residual layers, and 1 complex convolutional decoding block. The complex convolutional operations of the image used in the encoding block, decoding block, and residual block are defined as: W·z = (Ax - By) + i(Bx + Ay), and the activation function RELU is defined as RELU(z) = RELU(x) + iRELU(y), where the weight matrix W of the convolutional layer = A + iB, the layer input z = x + iy, i is the imaginary unit, A is the real part of the weight matrix, B is the imaginary part of the weight matrix, (x, y) is the coordinate representation of the complex plane of the layer input, and the output result of the generator is a complex matrix, and the square of the modulus is used as the output intensity image.

3. The highly active sperm detection method without chemical staining according to claim 2, characterized in that The two complex discriminators have the same structure. The complex discriminator includes 1 amplitude discrimination block composed of 5 layers of amplitude discrimination complex convolutional layers and 1 phase normalization discrimination block composed of 5 layers of phase normalization discrimination complex convolutional layers. The amplitude discrimination entrance receives the modulus of the complex matrix, and the phase discrimination entrance receives the phase information of the complex matrix. The sum of the weights of each layer of amplitude discrimination complex convolutional layer and phase normalization discrimination complex convolutional layer is used as the discriminator weight. Among them, the weight of the amplitude discrimination block is a complex number with an imaginary part of 0, and there is no such restriction on the weight of the phase normalization discrimination block. The discrimination results of amplitude and phase are multiplied and fused at the corresponding positions to output 1 complex matrix.

4. The highly active sperm detection method without chemical staining according to claim 3, characterized in that The first cycle reconstruction discriminator D cyc1 (c_G1, c_G2) and the second cycle reconstruction discriminator D cyc2 (c_G2, c_G1) both contain three discriminant modules: L1, L2, and L3, which are the color component loss, the pattern texture loss, and the image contrast loss respectively; The first adversarial loss is: L GAN (c_G1,D Y ,X,Y) = E y～Pdata(y) [logD Y (y)] + E x～Pdata(x) [log(1 - D Y (c_G1(x)))] The second adversarial loss is: L GAN (c_G2,D X ,Y,X) = E x～Pdata(x) [logD X (x)] + E y～Pdata(y) [log(1 - D X (c_G2(y)))] Among them, E y～Pdata(y) [logD Y (y)] is the real image loss in the Y domain, E x～Pdata(x) [logD X (x)] is the real image loss in the X domain, E x～Pdata(x) [log(1 - D Y (c_G1(x)))] is the transferred image loss in the X domain, E y～Pdata(y) [log(1 - D X (c_G2(y)))] is the transferred image loss in the Y domain.

5. The highly active sperm detection method without chemical staining according to claim 1, characterized in that, In Step 2, the training steps of the complex deep neural network color transfer model are specifically as follows: Step a: Randomly initialize the network weights of each layer of the complex deep neural network color transfer model; Step b: Collect digital images of sperm without chemical staining and digital images stained with the medical H&E staining standard as the training sample set, input them into the initial complex deep neural network color transfer model, and add a random initial phase, and perform color transfer and phase normalization through multiple training iterations; Step c: Wait for the generated result and the discriminant result to converge, and save the complex deep neural network color transfer model at this time as the complex deep neural network color transfer model.

Images