A deep learning-based semen quality evaluation method

Abstract

The application relates to the technical field of semen quality evaluation, and discloses a semen quality evaluation method based on deep learning, which comprises the following operation steps: S1, pre-processing a data set, selecting data of 85 male participants aged 18 or above, the data containing a video of a live sperm sample of each participant, using 60% of the data set for training, using 20% as a verification set, using 20% as a test set, pre-processing the training set video, and taking 9 continuous frames as a sample input. The semen quality evaluation method based on deep learning directly extracts video information of the semen video by using a light-weight MobileNet, improves the convolutional neural network, adds other data in addition to the video information, increases the prediction accuracy of the network, deploys the neural network by using a front-end and back-end separated Web end, solves the prediction efficiency problem, and makes the whole system better in operation and better in displaying the prediction result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of semen quality evaluation, in particular to a semen quality evaluation method based on deep learning. BACKGROUND

[0002] Male fertility depends on sperm concentration, the direction of sperm movement and its morphology (the shape of the sperm head and the size of the tail), based on these fertility factors, sperm motility analysis is very important to determine male fertility, such as more accurately detecting and tracking sperm, or directly extracting video information from sperm dynamic video, so as to more accurately diagnose infertility problems, the most common method of analyzing sperm is through experts observing sperm under a microscope and reporting their motility quality, quantity and morphology, which is very difficult, in addition to manual methods, computer-aided sperm analysis (CASA) systems are also used for sperm analysis, in the past few decades, CASA systems have been greatly improved, now they are faster and more accurate than manual observation, CASA systems use different algorithms from sperm image or video samples to detect sperm quantity, morphology, and especially motility, CASA systems mainly use classic image processing algorithms, which are high in cost and low in accuracy, and require high purity of samples, in the past few years, deep learning has become the latest technology in the field of computer vision, people have begun to apply deep learning artificial intelligence algorithms to sperm target detection and tracking, so as to achieve the purpose of detecting sperm activity and movement indicators, the general steps are first detecting the static coordinate position of sperm cells through a deep learning algorithm, second tracking sperm cells using various filters, and finally calculating the speed of cell movement using the distance and time of movement.

[0003] There were some early studies on sperm tracking in the 1980s and 1990s, in one study, a single sperm tracking algorithm was proposed, which tracked sperm by creating a region around them and using a modified four-class threshold, then performing post-collision analysis to determine the tracked sperm in the image, this method also used a speed check function to track sperm close to other sperm or particles, another study aimed to develop a robotic system for immobilized motile sperm (for clinical injection, artificial insemination) to track the head of a single sperm, which proposed another method using an improved Gaussian mixture filter (GM-PHD) to track multiple sperm at the same time, there was also a study that used the Laplacian of a Gaussian operator to detect the head of sperm, then tracked the moving sperm using a combination of particles and Kalman filters, most of the work on sperm tracking in the past few decades was based on single sperm tracking, when sperm collided with each other, the effect was not good, it would send tracking objects wrong, or lose track, thus causing errors in the calculation of subsequent sperm movement indicators, although there have been studies on multi-sperm tracking in recent years, many of these studies have only been evaluated on a small data set, the research only stays on the improvement of the accuracy of the algorithm, and has not been applied to the actual semen sample indicator detection, with the popularization and application of artificial intelligence algorithms, there have been studies that have proposed an automatic method for detecting and tracking sperm, which performs detection in the first frame of the video by applying the bag-of-words method and SVM classification, then detected sperm will be tracked in other frames by using the average shift method, although this study has achieved good results, its disadvantage is that it uses a small tree data set, the model generalization ability is insufficient, and it does not work well in samples with medium or high density of cells, therefore, a semen quality evaluation method based on deep learning is proposed to solve the above problems. SUMMARY

[0004] (I) Technical problems solved

[0005] The purpose of the present application is to solve the above problems, and a semen quality evaluation method based on deep learning is proposed.

[0006] (II) Technical solutions

[0007] The technical solutions of the present application to solve the above technical problems are as follows:

[0008] A semen quality evaluation method based on deep learning, comprising the following operation steps:

[0009] S1, pre-process the data set, select the data of 85 male participants aged 18 or above, the data contains the video of the live sperm sample of each participant, 60% of the data set is used for training, 20% is used as a validation set, and 20% is used as a test set, the training set video is pre-processed, and 9 consecutive frames are cut off as a sample input;

[0010] S2, training the improved convolutional neural network, using a light-weight convolutional network MobileNet as a backbone network, adding the levels of several fatty acids in sperm, serum levels of phospholipid fatty acids in the last fully connected layer in series, then using the improved light-weight convolutional network to extract information from the input continuous image frame, calculating the loss function by forward propagation, adjusting the parameters of the convolutional neural network using the gradient descent method by backward propagation, so that the result of the loss function decreases, when the calculation results of the loss function on the training set and the validation set tend to be stable, fine-tuning the learning rate, continuing to train the convolutional network, first calculating the loss function by forward propagation, then adjusting the parameters of the convolutional neural network using the gradient descent method by backward propagation, so that the result of the loss function decreases, until the loss of the training set and the validation set tends to be stable, using the test set to test the result of network training, and giving evaluation indicators for evaluating the convolutional neural network;

[0011] S3, sample inference, including front-end design and back-end design, the front-end design is two web pages, which are a video loading interface and a result display interface, the back-end design is to package the trained deep learning network and the trained weight parameters into the back-end framework, load any semen sample video, call the trained convolutional neural network to predict the motility activity index of the semen and the morphological index of the sperm, and pass to the front-end page.

[0012] On the basis of the above technical scheme, the application can also be improved as follows.

[0013] Preferably, the data set in step S1 further includes the levels of several fatty acids in the sperm of the participants, the serum levels of phospholipid fatty acids, the measured sex hormone serum levels in the blood, the age, abstinence time and body mass index of the participants.

[0014] Preferably, the data set in step S1 contains more than 35GB of video, each video lasts for two to seven minutes, the frame rate is 50 frames per second, and the data set is made and arranged by a professional medical institution, which is effective.

[0015] Preferably, the parameter setting in the training process is that the video image input into the training set is preprocessed and adjusted to 224x224x3 size, 9 consecutive image frames are taken as the input of the improved light-weight network, then the input sperm image is randomly inverted and cropped to 16x112x112 to realize spatial and temporal jitter, Adam is selected as the optimizer, the initial learning rate is 0.0001, the learning rate is divided by 2 when the loss function result no longer decreases, and the pre-training is performed on the UCF101 classic action classification data set.

[0016] Preferably, the convolutional neural network parameter is set as: the whole network has 28 layers, first a 3x3 standard convolution, then a deep convolution module is stacked, the deep convolution module has two kinds, bottleneck_1 is a deep convolution module with a step of 1, which does not change the size of the feature map, bottleneck_2 is a deep convolution module with a step of 2, which down-samples the input feature map and reduces the length and width of the input feature map, the network is stacked by multiple deep separable convolution modules, which extracts the time and space information of the video image, then an average convolution layer is used to change the feature map into 1x1, then a full connection layer is connected, and the medical health information other than the video information: the level of fatty acid, the serum level of phospholipid fatty acid, the serum level of blood sex hormone, the age of the sample, the abstinence time and the body mass index are normalized and stacked in the full connection layer, and finally a softmax layer is used to output the prediction result.

[0017] (III) Beneficial effects

[0018] Compared with the prior art, the technical scheme of the present application has the following beneficial technical effects:

[0019] The present application directly uses a light-weight MobileNet to directly extract video information from the semen video, and improves the convolutional neural network by adding other data in addition to the video information, increases the prediction accuracy of the network, uses a front-end and back-end separated Web end to deploy the neural network, solves the efficiency problem of prediction, and makes the whole system better operated and better display the prediction result. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 A flow chart of a semen quality evaluation method based on deep learning is provided.

[0021] Figure 2 A sperm or perhaps morphological index block diagram of the semen quality evaluation method based on deep learning is provided.

[0022] Figure 3 A forward propagation calculation loss function of the semen quality evaluation method based on deep learning is provided.

[0023] Figure 4 An evaluation index of the convolutional neural network of the semen quality evaluation method based on deep learning is provided. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0025] In the embodiments, the sperm quality evaluation method based on deep learning is given, which comprises the following operation steps: Figures 1-4 In the embodiments, the sperm quality evaluation method based on deep learning is given, which comprises the following operation steps:

[0026] S1, pre-process the data set, select the data of 85 male participants aged 18 or above, the data includes the video of the live sperm sample of each participant, 60% of the data set is used for training, 20% is used as a validation set, and 20% is used as a test set, the training set video is pre-processed, and 9 consecutive frames are intercepted as a sample input, the data set also includes the levels of several fatty acids in the sperm of the participants, the serum levels of phospholipid fatty acids, the serum levels of sex hormones measured in the blood, the age, abstinence time and body mass index of the participants, the data set contains more than 35 GB of video, each video lasts two to seven minutes, the frame rate is 50 frames per second, the data set is made and arranged by a professional medical institution, and is valid;

[0027] S2, training an improved convolutional neural network, using a lightweight convolutional network MobileNet as a backbone network, and adding the levels of several fatty acids in the sperm, the serum levels of phospholipid fatty acids in the last fully connected layer in series, then using the improved lightweight convolutional network to extract information from the input continuous image frames, calculating the loss function by forward propagation, adjusting the parameters of the convolutional neural network by using the gradient descent method by backward propagation, so that the result of the loss function decreases, when the calculation results of the loss function on the training set and the validation set tend to be stable, the learning rate is fine-tuned, and the convolutional network is continuously trained, the loss function is calculated by forward propagation, and the parameters of the convolutional neural network are adjusted by using the gradient descent method by backward propagation, so that the result of the loss function decreases, until the loss of the training set and the validation set tends to be stable, the test set is used to test the result of network training, and evaluation indexes for evaluating the convolutional neural network are given;

[0028] S3, sample inference, including front-end design and back-end design, the front-end design is two web pages, which are a video loading interface and a result display interface, the back-end design is to pack the trained deep learning network and the trained weight parameters into the back-end framework, load any semen sample video, call the trained convolutional neural network, predict the motility index of the semen and the morphology index of the sperm, and pass to the front-end page.

[0029] The parameter setting in the training process is as follows: the video image input of the training set is preprocessed, adjusted to 224x224x3 size, 9 continuous frames of images are taken as the input of the improved lightweight network, then the input sperm image is randomly inverted and cropped to 16x112x112 to realize spatial and temporal jitter, Adam is selected as the optimizer, the initial learning rate is 0.0001, when the loss function result no longer decreases, the learning rate is divided by 2, and the UCF101 classic action classification data set is pre-trained

[0030] The convolutional neural network parameter setting is as follows: the whole network has 28 layers, first a 3x3 standard convolution, then a deep convolution module is stacked, the deep convolution module has two kinds, bottleneck_1 is a deep convolution module with a step of 1, which does not change the size of the feature map, bottleneck_2 is a deep convolution module with a step of 2, which down-samples the input feature map, reduces the length and width of the input feature map, the network is stacked by multiple deep separable convolution modules, extracts the time and space information of the video image, then an average convolution layer is used to change the feature map to 1x1, then a fully connected layer is connected, the medical health information other than the video information: the level of fatty acid, the serum level of phospholipid fatty acid, the serum level of blood sex hormone, the age of the sample, the abstinence time and the body mass index These parameters are normalized and stacked in the fully connected layer, and finally a softmax layer is used to output the prediction result.

[0031] Expected effect: the application proposes to use an improved lightweight convolutional neural network to predict the motion parameters and morphological parameters of sperm in semen, the improvement point: the medical health parameters other than video information are normalized and processed in the form of characteristic values, which are stacked in the last layer of the convolutional neural network, which improves the prediction accuracy of the network and improves the generalization ability of the model. Solve this new regression problem of predicting morphology and motion from sperm sample video. And use front-end: html, css, js, back-end: python, flask and other technologies to deploy the trained convolutional neural network to the Web end, realize end-to-end video information prediction, make the prediction process more concise, the cost is more affordable, and the result is better display.

[0032] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting; it is not intended to exclude myriad other embodiments of the present application that other inventors can develop based on the same general inventive concepts embodied by the described embodiments. That is, although the present application is described in terms of particular embodiments and implementations, it is to be understood that the terminology used is for the purpose of descriptive clarity and that it is intended to be limited only by the words recited in the appended claims. It is to be understood that the terms "including", "comprising", "consisting" and variations thereof do not preclude the addition of further integers to the claimed combination of integers. It is to be understood that the terms "including", "comprising", "consisting" and variations thereof encompass the terms "consisting of" and "consisting essentially of". It is to be understood that the terms "including", "comprising", "consisting" and variations thereof are not intended to exclude other integers or additional integers not specifically recited. It is to be understood that the terms "including", "comprising", "consisting" and variations thereof are not to be construed as limiting the scope of the claimed concept to the complete reading of the claims. It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting; it is not intended to exclude myriad other embodiments of the present application that other inventors can develop based on the same general inventive concepts embodied by the described embodiments. That is, although the present application is described in terms of particular embodiments and implementations, it is to be understood that the terminology used is for the purpose of descriptive clarity and that it is intended to be limited only by the words recited in the appended claims.

[0033] While the embodiments of the application have been shown and described herein, it is to be understood that the application is not limited to the embodiments described, and it is not intended to exclude myriad other embodiments of the present application that other inventors can develop based on the same general inventive concepts embodied by the described embodiments. That is, although the present application is described in terms of particular embodiments and implementations, it is to be understood that the terminology used is for the purpose of descriptive clarity and that it is intended to be limited only by the words recited in the appended claims.

Claims

1.A method for evaluating semen quality based on deep learning, characterized by, The method comprises the following steps: S1, preprocessing the data set, selecting the data of 85 male participants aged 18 or above, the data including the video of each participant's live sperm sample, 60% of the data set is used for training, 20% is used as a validation set, and 20% is used as a test set, the video of the training set is preprocessed, and 9 continuous frames are intercepted as a sample input; S2, training the improved convolutional neural network, using a lightweight convolutional network MobileNet as the backbone network, adding the levels of several fatty acids in sperm and the serum levels of phospholipid fatty acids in the last fully connected layer, then using the improved lightweight convolutional network to extract information from the input continuous image frames, calculating the loss function by forward propagation, and adjusting the parameters of the convolutional neural network by using the gradient descent method for backward propagation, so that the result of the loss function decreases, when the calculation results of the loss function on the training set and the validation set tend to be stable, the learning rate is fine-tuned, and the convolutional network is continuously trained, the loss function is calculated by forward propagation, and the parameters of the convolutional neural network are adjusted by using the gradient descent method for backward propagation, so that the result of the loss function decreases, until the loss of the training set and the validation set tends to be stable, the test set is used to test the result of network training, and evaluation indexes for evaluating the convolutional neural network are given; S3, sample inference, including front-end design and back-end design, the front-end design is two web pages, which are a video loading interface and a result display interface, the back-end design is to package the trained deep learning network and the trained weight parameters into the back-end framework, load any semen sample video, call the trained convolutional neural network, predict the motility index of the semen and the morphology index of the sperm, and pass them to the front-end page. 2.The deep learning-based semen quality evaluation method of claim 1, wherein, In step S1, the data set further includes the levels of several fatty acids in the sperm of the participants, the serum levels of phospholipid fatty acids, the measured sex hormone serum levels in the blood, the age, abstinence time and body mass index of the participants. 3.The deep learning-based semen quality evaluation method of claim 1, wherein, In step S1, the data set contains more than 35GB of video, each video lasts for two to seven minutes, the frame rate is 50 frames per second, and the data set is made and arranged by a professional medical institution, which is valid. 4.The deep learning-based semen quality evaluation method of claim 1, wherein, In the training process, the parameter settings are as follows: the video images input into the training set are preprocessed and adjusted to 224x224x3 size, 9 continuous frames of images are taken as the input of the improved lightweight network, then the input sperm images are randomly inverted and cropped to 16x112x112 to realize spatial and temporal jitter, Adam is selected as the optimizer, the initial learning rate is 0.0001, the learning rate is divided by 2 when the loss function result no longer decreases, and the pre-training is performed on the UCF101 classic action classification data set. 5.The deep learning-based semen quality evaluation method of claim 1, wherein, The convolutional neural network parameter setting is that the whole network has 28 layers, first a 3x3 standard convolution, then a deep convolution module is stacked, the deep convolution module has two kinds, bottleneck_1 is a deep convolution module with a step of 1, which does not change the size of the feature map, bottleneck_2 is a deep convolution module with a step of 2, which down-samples the input feature map, reduces the length and width of the input feature map, the network is stacked by multiple deep separable convolution modules, extracts the time and space information of the video image, then an average convolution layer is used to change the feature map into 1x1, then a fully connected layer is connected, and the medical health information other than the video information: the level of fatty acid, the serum level of phospholipid fatty acid, the serum level of blood sex hormone, the age of the sample, the abstinence time and the body mass index are normalized, stacked in the fully connected layer, and finally the prediction result is output by using the softmax layer.

Images