Sperm motility analysis method and device, electronic equipment and readable storage medium

Abstract

The invention discloses a sperm motility analysis method and device, electronic equipment and a readable storage medium, and relates to the technical field of sperm detection. The method comprises the steps of obtaining a target video; a plurality of video clips are obtained from a target video, each video clip comprises Q image blocks, the Q image blocks contained in the same video clip belong to Q video frames which are continuous in a time domain in the target video, and the positions of the Q image blocks contained in the same video clip in the video frames which the Q image blocks belong to are the same, the positions of the image blocks contained in the different video clips in the video frames to which the image blocks belong are different, or the video frames to which the image blocks contained in the different video clips belong are different; respectively inputting the plurality of video clips into a sperm motility evaluation model to obtain a motility prediction value of each video clip; and determining the mean value of the activity predicted values of the plurality of video clips as the sperm activity value. According to the technical scheme disclosed by the invention, the sperm motility can be objectively, quickly and accurately analyzed.

Description

Technical Field

[0001] The present application belongs to the technical field of sperm detection, and in particular to a sperm motility analysis method, device, electronic device and readable storage medium. Background Art

[0002] With the rapid development of modern animal husbandry, artificial insemination technology is increasingly used in breeding and large-scale farming. In the process of artificial insemination, sperm motility is one of the key indicators for evaluating semen quality, which directly affects the success rate of insemination and reproductive efficiency. Sperm motility not only reflects the motility of sperm, but is also an important indicator of sperm fertilization ability, which has important guiding significance for the selection and breeding of male animals (e.g., boars) and breeding plans.

[0003] In actual production, rapid and accurate evaluation of sperm motility has a significant impact on the efficiency and economic benefits of the breeding industry. First, timely grasp of semen quality can ensure the best time for artificial insemination and improve conception rate; second, accurate evaluation of the reproductive capacity of male animals helps to optimize breeding programs; third, accurate motility testing helps to determine the semen dilution multiple and optimize the preservation program, thereby improving semen utilization efficiency.

[0004] Traditional methods for detecting semen motility mainly include: the first option, in which the tester uses a microscope to analyze sperm motility. This option is simple to operate, but is easily affected by the tester's experience and subjective factors, and the repeatability and comparability of the test results are poor; the second option is to use a computer assistant semen analysis system (CASA) to analyze sperm motility. Although this option can provide objective data, the equipment is expensive, the operation is complicated, and the detection process is time-consuming. Summary of the invention

[0005] In view of the above problems, the present application provides a sperm motility analysis method, device, electronic device and readable storage medium, which can achieve objective, rapid and accurate analysis of sperm motility without the need to configure a computer-assisted semen analysis system.

[0006] To achieve the above objectives, this application provides the following technical solutions:

[0007] The first aspect of the present application provides a method for analyzing sperm motility, comprising:

[0008] Obtaining a target video, wherein the target video is obtained by collecting images of a glass slide sample formed by the semen to be tested;

[0009] Obtaining multiple video clips from the target video, wherein each video clip includes Q image blocks, the Q image blocks contained in the same video clip belong to Q video frames that are continuous in the time domain in the target video, the Q image blocks contained in the same video clip have the same position in the video frames to which they belong, and the image blocks contained in different video clips have different positions in the video frames to which they belong, or the image blocks contained in different video clips belong to different video frames;

[0010] Inputting the plurality of video clips into a pre-trained sperm motility assessment model respectively to obtain a motility prediction value for each video clip;

[0011] An average of the vitality prediction values ​​of the multiple video clips is determined, and the average is determined as the vitality value of the sperm.

[0012] In a possible implementation, the training process of the sperm motility assessment model includes:

[0013] Obtain multiple sample videos;

[0014] Extracting P sample video frame sequences from each sample video, each sample video frame sequence includes Q sample video frames that are continuous in the time domain, where P is an integer not less than 1, and Q is an integer not less than 2;

[0015] Extracting M training samples from each sample video frame sequence, each training sample includes Q image blocks, wherein each sample video frame sequence is divided into M image blocks of the same size, the Q image blocks contained in the same training sample belong to Q sample video frames, the Q image blocks contained in the same training sample have the same position in the sample video frames to which they belong, the image blocks contained in different training samples have different positions in the video frames to which they belong, or the image blocks contained in different training samples belong to different sample video frames, M is an integer not less than 2, and each training sample is labeled with a vitality score value;

[0016] Inputting the training sample into a deep learning model to obtain a vitality prediction value output by the deep learning model;

[0017] The parameters of the deep learning model are optimized based on the vitality prediction value and vitality score value of the training sample until the preset training end condition is met, and the current deep learning model is determined as the sperm vitality evaluation model.

[0018] In a possible implementation, when P is greater than 1, there are no repeated sample video frames between the P sample video frame sequences.

[0019] In a possible implementation, the process of generating the sample video includes:

[0020] Obtaining a semen sample;

[0021] preparing a slide sample using the semen sample;

[0022] The image acquisition device of the microscope is used to acquire an image of the glass slide sample to obtain a sample video.

[0023] In a possible implementation, the image acquisition device using a microscope acquires an image of the glass slide sample to obtain a sample video, including:

[0024] The field of view of the microscope is adjusted to different areas of the glass slide sample in turn, and an image acquisition device is used to acquire images of multiple areas of the glass slide sample to generate multiple sample videos.

[0025] In a possible implementation, the vitality score value of the training sample is determined according to the following scheme:

[0026] For each sample video, obtain scores of multiple experts for the sample video, and determine the average of the multiple scores as the comprehensive score of the sample video;

[0027] The vitality score values ​​of multiple training samples extracted from the sample video are set as the comprehensive score of the sample video.

[0028] The second aspect of the present application provides a sperm motility analysis device, comprising:

[0029] A target video acquisition module is used to obtain a target video, wherein the target video is obtained by collecting images of a glass slide sample formed by the semen to be tested;

[0030] A video segment extraction module, used to obtain multiple video segments from the target video, wherein each video segment includes Q image blocks, the Q image blocks contained in the same video segment belong to Q video frames that are continuous in the time domain in the target video, the Q image blocks contained in the same video segment have the same position in the video frames to which they belong, the image blocks contained in different video segments have different positions in the video frames to which they belong, or the image blocks contained in different video segments belong to different video frames;

[0031] A video clip processing module, used for inputting the plurality of video clips into a pre-trained sperm motility assessment model to obtain a motility prediction value for each video clip;

[0032] The sperm motility analysis module is used to determine the average of the motility prediction values ​​of the multiple video clips, and determine the average as the sperm motility value.

[0033] In a possible implementation, a training module is also included, and the training module includes:

[0034] A sample video acquisition submodule is used to obtain multiple sample videos;

[0035] A video frame sequence extraction submodule is used to extract P sample video frame sequences from each sample video, each sample video frame sequence includes Q sample video frames that are continuous in the time domain, P is an integer not less than 1, and Q is an integer not less than 2;

[0036] A training sample extraction submodule is used to extract M training samples from each sample video frame sequence, each training sample includes Q image blocks, wherein each sample video frame sequence is divided into M image blocks of the same size, the Q image blocks contained in the same training sample belong to Q sample video frames, the Q image blocks contained in the same training sample have the same position in the sample video frame to which they belong, the image blocks contained in different training samples have different positions in the video frame to which they belong, or the image blocks contained in different training samples belong to different sample video frames, M is an integer not less than 2, and each training sample is labeled with a vitality score value;

[0037] A training sample processing submodule, used to input the training sample into a deep learning model to obtain a vitality prediction value output by the deep learning model;

[0038] The parameter adjustment submodule is used to optimize the parameters of the deep learning model based on the vitality prediction value and vitality score value of the training sample until the preset training end condition is met, and the current deep learning model is determined as the sperm vitality evaluation model.

[0039] A third aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0040] The memory is used to store computer programs;

[0041] The processor is used to execute the computer program so that the electronic device implements any one of the above sample point identification methods.

[0042] A fourth aspect of the present application provides a readable storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device implements any one of the above-mentioned sperm motility analysis methods.

[0043] A fifth aspect of the present application provides a computer program product, comprising computer-readable instructions, which, when executed on an electronic device, enables the electronic device to implement any one of the above-mentioned sperm motility analysis methods.

[0044] It can be seen that the beneficial effects of this application are:

[0045] The technical solution disclosed in the present application obtains a target video, that is, a video obtained by collecting images of a glass slide sample formed by the semen to be tested, and obtains multiple video clips from the target video, each video clip includes Q image blocks, and the Q image blocks contained in the same video clip belong to Q video frames that are continuous in the time domain in the target video, and the Q image blocks contained in the same video clip have the same position in the video frame to which they belong, and the image blocks contained in different video clips have different positions in the video frame to which they belong, or the image blocks contained in different video clips belong to different video frames, and each video clip is processed using a pre-trained sperm motility assessment model to obtain a predicted motility value of each video clip, and the average of the multiple predicted motility values ​​is determined as the motility value of the sperm in the semen to be tested. Based on the technical solution disclosed in the present application, an objective and accurate analysis of sperm motility is achieved without configuring a computer-assisted semen analysis system. Moreover, in this solution, the video clips are input into the sperm motility assessment model, and then the motility value of the sperm in the semen to be tested is determined based on the motility prediction values ​​of multiple video clips. This can significantly reduce the computational complexity and complete the sperm motility analysis in a shorter time, which enables this solution to meet the needs of large-scale detection scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 A flowchart of the training method of the sperm motility assessment model disclosed in this application;

[0048] Figure 2 A schematic diagram of the principle of extracting training samples from a sample video disclosed in this application;

[0049] Figure 3 A schematic diagram of the training samples in this application;

[0050] Figure 4 This is a schematic diagram of the structure of the sperm motility assessment model disclosed in this application;

[0051] Figure 5 This is a flow chart of a sperm motility analysis method disclosed in this application;

[0052] Figure 6 This is a schematic structural diagram of a sperm motility analysis device disclosed in this application.

[0053] Figure 7This is a schematic diagram of the structure of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0054] The present application discloses a sperm motility analysis method, device, electronic device and readable storage medium, which can achieve objective, rapid and accurate analysis of sperm motility without configuring a computer-assisted semen analysis system.

[0055] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0056] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable in appropriate circumstances, which is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only include those elements, but also include other elements that are not clearly listed, or also include elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements limited by the statement "include one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements.

[0057] The following is an explanation of the training process of the sperm motility assessment model used in the sperm motility analysis process of this application.

[0058] See also Figure 1 , Figure 1 The flowchart of the training method of the sperm motility assessment model disclosed in the present application. The training method comprises the following steps:

[0059] S101: Obtain multiple sample videos.

[0060] The process of generating a sample video is described below.

[0061] 1) Obtain a semen sample.

[0062] As an alternative, a fresh semen sample can be obtained using conventional sperm collection.

[0063] As another optional solution, pre-stored semen is obtained, usually stored in a constant temperature box, and the semen is warmed up to wake up the sperm, and the warmed semen is used as a semen sample. For example, a semen sample is collected in advance and stored in a constant temperature box at a preset temperature (e.g., 17°C), and then the semen is taken out of the constant temperature box and placed in an environment with a preset temperature range (e.g., between 37.5°C and 38.5°C) to warm up to wake up the sperm.

[0064] 2) Prepare slide samples using the semen samples obtained.

[0065] A predetermined volume (eg, 20 μL) of semen sample is dropped onto a clean glass slide using a pipette or other instruments to form a slide sample.

[0066] 3) Use the image acquisition device of the microscope to acquire images of the slide sample to obtain a sample video.

[0067] The slide sample is left to stand for a preset time (e.g., 15 to 20 seconds), and then placed on the stage of the microscope and fixed to prevent the slide sample from drifting on the stage. Preferably, the temperature of the stage is controlled within the aforementioned temperature range (e.g., between 37.5° C. and 38.5° C.).

[0068] Configure the parameters of the microscope and the parameters of the image acquisition device. For example, use a 10x objective lens and a 100x magnification. Set the resolution of the image acquisition device to 1920×1080, and set the frame rate of the image acquisition device to 30fps. Of course, the above are only examples of the parameters of the microscope and the parameters of the image acquisition device, and are not limited thereto.

[0069] At least one sample video is acquired for each slide sample.

[0070] The number of sample video frames included in the sample video needs to reach a first value, wherein the first value is determined according to the number of image blocks required to be included in each training sample, and the first value is not less than the number of image blocks required to be included in each training sample. Preferably, the first value is not less than twice the number of image blocks required to be included in each training sample.

[0071] For example, it is pre-set that each training sample contains 8 image blocks, so each sample video must contain at least 8 sample video frames. It is understandable that if a sample video contains more sample video frames, more training samples can be generated based on the sample video. In other words, more training samples can be generated when the number of slide samples remains unchanged.

[0072] Optionally, multiple sample videos are acquired for each slide sample.

[0073] Specifically, the field of view of the microscope is adjusted to different areas of the slide sample in turn, and the image acquisition device is used to acquire images of multiple areas of the slide sample to generate multiple sample videos. Based on this solution, more training samples can be generated while the number of slide samples remains unchanged. Moreover, the model can learn the vitality characteristics of sperm under different fields of view.

[0074] For example, first adjust the field of view of the microscope to the first area of ​​the glass slide sample, and use the image acquisition device to capture the image of the first area of ​​the glass slide sample to generate a sample video. Then, adjust the field of view of the microscope to the second area of ​​the glass slide sample, and use the image acquisition device to capture the image of the second area of ​​the glass slide sample to generate a sample video. And so on, generate a sample video for each of the multiple areas of the glass slide sample.

[0075] As an optional solution, there is no overlap between the multiple regions of the slide sample. As another optional solution, there can be overlap between the multiple regions of the slide sample, but the size of the overlapping region between any two regions cannot exceed a preset limit.

[0076] S102: Extract P sample video frame sequences from each sample video.

[0077] Each sample video frame sequence includes Q sample video frames that are consecutive in the time domain, where P is an integer not less than 1, and Q is an integer not less than 2.

[0078] It can be understood that the number Q of sample video frames contained in each sample video frame sequence is the same as the number of image blocks required to be contained in each training sample set in advance. Therefore, the number of sample video frame sequences that can be extracted from each sample video is determined by the number of sample video frames contained in the sample video and the number of image blocks required to be contained in each training sample set in advance.

[0079] As an optional solution, when P is an integer greater than 1, the P sample video frame sequences extracted from a sample video do not contain repeated sample video frames. That is to say, any sample video frame in the sample video belongs to only one sample video frame sequence. In this solution, the number of sample video frame sequences that can be extracted from each sample video is the result of rounding down the number of sample video frames contained in the sample video and the number Q of sample video frames contained in each sample video frame sequence.

[0080] S103: Extract M training samples from each sample video frame sequence.

[0081] Each training sample includes Q image blocks, wherein each sample video frame is divided into M image blocks of the same size, the Q image blocks contained in the same training sample belong to Q sample video frames, the Q image blocks contained in the same training sample have the same position in the sample video frames to which they belong, and the image blocks contained in different training samples have different positions in the video frames to which they belong, or the image blocks contained in different training samples belong to different sample video frames, and M is an integer not less than 2. In addition, each training sample is annotated with a vitality score value.

[0082] To facilitate understanding of the scheme for generating training samples based on sample videos, an example is provided below.

[0083] It is preset that each training sample contains 8 image blocks that are continuous in the time domain. The resolution of the image acquisition device is 1920×1080, that is, each sample video frame in the sample video generated by the image acquisition device contains 1920 pixels in the horizontal direction and 1080 pixels in the vertical direction.

[0084] Assume that a sample video generated by the image acquisition device contains 17 sample video frames. The 17 sample video frames are recorded as sample video frame 1 to sample video frame 17 in order of generation time.

[0085] Here, a scheme is adopted in which P sample video frame sequences are extracted from a sample video, and the scheme does not contain repeated sample video frames. Then, 2 sample video frame sequences can be extracted from the above sample video. For example, the 2 sample video frame sequences extracted from the above sample video frames are specifically: the first sample video frame sequence includes sample video frame 1 to sample video frame 8, and the second sample video frame sequence includes sample video frame 9 to sample video frame 16. Of course, this is only a scheme, as long as the two sample video frame sequences each contain 8 sample video frames that are continuous in time sequence, and the two sample video frame sequences do not contain repeated sample video frames.

[0086] Extract M training samples from each sample video frame sequence. Specifically:

[0087] Each sample video frame in the sample video frame sequence is divided into M image blocks of the same size. For example, a sample video frame of 1920×1080 is divided into 12 image blocks of 480×360, each training sample includes 8 image blocks of 480×360, the 8 image blocks contained in the same training sample belong to 8 sample video frames, the 8 image blocks contained in the same training sample have the same position in the sample video frames to which they belong, and the image blocks contained in different training samples have different positions in the video frames to which they belong, or the image blocks contained in different training samples belong to different sample video frames.

[0088] See also Figure 2 , Figure 2 This is a schematic diagram of the principle of extracting training samples from a sample video disclosed in this application.

[0089] Taking the first sample video frame sequence in the above text as an example, the sample video frames it contains are sample video frame 1 to sample video frame 8. Each sample video frame of size 1920×1080 is divided into 12 image blocks of size 480×360. According to the positions of the 12 image blocks in the sample video frame, the 4 image blocks in the first row are sequentially referred to as the first image block to the fourth image block, the 4 image blocks in the second row are sequentially referred to as the fifth image block to the eighth image block, and the 4 image blocks in the third row are sequentially referred to as the ninth image block to the twelfth image block. The first image block in sample video frame 1 to sample video frame 8 constitutes a training sample, the second image block in sample video frame 1 to sample video frame 8 constitutes a training sample, and so on. A total of 12 training samples are extracted from the first sample video frame sequence.

[0090] like Figure 3 As shown, Figure 3 This is an example of a training sample in this application. The training sample includes 8 image blocks, which belong to 8 sample video frames that are continuous in the time domain, and the positions of the 8 image blocks in the sample video frames to which they belong are the same. Figure 3 In , N is the number of sample video frames contained in the sample video. Figure 3 In the figure, the black areas are formed by sperm, and the other areas are formed by other components of semen.

[0091] The scheme for labeling training samples is described below.

[0092] 1) For each sample video, obtain scores from multiple experts for the sample video, and determine the average of the multiple scores as the comprehensive score of the sample video.

[0093] 2) The vitality score values ​​of multiple training samples extracted from the sample video are determined as the comprehensive score of the sample video.

[0094] The experts score the sample videos based on preset indicators (e.g., the percentage of sperm in progressive motion, or other indicators). Optionally, the score is between 0 and 1.

[0095] In this application, the training samples include multiple image blocks that are continuous in the time domain, which ensures that the model can effectively capture the characteristics of sperm movement state. By setting the number of image blocks included in the training samples, the model can effectively capture the characteristics of sperm movement state and reduce the computational complexity.

[0096] In addition, it should be noted that in this application, instead of directly using multiple temporally continuous sample video frames in the sample video as training samples, local images (i.e., image blocks) in multiple temporally continuous sample video frames are used as training samples. This processing has the following advantages: First, the image blocks obtained by segmenting the sample video frames contain enough sperm images, which can significantly reduce the computational complexity while ensuring the prediction accuracy of the model. For example, the amount of computation required to process a 1920×1080 image is greater than the amount of computation required to process 12 480×360 images; second, multiple training samples can be extracted from a sample video frame sequence, thereby achieving an exponential increase in training samples and improving the generalization ability of the model.

[0097] S104: Input the training sample into the deep learning model to obtain the vitality prediction value output by the deep learning model.

[0098] S105: Optimize the parameters of the deep learning model based on the vitality prediction value and vitality score value of the training sample until the preset training end condition is met, and determine the current deep learning model as the sperm vitality evaluation model.

[0099] As an optional solution, the preset training end condition is: the value of the preset loss function is less than a preset threshold value. The loss function represents the deviation between the vitality prediction value of the training sample or the verification sample and the corresponding vitality score value.

[0100] As another optional solution, the preset training end condition is: the number of iterations reaches a preset number.

[0101] As an optional solution, the sperm motility assessment model adopts a convolutional neural network. Optionally, the convolution kernel in the convolutional neural network adopts a (2+1)D convolution kernel.

[0102] Figure 4 This is a schematic diagram of the structure of the sperm motility assessment model disclosed in the present application, including: an input layer, multiple convolutional layers, a pooling layer, a fully connected layer and an output layer. Among them, each convolutional layer adopts a (2+1)D convolution kernel. It should also be noted that the current R(2+1)D convolutional neural network is widely used in video classification tasks. In the present application, the R(2+1)D convolutional neural network is used to perform regression tasks, so it is necessary to improve the R(2+1)D convolutional neural network. Specifically: the original L×C dimensional fully connected layer (where L is the feature dimension and C is the number of categories) is modified to the L×1 fully connected layer with a Sigmoid activation function. It should be noted that in Figure 4 The number of convolutional layers is just an illustration.

[0103] In the implementation, the training set and validation set are pre-constructed, the training set is used to optimize the parameters of the model, and the validation set is used to adjust the hyperparameters of the model and evaluate the performance of the model.

[0104] The applicant's evaluation results of the sperm motility evaluation model are shown in Table 1.

[0105] From the evaluation results shown in Table 1, we can see that the average error of the six batches is between 0.0316 and 0.0445, the processing time of a single video is between 0.58 seconds and 1.81 seconds, and the test results across devices and scenarios are stable. The sperm motility evaluation model in this application has high accuracy, processing efficiency and stability.

[0106] Table 1

[0107]

[0108] The training method of the sperm motility assessment model disclosed above in the present application extracts P sample video frame sequences from a sample video, and then extracts M training samples from a sample video frame sequence, each training sample includes Q image blocks, the Q image blocks contained in the same training sample belong to Q sample video frames, the Q image blocks contained in the same training sample have the same position in the sample video frames to which they belong, the image blocks contained in different training samples have different positions in the video frames to which they belong, or the image blocks contained in different training samples belong to different sample video frames, and then the deep learning model is trained using the training model until the preset training end conditions are met, and the current deep learning model is determined as the sperm motility assessment model. In the above training method, the training samples are contained in multiple image blocks that are continuous in the time domain, which ensures that the model can effectively capture the characteristics of sperm movement state; in addition, using local images (that is, image blocks) in multiple temporally continuous sample video frames as training samples has the following advantages: First, the image blocks obtained by segmenting the sample video frames contain enough sperm images, which can significantly reduce the computational complexity while ensuring the prediction accuracy of the model; second, multiple training samples can be extracted from a sample video frame sequence, thereby achieving an exponential increase in training samples and improving the generalization ability of the model.

[0109] The sperm motility analysis method is described below.

[0110] See also Figure 5 , Figure 5 This is a flow chart of a sperm motility analysis method disclosed in this application. The method comprises:

[0111] S501: Obtain a target video.

[0112] The target video is obtained by collecting images of a glass slide sample formed by the semen to be tested.

[0113] It should be noted that the process of generating the target video for the semen to be tested is similar to the process of generating the sample video in the previous article. Here is a brief description:

[0114] 1) Obtain the semen to be tested.

[0115] Usually, the semen to be tested is fresh semen. Of course, the semen to be tested may also be pre-stored semen, such as semen pre-stored in a constant temperature box. In the case where the semen to be tested is pre-stored semen, the semen is subjected to a temperature-recovery treatment to wake up the sperm, and the semen after the temperature-recovery treatment is used as the semen to be tested. For the temperature-recovery treatment of the semen, please refer to the description in the previous text.

[0116] 2) Generate a slide sample using the semen to be tested.

[0117] A predetermined volume (eg, 20 μL) of semen to be tested is dropped onto a clean glass slide using a pipette or other instrument to form a slide sample.

[0118] 3) Use the image acquisition device of the microscope to acquire images of the glass slide sample to obtain the target video.

[0119] The slide sample is left to stand for a preset time (e.g., 15 to 20 seconds), and then placed on the stage of the microscope and fixed to prevent the slide sample from drifting on the stage. Preferably, the temperature of the stage is controlled within the aforementioned temperature range (e.g., between 37.5° C. and 38.5° C.).

[0120] Configure the parameters of the microscope and the image acquisition device. For example, use a 10x objective lens and a 100x magnification. Set the resolution of the image acquisition device to 1920×1080 and the frame rate of the image acquisition device to 30fps. Use the image acquisition device to acquire images of the preset area of ​​the slide sample and generate the target image.

[0121] S502: Obtain multiple video clips from the target video.

[0122] Each video clip includes Q image blocks, the Q image blocks contained in the same video clip belong to Q video frames that are continuous in the time domain in the target video, the Q image blocks contained in the same video clip have the same position in the video frames to which they belong, and the image blocks contained in different video clips have different positions in the video frames to which they belong, or the image blocks contained in different video clips belong to different video frames.

[0123] S503: Inputting the plurality of video clips into the pre-trained sperm motility assessment model respectively to obtain a motility prediction value of each video clip.

[0124] S504: Determine an average of the vitality prediction values ​​of the multiple video clips, and determine the average as the vitality value of the sperm.

[0125] It can be understood that in this scheme, the motility value of the sperm in the semen to be tested is determined based on the motility prediction values ​​of multiple video clips. The more video clips there are, the more accurate the sperm motility value will be. However, this will also bring more calculations. The accuracy of the results and the amount of calculations can be balanced by setting the number of video clips.

[0126] The sperm motility analysis method disclosed in the present application obtains a target video, that is, a video obtained by collecting images of a glass slide sample formed by the semen to be tested, and obtains multiple video clips from the target video, each video clip includes Q image blocks, the Q image blocks contained in the same video clip belong to Q video frames that are continuous in the time domain in the target video, the Q image blocks contained in the same video clip have the same position in the video frame to which they belong, and the image blocks contained in different video clips have different positions in the video frame to which they belong, or the image blocks contained in different video clips belong to different video frames, and each video clip is processed using a pre-trained sperm motility evaluation model to obtain a predicted motility value of each video clip, and the average of the multiple predicted motility values ​​is determined as the motility value of the sperm in the semen to be tested. Based on the scheme disclosed in the present application, an objective and accurate analysis of sperm motility is achieved without configuring a computer-assisted semen analysis system. Moreover, in this solution, the video clips are input into the sperm motility assessment model, and then the motility value of the sperm in the semen to be tested is determined based on the motility prediction values ​​of multiple video clips. This can significantly reduce the computational complexity and complete the sperm motility analysis in a shorter time, which enables this solution to meet the needs of large-scale detection scenarios.

[0127] The present application discloses a sperm motility analysis method as mentioned above, and correspondingly, the present application also discloses a sperm motility analysis device, and the descriptions of the two in the specification can be referenced to each other.

[0128] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of a sperm motility analysis device disclosed in this application. The device includes:

[0129] A target video acquisition module 601 is used to obtain a target video, where the target video is obtained by collecting images of a glass slide sample formed by the semen to be tested;

[0130] The video segment extraction module 602 is used to obtain multiple video segments from the target video, wherein each video segment includes Q image blocks, the Q image blocks contained in the same video segment belong to Q video frames that are continuous in the time domain in the target video, the Q image blocks contained in the same video segment have the same position in the video frames to which they belong, and the image blocks contained in different video segments have different positions in the video frames to which they belong, or the image blocks contained in different video segments belong to different video frames;

[0131] The video clip processing module 603 is used to input the multiple video clips into the pre-trained sperm motility assessment model to obtain the motility prediction value of each video clip;

[0132] The sperm motility analysis module 604 is used to determine the average of the motility prediction values ​​of the multiple video clips, and determine the average as the sperm motility value.

[0133] Optional, in Figure 6 Based on the sperm motility analysis device shown, a training module is further provided. The training module includes:

[0134] A sample video acquisition submodule is used to obtain multiple sample videos;

[0135] A video frame sequence extraction submodule is used to extract P sample video frame sequences from each sample video, each sample video frame sequence includes Q sample video frames that are continuous in the time domain, P is an integer not less than 1, and Q is an integer not less than 2;

[0136] A training sample extraction submodule is used to extract M training samples from each sample video frame sequence, each training sample includes Q image blocks, wherein each sample video frame sequence is divided into M image blocks of the same size, the Q image blocks contained in the same training sample belong to Q sample video frames, the Q image blocks contained in the same training sample have the same position in the sample video frame to which they belong, the image blocks contained in different training samples have different positions in the video frame to which they belong, or the image blocks contained in different training samples belong to different sample video frames, M is an integer not less than 2, and each training sample is labeled with a vitality score value;

[0137] A training sample processing submodule is used to input the training sample into the deep learning model to obtain the vitality prediction value output by the deep learning model;

[0138] The parameter adjustment submodule is used to optimize the parameters of the deep learning model based on the vitality prediction value and vitality score value of the training sample until the preset training end conditions are met, and the current deep learning model is determined as the sperm vitality evaluation model.

[0139] It should be noted that the specific schemes of each sub-module in the training module can be found in the description above.

[0140] The application also discloses an electronic device.

[0141] See also Figure 7 , Figure 7 The hardware structure of the electronic device is shown. The electronic device includes: a processor 701 , a memory 702 , a communication interface 703 , and a communication bus 704 .

[0142] In the embodiment of the present application, the number of the processor 701, the memory 702, the communication interface 703, and the communication bus 704 is at least one, and the processor 701, the memory 702, and the communication interface 703 communicate with each other through the communication bus 704. The communication bus 704 can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0143] It should be noted that those skilled in the art can understand that Figure 7 The structure of the electronic device shown in the figure does not constitute a limitation on the electronic device, and the electronic device may include Figure 7 More or fewer components may be shown, or certain components may be combined, or the components may be arranged differently.

[0144] Combine the following Figure 7 Provide a detailed introduction to the various components of electronic equipment.

[0145] The processor 701 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 702 and calling data stored in the memory 702, thereby monitoring the electronic device as a whole.

[0146] The processor 701 may be a central processing unit (CPU), or an application specific integrated circuit ASIC (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, etc.;

[0147] The memory 702 may include a memory, such as a high-speed random-access memory (RAM) and a read-only memory (ROM), and may also include a large-capacity storage device, such as at least one disk storage.

[0148] The memory 702 stores a computer program, and the processor 701 is used to execute the computer program stored in the memory 701, so that the electronic device can implement any sperm motility analysis method disclosed in the embodiments of the present application.

[0149] The present application also discloses a readable storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device implements any sperm motility analysis method disclosed in the embodiments of the present application.

[0150] The present application also discloses a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any sperm motility analysis method provided in the embodiments of the present application.

[0151] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. The technical features in each embodiment can be arranged and combined to form a new embodiment. For the sperm motility analysis device, electronic device, readable storage medium and computer program product disclosed in the embodiment, since they correspond to the sperm motility analysis method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0152] In addition, it should be noted that the device embodiments described above are merely schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, and may be located in one place or distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0153] Through the description of the above implementation methods, the technicians in the relevant field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation method in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device to execute the methods described in each embodiment of the present application.

[0154] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0155] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A method for analyzing sperm motility, characterized in that: include: Obtaining a target video, wherein the target video is obtained by collecting images of a glass slide sample formed by the semen to be tested; Obtaining multiple video clips from the target video, wherein each video clip includes Q image blocks, the Q image blocks contained in the same video clip belong to Q video frames that are continuous in the time domain in the target video, the Q image blocks contained in the same video clip have the same position in the video frames to which they belong, and the image blocks contained in different video clips have different positions in the video frames to which they belong, or the image blocks contained in different video clips belong to different video frames, and Q is an integer not less than 2; Inputting the plurality of video clips into a pre-trained sperm motility assessment model respectively to obtain a motility prediction value for each video clip; An average of the vitality prediction values ​​of the multiple video clips is determined, and the average is determined as the vitality value of the sperm.

2. The method for analyzing sperm motility according to claim 1, characterized in that: The training process of the sperm motility assessment model includes: Obtain multiple sample videos; Extracting P sample video frame sequences from each sample video, each sample video frame sequence includes Q sample video frames that are continuous in the time domain, and P is an integer not less than 1; Extracting M training samples from each sample video frame sequence, each training sample includes Q image blocks, wherein each sample video frame sequence is divided into M image blocks of the same size, the Q image blocks contained in the same training sample belong to Q sample video frames, the Q image blocks contained in the same training sample have the same position in the sample video frames to which they belong, the image blocks contained in different training samples have different positions in the video frames to which they belong, or the image blocks contained in different training samples belong to different sample video frames, M is an integer not less than 2, and each training sample is labeled with a vitality score value; Inputting the training sample into a deep learning model to obtain a vitality prediction value output by the deep learning model; The parameters of the deep learning model are optimized based on the vitality prediction value and vitality score value of the training sample until the preset training end condition is met, and the current deep learning model is determined as the sperm vitality evaluation model.

3. The method for analyzing sperm motility according to claim 2, characterized in that: When P is greater than 1, there are no repeated sample video frames between the P sample video frame sequences.

4. The method for analyzing sperm motility according to claim 2, characterized in that: The generation process of the sample video includes: Obtaining a semen sample; preparing a slide sample using the semen sample; The image acquisition device of the microscope is used to acquire an image of the glass slide sample to obtain a sample video.

5. The method for analyzing sperm motility according to claim 4, characterized in that: The image acquisition device of the microscope is used to acquire an image of the glass slide sample to obtain a sample video, including: The field of view of the microscope is adjusted to different areas of the glass slide sample in sequence, and an image acquisition device is used to acquire images of multiple areas of the glass slide sample to generate multiple sample videos.

6. The method for analyzing sperm motility according to claim 2, characterized in that: The vitality score value of the training sample is determined according to the following scheme: For each sample video, obtain scores of multiple experts for the sample video, and determine the average of the multiple scores as the comprehensive score of the sample video; The vitality score values ​​of multiple training samples extracted from the sample video are set as the comprehensive score of the sample video.

7. A sperm motility analysis device, characterized in that: include: A target video acquisition module is used to obtain a target video, wherein the target video is obtained by collecting images of a glass slide sample formed by the semen to be tested; A video segment extraction module, used for obtaining multiple video segments from the target video, wherein each video segment includes Q image blocks, the Q image blocks contained in the same video segment belong to Q video frames that are continuous in the time domain in the target video, the Q image blocks contained in the same video segment have the same position in the video frames to which they belong, and the image blocks contained in different video segments have different positions in the video frames to which they belong, or the image blocks contained in different video segments belong to different video frames; A video clip processing module, used for inputting the plurality of video clips into a pre-trained sperm motility assessment model to obtain a motility prediction value for each video clip; The sperm motility analysis module is used to determine the average of the motility prediction values ​​of the multiple video clips, and determine the average as the sperm motility value.

8. The sperm motility analysis device according to claim 7, characterized in that: Also included is a training module, the training module comprising: A sample video acquisition submodule is used to obtain multiple sample videos; A video frame sequence extraction submodule is used to extract P sample video frame sequences from each sample video, each sample video frame sequence includes Q sample video frames that are continuous in the time domain, P is an integer not less than 1, and Q is an integer not less than 2; A training sample extraction submodule is used to extract M training samples from each sample video frame sequence, each training sample includes Q image blocks, wherein each sample video frame sequence is divided into M image blocks of the same size, the Q image blocks contained in the same training sample belong to Q sample video frames, the Q image blocks contained in the same training sample have the same position in the sample video frame to which they belong, the image blocks contained in different training samples have different positions in the video frame to which they belong, or the image blocks contained in different training samples belong to different sample video frames, M is an integer not less than 2, and each training sample is labeled with a vitality score value; A training sample processing submodule, used to input the training sample into a deep learning model to obtain a vitality prediction value output by the deep learning model; The parameter adjustment submodule is used to optimize the parameters of the deep learning model based on the vitality prediction value and vitality score value of the training sample until the preset training end condition is met, and the current deep learning model is determined as the sperm vitality evaluation model.

9. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device implements the sperm motility analysis method as described in any one of claims 1 to 6.

10. A readable storage medium, characterized in that: The readable storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device implements the sperm motility analysis method as described in any one of claims 1 to 6.

Images