Sperm flagellum three-dimensional motion reconstruction method based on deep learning

Abstract

The invention relates to a sperm flagellum three-dimensional motion reconstruction method based on deep learning, and the method comprises the steps: obtaining a two-dimensional image of a to-be-processed sperm, carrying out the image preprocessing of the two-dimensional image of the to-be-processed sperm, and obtaining a preprocessed image; inputting the preprocessed image into a trained sperm segmentation model to obtain a sperm flagellum binary image; skeleton processing is carried out on the sperm flagellum binary image, and a sperm flagellum center line, N equidistant points and position coordinates corresponding to each equidistant point are obtained; obtaining N sperm flagellum local images based on the N equidistant points, inputting each sperm flagellum local image into a trained depth prediction model, and obtaining depth information of each sperm flagellum local image; and reconstructing a three-dimensional model of the sperm flagellum based on the depth information of each local image of the sperm flagellum and the position coordinate corresponding to each equidistant point. A reliable solution is provided for research of sperm movement behaviors and development of an assisted reproduction technology.

Description

Technical Field

[0001] The present invention relates to the technical field of reproductive biology, and in particular to a method for reconstructing the three-dimensional motion of sperm flagella based on deep learning. Background Art

[0002] In reproductive biology research, the flagellar movement of sperm is not a simple planar movement, but a complex three-dimensional movement. This three-dimensional movement is crucial for sperm to successfully find the egg cell and complete the fertilization process. The three-dimensional swinging pattern of the flagellum can enhance the propulsion efficiency of sperm in the viscous reproductive tract environment, thereby increasing the probability of fertilization. Understanding and analyzing the three-dimensional movement of sperm flagella is not only of great significance for the diagnosis and treatment of infertility, the optimization of assisted reproductive technology, and basic reproductive biology research, but also has great research value in bionics.

[0003] Clinically, doctors select sperm based mainly on their morphology and dynamics assessment, which usually relies on two-dimensional (2-D) imaging techniques such as bright field microscopy (BFM), differential interference contrast microscopy (DIC), Hoffman modulation contrast microscopy (HMC), and Zernike phase contrast microscopy. However, the structure of sperm flagella is very slender, and sperm used in assisted reproductive technology cannot be stained. In addition, their movement characteristics are three-dimensional and easily out of focus under the microscope, resulting in many challenges in the imaging process. Therefore, the current two-dimensional imaging technology cannot fully capture the movement characteristics of sperm in three-dimensional space. The limitation of this technology has led to an incomplete overall understanding of sperm dynamics, which may affect the accuracy of sperm selection and the success rate of assisted reproductive technology.

[0004] Currently, there are some imaging techniques or equipment that can be used to reconstruct the three-dimensional morphology of samples, such as confocal microscopy and light sheet microscopy. However, these imaging methods have limitations in reconstructing the three-dimensional movement of sperm flagella, mainly because of their limited temporal resolution. The swing frequency of sperm flagella is usually around twenty Hz. In contrast, a confocal microscope may take several seconds to tens of seconds to perform a three-dimensional scan, while the three-dimensional imaging time of a light sheet microscope may be between hundreds of milliseconds and seconds. This makes it impossible for these imaging techniques to capture the high-speed dynamic process of sperm flagella in real time. In recent years, many studies have been devoted to accurately tracking the position of the entire flagellum in time and three-dimensional space. The degree of defocus of the image can be used to infer the position of the object on the z-axis. The basic principle is that when the object is in the focal plane of the microscope, the image is clearest; when the object leaves the focal plane, the image becomes blurred. By measuring and analyzing the blur (i.e., the degree of defocus) of the image, the position of the object on the z-axis can be inferred. Bukatin et al. developed an algorithm to reconstruct flagella perpendicular to the image plane from 2D BFM images by analyzing pixel intensity levels to identify the projected 2D shape of the flagellum and then estimating the z coordinate by analyzing the intensity distribution along the cross section. This method exploits the fact that the width of the halo is related to the z displacement from the focal plane. Although this method is simple and practical, its main disadvantages are that it depends on image quality, its accuracy is easily affected by image noise, and its effect is limited when dealing with complex 3D motion. This method also requires accurate calibration of the relationship between the image and the z position. Jan N. Hansen combined the defocus depth method with MFI (multi-focus imaging) and successfully reconstructed the 3D motion of spherical objects and sperm flagella using a dark-field microscope and a multi-focus adapter. Although this method has advantages in improving spatial resolution, it has high equipment requirements and requires a multi-focus adapter. In addition, this method is very sensitive to noise, and 3D reconstruction from multi-focal plane data requires a lot of calculations, resulting in poor real-time performance. The study by Silva-Villalobos et al. used an oscillating objective mounted on a bright-field optical microscope. They scanned a 16-micron depth of the sample at a speed of 5,000 images per second. They were able to accurately find the z-axis position of the best focus on each part of the flagellum along the flagellum to achieve three-dimensional reconstruction of the sperm flagellum. Although this method performs well in improving temporal resolution, its main disadvantage is that the equipment is complex and costly, making it difficult to promote in practical applications. Although these existing methods have their own advantages in three-dimensional reconstruction of sperm flagellar movement, they generally have limitations in spatiotemporal resolution, computational complexity, equipment cost, and complexity of optical path settings.

[0005] Therefore, there is an urgent need for a method to reconstruct the three-dimensional motion of sperm flagella based on deep learning. Summary of the invention

[0006] 1. Technical issues to be resolved

[0007] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a method for reconstructing the three-dimensional motion of sperm flagella based on deep learning, which solves the technical problems in the prior art of realizing three-dimensional reconstruction of sperm flagella, such as low temporal and spatial resolution, complex calculation, high equipment cost and complex optical path setting.

[0008] (II) Technical solution

[0009] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0010] In a first aspect, an embodiment of the present invention provides a method for reconstructing three-dimensional motion of sperm flagella based on deep learning, comprising:

[0011] S100, obtaining a two-dimensional image of sperm to be processed, and performing image preprocessing on the two-dimensional image of sperm to be processed to obtain a preprocessed image;

[0012] S200, inputting the pre-processed image into a trained sperm segmentation model to obtain a sperm flagella binary image; the sperm segmentation model is a YOLOv8 model;

[0013] S300, skeletonizing the binary image of the sperm flagellum to obtain the center line of the sperm flagellum; based on the center line of the sperm flagellum, obtaining N equidistant points and the position coordinates corresponding to each equidistant point;

[0014] S400, based on the N equidistant points, obtaining N sperm flagella local images, and inputting each sperm flagella local image into a trained depth prediction model to obtain depth information of each sperm flagella local image; the depth prediction model is a convolutional neural network model;

[0015] S500: reconstructing a three-dimensional model of the sperm flagellum based on the depth information of each sperm flagellum local image and the position coordinates corresponding to each equidistant point.

[0016] Optionally, the S100 includes:

[0017] Using a filter to enhance the two-dimensional image of the sperm to be processed to obtain a pre-processed image;

[0018] The two-dimensional image of the sperm to be processed is acquired by bright field microscopy.

[0019] Optionally, in S300,

[0020] The N equidistant points are obtained by intercepting the center line of the sperm flagellum at equidistant intervals of X μm; X is a preset distance.

[0021] Optionally, in S400, based on the N equidistant points, acquiring N sperm flagella local images includes:

[0022] Taking each of the N equidistant points as the center, N sperm flagella partial images are intercepted; each of the N sperm flagella partial images contains 1-2 μm sperm flagella image information.

[0023] Optionally, before S100, the step further includes:

[0024] S000. Use a training data set to train the sperm segmentation model and the depth prediction model to obtain a trained sperm segmentation model and a depth prediction model; the training data set includes: a sperm segmentation model training data set and a depth prediction model training data set.

[0025] Optionally, the sperm segmentation model training set includes: sperm images and bounding box coordinates corresponding to each sperm image;

[0026] The sperm image is obtained through a bright field microscope; the bounding box coordinates corresponding to each sperm image are obtained by annotating the sperm image through Labelbox or LabelMe.

[0027] Optionally, the depth prediction model training data set includes: sperm flagella local images at different focal planes and depth information corresponding to each sperm flagella local image;

[0028] The local images of sperm flagella at different focal planes and the depth information corresponding to each local image of sperm flagella are obtained by photographing sperm flagella images at different focal planes through a piezoelectric platform and intercepting the photographed sperm flagella images; the local images of sperm flagella include 1-2 μm sperm flagella image information. Optionally, S000 specifically includes:

[0029] S010, inputting each data in the sperm segmentation model training set into the corresponding sperm segmentation model, using the target loss function and Adam optimizer to obtain a trained sperm segmentation model;

[0030] The target loss function includes a bounding box regression loss function, a target confidence loss function and a category classification loss function;

[0031] S020, input each data in the depth prediction model training set into the corresponding depth prediction model, use the mean square error loss function and Adam optimizer, with an initial learning rate of K, until M iterations are performed to obtain a trained depth prediction model;

[0032] The K and M are pre-set according to the amount of data in the depth prediction model training set.

[0033] Optionally, in S400,

[0034] The depth prediction model includes an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer which are cascaded to each other in sequence;

[0035] Among them, the input layer receives the local image of sperm flagella; the convolution layer extracts features in the image by gradually increasing the number and depth of convolution kernels; the pooling layer is used to reduce the spatial size of the features extracted by the convolution layer, and the fully connected layer processes the features pooled by the pooling layer through linear combination and nonlinear activation function; the output layer is used to perform regression tasks, obtain and output the depth information corresponding to each local image of sperm flagella.

[0036] In a second aspect, an embodiment of the present invention provides a computer device, including: a memory and a processor;

[0037] The memory stores a computer program, and the processor executes the computer program in the memory, specifically performing the method for reconstructing the three-dimensional motion of sperm flagella based on deep learning as described in any of the above claims.

[0038] (III) Beneficial effects

[0039] The beneficial effects of the present invention are as follows: a method for reconstructing the three-dimensional motion of sperm flagella based on deep learning of the present invention can complete image analysis and three-dimensional reconstruction in a short time due to the use of a sperm image segmentation model based on deep learning and a depth prediction model, thereby improving real-time performance, reducing computing time and resource consumption, and achieving high-precision three-dimensional reconstruction; the present invention relies on a bright field microscope and a computer, does not require complex optical settings and calibration processes, avoids the need for expensive multiple shorter adapters and other complex equipment, has simple equipment, simple experimental operations, relatively low maintenance and operating costs, and has wide applicability; the present invention adopts label-free imaging technology, avoids staining of sperm samples, maintains sperm activity and integrity, is suitable for sperm evaluation and selection in clinical and assisted reproductive technology, and has practical application value; the present invention can track the three-dimensional motion trajectory of sperm flagella in real time by reconstructing a three-dimensional model of sperm flagella, and adapts to samples with rapid motion and complex dynamic changes. It provides more comprehensive sperm dynamics information, assists doctors in more accurate sperm evaluation and selection, and improves the success rate of assisted reproductive technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic flow chart of a method for reconstructing three-dimensional motion of sperm flagella based on deep learning according to an embodiment of the present invention;

[0041] Figure 2 This is a bright field image of sperm in Example 2 of the present invention;

[0042] Figure 3 A partial image of the sperm flagellum in Example 2 of the present invention;

[0043] Figure 4 middle, Figure 4 (a) is a bright field image of sperm. Figure 4 (b) is the sperm binary image, Figure 4 (c) is the sperm flagella skeleton, Figure 4 (d) is a schematic diagram of the local image segmentation of sperm flagella;

[0044] Figure 5 It is a scatter plot and regression line diagram of the true value and the predicted value in Example 2;

[0045] Figure 6 The schematic diagram of the three-dimensional trajectory of the sperm flagellum after reconstruction;

[0046] Figure 7 Schematic diagram of the training process of the sperm segmentation model and deep learning model and the reconstruction of the three-dimensional motion of sperm flagella in Example 2 of the present invention. DETAILED DESCRIPTION

[0047] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0048] The embodiment of the present invention proposes a method for reconstructing the three-dimensional motion of sperm flagella based on deep learning. By using a deep learning sperm image segmentation model and a depth prediction model, it can reduce computing time and resource consumption, and does not require sperm staining, thereby maintaining the activity and integrity of the sperm. It has practical application value and wide practicability. At the same time, the method can track the three-dimensional motion trajectory of sperm flagella in real time, adapt to samples with rapid movement and complex dynamic changes, provide more comprehensive sperm dynamics information, assist doctors in more accurate sperm evaluation and selection, and improve the success rate of assisted reproduction.

[0049] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0050] Specific embodiment description part

[0051] Example 1

[0052] See also Figure 1 , a method for reconstructing three-dimensional motion of sperm flagella based on deep learning in an embodiment of the present invention comprises:

[0053] Step S100, obtaining a two-dimensional image of the sperm to be processed, and performing image preprocessing on the two-dimensional image of the sperm to be processed to obtain a preprocessed image;

[0054] Step S200, input the preprocessed image into the trained sperm segmentation model to obtain a sperm flagella binary image; the sperm segmentation model is a YOLOv8 model;

[0055] Step S300, skeletonizing the binary image of the sperm flagellum to obtain the center line of the sperm flagellum; based on the center line of the sperm flagellum, obtaining N equidistant points and the position coordinates corresponding to each equidistant point;

[0056] Step S400, based on N equidistant points, obtaining N sperm flagella local images, and inputting each sperm flagella local image into a trained depth prediction model to obtain depth information of each sperm flagella local image; the depth prediction model is a convolutional neural network model;

[0057] Step S500: reconstructing a three-dimensional model of the sperm flagellum based on the depth information of each sperm flagellum local image and the position coordinates corresponding to each equidistant point.

[0058] In this embodiment, Matlab is used to reconstruct the three-dimensional model.

[0059] The technical solution in this embodiment combines advanced deep learning technology to establish a sperm segmentation model and a depth prediction model. The two-dimensional image of sperm is input into the sperm segmentation model and the depth prediction model to obtain the three-dimensional coordinate points of the sperm flagellum, thereby efficiently and accurately reconstructing the three-dimensional model of the sperm flagellum, providing a reliable solution for the study of sperm motility behavior and the development of assisted reproductive technology.

[0060] In addition, sperm motility is a key indicator for assessing male fertility. Through 3D reconstruction, sperm motility and quality can be more accurately assessed, such as whether sperm has a normal swimming pattern. This detailed assessment helps doctors develop personalized treatment plans and improve the success rate of artificial reproductive technologies such as in vitro fertilization. It can also effectively guide drug development and the optimization of treatment plans.

[0061] Example 2

[0062] See also Figure 7 , a method for reconstructing the three-dimensional motion of sperm flagella based on deep learning in this embodiment includes:

[0063] Step S100: Obtain a two-dimensional image of the sperm to be processed, such as Figure 2 As shown, the two-dimensional image of the sperm to be processed is subjected to image preprocessing to obtain a preprocessed image;

[0064] Step S200, input the preprocessed image into the trained sperm segmentation model to obtain a sperm flagella binary image; the sperm segmentation model is a YOLOv8 model;

[0065] Step S300, skeletonizing the binary image of the sperm flagellum to obtain the center line of the sperm flagellum; based on the center line of the sperm flagellum, obtaining N equidistant points and the position coordinates corresponding to each equidistant point;

[0066] Step S400, based on N equidistant points, obtaining N sperm flagella local images, and inputting each sperm flagella local image into a trained depth prediction model to obtain depth information of each sperm flagella local image; the depth prediction model is a convolutional neural network model;

[0067] Step S500: reconstructing a three-dimensional model of the sperm flagellum based on the depth information of each sperm flagellum local image and the position coordinates corresponding to each equidistant point.

[0068] In this embodiment, by reconstructing the three-dimensional model of the sperm flagellum, the three-dimensional motion trajectory of the sperm flagellum can be obtained, such as Figure 6 shown.

[0069] In the specific implementation process, step S100 includes:

[0070] The two-dimensional image of the sperm to be processed is enhanced by using a filter to obtain a preprocessed image, such as Figure 4 As shown in (a); the two-dimensional image of the sperm to be processed is acquired by bright field microscopy.

[0071] For example, in order to reduce noise in an image and smooth the image, a Gaussian filter is used for image processing; the Gaussian filter removes high-frequency noise by averaging each pixel and its neighborhood in a two-dimensional image of sperm using Gaussian-distributed weights. The Gaussian kernel of the Gaussian filter is usually selected based on the resolution of the two-dimensional image of sperm and the expected noise level.

[0072] In addition, after filter enhancement, in order to further optimize the sperm outline, morphological operations such as dilation and erosion can be applied to help remove small noise points and broken connections, making the characteristics of sperm flagella more obvious.

[0073] In this embodiment, in step S300,

[0074] N equidistant points are obtained by intercepting the center line of the sperm flagellum at equidistant intervals of X μm; X is a preset distance.

[0075] In this embodiment, X is set to 0.27.

[0076] Optionally, in step S400, based on N equidistant points, obtaining N sperm flagella local images includes:

[0077] Taking each of the N equidistant points as the center, N sperm flagella partial images are intercepted; each of the N sperm flagella partial images contains 1-2 μm sperm flagella image information.

[0078] In the specific implementation process, the sperm flagella binary image is an image with a black background and white sperm flagella, such as Figure 4 (b) As shown; In this embodiment, the binary image of the sperm flagellum is skeletonized using the OpenCV algorithm. The skeletonized image has processed the sperm flagellum into a line with a pixel width, such as Figure 4 (c) as shown.

[0079] In this embodiment, equidistant points are taken along the center line of the sperm flagellum, that is, a point is selected every 0.27 μm as an equidistant point. Assuming that the total length of the sperm flagellum is 55 mm, 203 equidistant points can be obtained; see Figure 4 (d) Based on the above equidistant points, a sperm flagella local image with a size of 101×21 pixels is captured with each equidistant point as the center. At this time, each sperm flagella local image contains 1-2 μm sperm flagella image information, such as Figure 3 shown.

[0080] In this embodiment, before step S100, the following steps are also included:

[0081] Step S000: Use a training data set to train a sperm segmentation model and a depth prediction model to obtain a trained sperm segmentation model and a depth prediction model; the training data set includes: a sperm segmentation model training data set and a depth prediction model training data set.

[0082] Optionally, the sperm segmentation model training set includes: sperm images and bounding box coordinates corresponding to each sperm image;

[0083] Sperm images were acquired through a bright field microscope; the bounding box coordinates corresponding to each sperm image were acquired by annotating the sperm image using Labelbox or LabelMe.

[0084] Optionally, the depth prediction model training data set includes: sperm flagella local images at different focal planes and depth information corresponding to each sperm flagella local image;

[0085] The local images of sperm flagella at different focal planes and the depth information corresponding to each local image of sperm flagella are obtained by photographing the images of sperm flagella at different focal planes through a piezoelectric platform and intercepting the photographed sperm flagella images; the local images of sperm flagella contain sperm flagella image information of about 1-2 μm. Step S000 specifically includes:

[0086] Step S010, input each data in the sperm segmentation model training set into the corresponding sperm segmentation model, use the target loss function and Adam optimizer to obtain the trained sperm segmentation model;

[0087] The target loss function includes a bounding box regression loss function, a target confidence loss function and a category classification loss function;

[0088] Step S020: input each data in the depth prediction model training set into the corresponding depth prediction model, use the mean square error loss function and Adam optimizer, with an initial learning rate of K, until M iterations are performed to obtain a trained depth prediction model;

[0089] The K and M are pre-set according to the amount of data in the depth prediction model training set.

[0090] In the specific implementation process, K is set to 0.001 and M is set to 150 times.

[0091] In this embodiment, the sperm segmentation model divides the data set into a training set, a validation set, and a test set in a ratio of 8:1:1; the training and validation of the depth prediction model adopt a 10-fold cross-validation method.

[0092] In step S010, each data in the training set and validation set in the sperm segmentation model training set is first input into the preset initialized sperm segmentation model, and the input image is passed to the network through forward propagation to generate predicted bounding box coordinates and category probabilities. Next, the difference between the predicted bounding box coordinates and the actual bounding box coordinates is calculated by the loss function, and the loss function includes bounding box regression loss, target confidence loss and category classification loss. Then, the gradient of the loss function relative to the model parameters is calculated by back propagation, and it is propagated back layer by layer from the output layer. Finally, the model parameters are updated by the optimization algorithm Adam to minimize the loss function. This process is repeated in each training iteration to gradually optimize the parameters and improve the detection performance of the model. At the same time, the early stopping strategy is adopted in the training process of the sperm segmentation model. Specifically, if the performance index of the validation set does not improve significantly in 50 consecutive iterations, the training is stopped and the sperm segmentation model is obtained. This strategy helps prevent model overfitting and ensures that training is stopped at the best time. After the training and validation process is completed, each data in the test set is input into the sperm segmentation model, and the average accuracy, recall rate and precision of the model at this time are obtained to evaluate the final performance of the sperm segmentation model.

[0093] In step S020, a 10-fold cross-validation method is used to divide the depth prediction model training set into 10 subsets of equal size. In each iteration, 9 folds are used as training sets, and the remaining 1 fold is used as a validation set to evaluate the performance of the model.

[0094] In the specific implementation process, each data in the depth prediction model training set is input into the preset initialized depth prediction model, and the mean square error loss function is used to measure the prediction error. Through continuous iteration, the model parameters are adjusted to minimize the loss function to obtain the optimized depth prediction model. During the entire training process, a total of 150 training rounds are performed, and the accuracy of the validation set in the depth prediction model is calculated at the same time. After the training and validation process is completed, the mean square error and determination coefficient R2 value of the final model are calculated to evaluate the final performance of the model.

[0095] For example, there are 2000 data in the depth prediction model training set. After the training and verification process is completed, the determination coefficient R obtained by linear fitting of the final actual value and the predicted value obtained by the depth prediction model is 2 The value reaches 0.99, such as Figure 5 As shown, this indicates that the model can accurately predict the depth information of the image. The depth prediction value obtained by the depth prediction model is highly correlated with the actual value, verifying the reliability and stability of the model.

[0096] In this embodiment, the Adam optimizer can also be changed to an SGD optimizer.

[0097] In step S400,

[0098] The depth prediction model includes a cascaded input layer, convolutional layer, pooling layer, fully connected layer, and output layer in sequence;

[0099] The input layer of the deep prediction model receives a local image of a sperm flagellum of a specified size. The convolution layer extracts basic and advanced features from the image by gradually increasing the number and depth of convolution kernels. The pooling layer reduces the spatial size of the feature map while retaining important feature information. The fully connected layer processes and refines the features through linear combinations and nonlinear activation functions, and finally performs regression tasks through the output layer to predict the target value. The entire deep prediction model is optimized through training to minimize the mean square error loss function to obtain an accurate prediction model.

[0100] For example, the input layer receives a single-channel sperm flagellum partial image of size 101×21 pixels. Then, the model contains three convolutional layers, each of which uses a 3×3 convolution kernel. The first layer uses 16 convolution kernels, the second layer uses 32 convolution kernels, and the third layer uses 64 convolution kernels. The pooling layer includes two layers of maximum pooling. The first layer uses a 3×3 pooling window with a stride of [3, 3]. The second pooling layer also uses a 3×3 pooling window with a stride of [3, 3]. The fully connected layer contains 256 neurons, and the activation function is ReLU. The next layers include 128 neurons, 64 neurons, and 32 neurons, all using the ReLU activation function. Finally, the output layer contains 1 neuron for regression tasks to predict depth.

[0101] The method of reconstructing the three-dimensional motion of sperm flagella by deep learning of the present embodiment can complete image analysis and three-dimensional reconstruction in a short time through the fast processing capability of the deep learning sperm segmentation model and the deep prediction model, thereby improving the real-time performance, reducing the computing time and resource consumption, and can accurately segment and predict the focal position of the sperm flagellum, thereby achieving high-precision three-dimensional reconstruction. In addition, the present invention mainly relies on a bright field microscope and a computer, which can avoid the need for expensive and complex equipment. The equipment is simple, and the maintenance and operation costs are relatively low, which is suitable for large-scale application and promotion. The bright field microscope is used for image acquisition, which does not require complex optical settings and calibration processes, simplifies experimental operations, is applicable to a variety of microscope types, and has a wide range of applicability.

[0102] At the same time, this embodiment adopts label-free imaging technology, avoids staining of sperm samples, ensures sperm activity and integrity, and is suitable for sperm evaluation and selection in clinical and assisted reproductive technology, and has practical application value. In addition, the present invention can obtain a three-dimensional model of sperm flagella, that is, it can obtain the three-dimensional motion trajectory of sperm flagella, thereby providing more comprehensive sperm dynamics information, allowing doctors to more accurately evaluate and select sperm, and improve the success rate of assisted reproductive technology.

[0103] Example 3

[0104] A computer device of this embodiment includes a memory and a processor;

[0105] The computer program is stored in the memory, and the processor executes the computer program in the memory, specifically executing a method for reconstructing the three-dimensional motion of sperm flagella based on deep learning in Example 1 or Example 2.

[0106] A computer device of the present invention can reconstruct the three-dimensional model of sperm flagella more efficiently and accurately by executing a method for reconstructing the three-dimensional motion of sperm flagella based on deep learning, providing a reliable solution for the study of sperm movement behavior and the development of assisted reproductive technology.

[0107] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0108] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0109] In the present invention, unless otherwise clearly specified and limited, when a first feature is “on” or “below” a second feature, it may be that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Moreover, when a first feature is “above”, “above” or “above” a second feature, it may be that the first feature is directly above or obliquely above the second feature, or it may simply mean that the first feature is higher in level than the second feature. When a first feature is “below”, “below” or “below” a second feature, it may be that the first feature is directly below or obliquely below the second feature, or it may simply mean that the first feature is lower in level than the second feature.

[0110] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0111] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may alter, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for reconstructing the three-dimensional motion of sperm flagella based on deep learning, It is characterized in that include: S100, obtaining a two-dimensional image of sperm to be processed, and performing image preprocessing on the two-dimensional image of sperm to be processed to obtain a preprocessed image; S200, inputting the pre-processed image into a trained sperm segmentation model to obtain a sperm flagella binary image; the sperm segmentation model is a YOLOv8 model; S300, skeletonizing the binary image of the sperm flagellum to obtain the center line of the sperm flagellum; based on the center line of the sperm flagellum, obtaining N equidistant points and the position coordinates corresponding to each equidistant point; S400, based on the N equidistant points, obtaining N sperm flagella local images, and inputting each sperm flagella local image into a trained depth prediction model to obtain depth information of each sperm flagella local image; the depth prediction model is a convolutional neural network model; S500: reconstructing a three-dimensional model of the sperm flagellum based on the depth information of each sperm flagellum local image and the position coordinates corresponding to each equidistant point.

2. The method for reconstructing the three-dimensional motion of sperm flagella based on deep learning according to claim 1, It is characterized in that The S100 includes: Using a filter to enhance the two-dimensional image of the sperm to be processed to obtain a pre-processed image; The two-dimensional image of the sperm to be processed is acquired by bright field microscopy.

3. The method for reconstructing the three-dimensional motion of sperm flagella based on deep learning according to claim 1, It is characterized in that In the S300, The N equidistant points are obtained by intercepting the center line of the sperm flagellum at equidistant intervals of X μm; X is a preset distance.

4. The method for reconstructing the three-dimensional motion of sperm flagella based on deep learning according to claim 1, It is characterized in that In S400, based on the N equidistant points, obtaining N sperm flagella local images includes: Taking each of the N equidistant points as the center, N sperm flagella partial images are intercepted; each of the N sperm flagella partial images contains 1-2 μm sperm flagella image information.

5. The method for reconstructing the three-dimensional motion of sperm flagella based on deep learning according to claim 1, It is characterized in that The S100 also includes: S000. Use a training data set to train the sperm segmentation model and the depth prediction model to obtain a trained sperm segmentation model and a depth prediction model; the training data set includes: a sperm segmentation model training data set and a depth prediction model training data set.

6. The method for reconstructing the three-dimensional motion of sperm flagella based on deep learning according to claim 5, It is characterized in that The sperm segmentation model training set includes: sperm images and bounding box coordinates corresponding to each sperm image; The sperm image is obtained through a bright field microscope; the bounding box coordinates corresponding to each sperm image are obtained by annotating the sperm image through Labelbox or LabelMe.

7. The method for reconstructing the three-dimensional motion of sperm flagella based on deep learning according to claim 5, It is characterized in that The depth prediction model training data set includes: sperm flagella local images at different focal planes and depth information corresponding to each sperm flagella local image; The local images of sperm flagella at different focal planes and the depth information corresponding to each local image of sperm flagella are obtained by photographing sperm flagella images at different focal planes through a piezoelectric platform and intercepting the photographed sperm flagella images; the local images of sperm flagella contain 1-2 μm sperm flagella image information.

8. The method for reconstructing the three-dimensional motion of sperm flagella based on deep learning according to claim 5, It is characterized in that The S000 specifically includes: S01 0. Input each data in the sperm segmentation model training set into the corresponding sperm segmentation model, and use the target loss function and Adam optimizer to obtain the trained sperm segmentation model; The target loss function includes a bounding box regression loss function, a target confidence loss function and a category classification loss function; S020, input each data in the depth prediction model training set into the corresponding depth prediction model, use the mean square error loss function and Adam optimizer, with an initial learning rate of K, until M iterations are performed to obtain a trained depth prediction model; The K and M are pre-set according to the amount of data in the depth prediction model training set.

9. The method for reconstructing the three-dimensional motion of sperm flagella based on deep learning according to claim 1, It is characterized in that In the S400, The depth prediction model includes an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer which are cascaded to each other in sequence; Among them, the input layer receives the local image of sperm flagella; the convolution layer extracts features in the image by gradually increasing the number and depth of convolution kernels; the pooling layer is used to reduce the spatial size of the features extracted by the convolution layer, and the fully connected layer processes the features pooled by the pooling layer through linear combination and nonlinear activation function; the output layer is used to perform regression tasks, obtain and output the depth information corresponding to each local image of sperm flagella.

10. A computer device, It is characterized in that include: Memory and processor; The memory stores a computer program, and the processor executes the computer program in the memory, specifically performing the method for reconstructing the three-dimensional motion of sperm flagella based on deep learning as described in any one of claims 1 to 9.

Images