A method for reconstructing three-dimensional motion of sperm flagella based on deep learning

Abstract

The present invention relates to a method for reconstructing the three-dimensional motion of sperm flagella based on deep learning, comprising the steps of: obtaining a two-dimensional image of a sperm to be processed, performing image preprocessing on the two-dimensional image of the sperm to be processed, and obtaining a preprocessed image; inputting the preprocessed image into a trained sperm segmentation model to obtain a binary image of the sperm flagellum; skeletonizing the binary image of the sperm flagellum to obtain the centerline of the sperm flagellum, N equidistant points, and the position coordinates corresponding to each equidistant point; obtaining N partial images of the sperm flagellum based on the N equidistant points, and inputting each partial image of the sperm flagellum into a trained depth prediction model to obtain depth information of each partial image of the sperm flagellum; and reconstructing a three-dimensional model of the sperm flagellum based on the depth information of each partial image of the sperm flagellum and the position coordinates corresponding to each equidistant point. The present invention provides a reliable solution for the study of sperm motility behavior and the development of assisted reproductive technology.

Description

Technical Field

[0001] The present invention relates to the technical field of reproductive biology, and in particular to a method for reconstructing the three-dimensional motion of sperm flagella based on deep learning. Background Art

[0002] In reproductive biology research, the movement of sperm flagella is not a simple two-dimensional motion, but a complex three-dimensional movement. This three-dimensional motion is crucial for sperm to successfully locate the egg and complete fertilization. The three-dimensional oscillation pattern of the flagellum enhances sperm propulsion efficiency in the viscous environment of the reproductive tract, thereby increasing the probability of fertilization. Understanding and analyzing the three-dimensional motion patterns of sperm flagella is not only of great significance for the diagnosis and treatment of infertility, the optimization of assisted reproductive technology, and basic reproductive biology research, but also has significant research value in biomimetics.

[0003] Clinically, doctors select sperm based primarily on their morphology and dynamics, an assessment that typically relies on two-dimensional (2-D) imaging techniques, such as brightfield microscopy (BFM), differential interference contrast microscopy (DIC), Hoffman modulation contrast microscopy (HMC), and Zernike phase contrast microscopy. However, the structure of sperm flagella is very slender, and sperm used in assisted reproductive technology cannot be stained. Furthermore, their motility is three-dimensional and easily out of focus under a microscope, leading to many challenges in the imaging process. Therefore, current two-dimensional imaging technology cannot fully capture the motion characteristics of sperm in three-dimensional space. This limitation of the technology has led to an incomplete understanding of sperm dynamics, which may affect the accuracy of sperm selection and the success rate of assisted reproductive technology.

[0004] Currently, several imaging techniques or devices can be used to reconstruct the three-dimensional morphology of a sample, such as confocal microscopy and light-sheet microscopy. However, these imaging methods have limitations when reconstructing the three-dimensional motion of sperm flagella, primarily due to their limited temporal resolution. The sperm flagellum typically beats at a frequency of around 20 Hz. In comparison, a single 3D scan with a confocal microscope can take several seconds to tens of seconds, while a 3D image with a light-sheet microscope can take between hundreds of milliseconds and a few seconds. This makes these imaging techniques incapable of capturing the high-speed dynamics of the sperm flagellum in real time. In recent years, much research has focused on accurately tracking the position of the entire flagellum in time and three-dimensional space. The degree of defocus in an image can be used to infer the z-axis position of an object. The basic principle is that an image is sharpest when the object is in the focal plane of the microscope; when the object leaves the focal plane, the image becomes blurred. By measuring and analyzing the degree of blur (i.e., the degree of defocus) in the image, the z-axis position of the object can be inferred. Bukatin et al. developed an algorithm that identifies the projected 2D shape of the flagellum by analyzing pixel intensity levels. The z-coordinate is then estimated by analyzing the intensity distribution along a cross section, allowing the flagellum to be reconstructed perpendicular to the image plane from a 2D BFM image. This method exploits the fact that the width of the halo is related to the z-displacement from the focal plane. While simple and practical, this method has major drawbacks: its dependence on image quality, its accuracy is easily affected by image noise, and its effectiveness in handling complex 3D motion is limited. This method also requires precise calibration of the image relative to the z-position. Jan N. Hansen combined the depth-from-defocus method with MFI (multifocus imaging) to successfully reconstruct the 3D motion of spherical objects and sperm flagella using a dark-field microscope and a multifocus adapter. Although this method offers advantages in terms of improved spatial resolution, it is highly demanding on the equipment and requires a multifocus adapter. Furthermore, this method is highly sensitive to noise, and 3D reconstruction from multifocal plane data is computationally intensive, resulting in poor real-time performance. The study by Silva-Villalobos et al. used an oscillating objective lens mounted on a bright-field optical microscope. They scanned a 16-micron depth of the sample at a rate of 5,000 images per second. They were able to precisely locate the z-axis position along the flagellum where optimal focus was achieved for each part of the flagellum, enabling three-dimensional reconstruction of the sperm flagellum. While this method excels in improving temporal resolution, its main drawback is the complexity and high cost of the equipment, making it difficult to promote in practical applications. While these existing methods each have their own advantages for three-dimensionally reconstructing sperm flagellar motion, they all suffer from limitations in spatiotemporal resolution, computational complexity, equipment cost, and the complexity of the optical path setup.

[0005] Therefore, there is an urgent need for a method to reconstruct the three-dimensional motion of sperm flagella based on deep learning. Summary of the Invention

[0006] (1) Technical issues to be resolved

[0007] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a method for reconstructing the three-dimensional motion of sperm flagella based on deep learning, which solves the technical problems in the prior art of achieving three-dimensional reconstruction of sperm flagella, such as low spatiotemporal resolution, complex calculations, high equipment costs, and complex optical path settings.

[0008] (2) Technical solution

[0009] In order to achieve the above objectives, the main technical solutions adopted by the present invention include:

[0010] In a first aspect, an embodiment of the present invention provides a method for reconstructing three-dimensional motion of sperm flagella based on deep learning, comprising:

[0011] S100, obtaining a two-dimensional image of sperm to be processed, and performing image preprocessing on the two-dimensional image of sperm to be processed to obtain a preprocessed image;

[0012] S200, inputting the pre-processed image into a trained sperm segmentation model to obtain a sperm flagellum binary image; the sperm segmentation model is a YOLOv8 model;

[0013] S300, skeletonizing the binary image of the sperm flagellum to obtain the centerline of the sperm flagellum; based on the centerline of the sperm flagellum, obtaining N equidistant points and the position coordinates corresponding to each equidistant point;

[0014] S400, based on the N equidistant points, obtaining N sperm flagella partial images, and inputting each sperm flagella partial image into a trained depth prediction model to obtain depth information of each sperm flagella partial image; the depth prediction model is a convolutional neural network model;

[0015] S500 : Reconstructing a three-dimensional model of the sperm flagellum based on the depth information of each sperm flagellum local image and the position coordinates corresponding to each equidistant point.

[0016] Optionally, the S100 includes:

[0017] Performing filter enhancement on the two-dimensional image of the sperm to be processed using a filter to obtain a pre-processed image;

[0018] The two-dimensional image of the sperm to be processed is acquired by bright field microscopy.

[0019] Optionally, in S300,

[0020] The N equidistant points are obtained by intercepting the center line of the sperm flagellum at equal distances every X μm; X is a preset distance.

[0021] Optionally, in S400, acquiring N sperm flagella local images based on the N equidistant points includes:

[0022] Taking each of the N equidistant points as the center, N sperm flagella partial images are intercepted; each of the N sperm flagella partial images contains 1-2 μm sperm flagella image information.

[0023] Optionally, before S100, the step further includes:

[0024] S000. Use a training data set to train the sperm segmentation model and the depth prediction model to obtain a trained sperm segmentation model and a depth prediction model; the training data set includes: a sperm segmentation model training data set and a depth prediction model training data set.

[0025] Optionally, the sperm segmentation model training set includes: sperm images and bounding box coordinates corresponding to each sperm image;

[0026] The sperm image is obtained by bright field microscopy; the bounding box coordinates corresponding to each sperm image are obtained by annotating the sperm image using Labelbox or LabelMe.

[0027] Optionally, the depth prediction model training data set includes: local images of sperm flagella at different focal planes and depth information corresponding to each local image of sperm flagella;

[0028] The partial images of sperm flagella at different focal planes and the depth information corresponding to each partial image of sperm flagella are obtained by photographing sperm flagella images at different focal planes using a piezoelectric platform and intercepting the photographed sperm flagella images; the partial images of sperm flagella include sperm flagella image information of 1-2 μm. Optionally, S000 specifically includes:

[0029] S010, inputting each data in the sperm segmentation model training set into the corresponding sperm segmentation model, using the target loss function and Adam optimizer to obtain a trained sperm segmentation model;

[0030] The target loss function includes a bounding box regression loss function, a target confidence loss function and a category classification loss function;

[0031] S020, input each data in the depth prediction model training set into the corresponding depth prediction model, use the mean square error loss function and Adam optimizer, with an initial learning rate of K, until M iterations are performed to obtain a trained depth prediction model;

[0032] The K and M are pre-set according to the amount of data in the depth prediction model training set.

[0033] Optionally, in S400,

[0034] The depth prediction model includes an input layer, a convolutional layer, a pooling layer, a fully connected layer and an output layer that are cascaded to each other in sequence;

[0035] Among them, the input layer receives the local image of the sperm flagellum; the convolution layer extracts features in the image by gradually increasing the number and depth of the convolution kernel; the pooling layer is used to reduce the spatial size of the features extracted by the convolution layer, and the fully connected layer processes the features pooled by the pooling layer through linear combination and nonlinear activation function; the output layer is used to perform regression tasks, obtain and output the depth information corresponding to each local image of the sperm flagellum.

[0036] In a second aspect, an embodiment of the present invention provides a computer device, including: a memory and a processor;

[0037] The memory stores a computer program, and the processor executes the computer program in the memory, specifically performing the method for reconstructing the three-dimensional motion of sperm flagella based on deep learning as described in any of the above claims.

[0038] (3) Beneficial effects

[0039] The present invention has the following beneficial effects: a method for reconstructing the three-dimensional motion of sperm flagella based on deep learning, which, due to the use of a sperm image segmentation model and a depth prediction model based on deep learning, can complete image analysis and three-dimensional reconstruction in a short time, improving real-time performance, reducing computing time and resource consumption, and achieving high-precision three-dimensional reconstruction; the present invention relies on a brightfield microscope and a computer, does not require complex optical settings and calibration processes, and avoids the need for expensive multiple short adapters and other complex equipment. The equipment is simple, the experimental operation is simple, the maintenance and operation costs are relatively low, and it has wide applicability; the present invention uses label-free imaging technology, avoids staining of sperm samples, maintains sperm activity and integrity, and is suitable for sperm assessment and selection in clinical and assisted reproductive technology, with practical application value; by reconstructing a three-dimensional model of the sperm flagellum, the present invention can track the three-dimensional motion trajectory of the sperm flagellum in real time, and is adaptable to samples with rapid movement and complex dynamic changes. It provides more comprehensive sperm dynamics information, assists doctors in more accurate sperm assessment and selection, and improves the success rate of assisted reproductive technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Schematic diagram of a flow chart of a method for reconstructing three-dimensional motion of sperm flagella based on deep learning according to an embodiment of the present invention;

[0041] Figure 2 This is a bright field image of sperm in Example 2 of the present invention;

[0042] Figure 3 This is a partial image of the sperm flagellum in Example 2 of the present invention;

[0043] Figure 4 middle, Figure 4 (a) is the bright field image of sperm, Figure 4 (b) is the sperm binary image, Figure 4 (c) is the sperm flagella skeleton, Figure 4 (d) is a schematic diagram of the local image segmentation of sperm flagella;

[0044] Figure 5 The scatter plot and regression line diagram of the actual value and the predicted value in Example 2;

[0045] Figure 6 Schematic diagram of the three-dimensional trajectory of the reconstructed sperm flagellum;

[0046] Figure 7 Schematic diagram of the training process of the sperm segmentation model and deep learning model and the reconstruction of the three-dimensional motion of sperm flagella in Example 2 of the present invention. DETAILED DESCRIPTION

[0047] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation methods in conjunction with the accompanying drawings.

[0048] An embodiment of the present invention proposes a method for reconstructing the three-dimensional motion of sperm flagella based on deep learning. By using a deep learning sperm image segmentation model and a depth prediction model, it can reduce computing time and resource consumption, and does not require sperm staining, thereby maintaining the activity and integrity of the sperm. It has practical application value and wide practicality. At the same time, this method can track the three-dimensional motion trajectory of sperm flagella in real time, adapt to samples with rapid movement and complex dynamic changes, provide more comprehensive sperm dynamics information, assist doctors in more accurate sperm evaluation and selection, and improve the success rate of assisted reproduction.

[0049] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. Instead, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0050] Specific embodiment description part

[0051] Example 1

[0052] See also Figure 1 , a method for reconstructing three-dimensional motion of sperm flagella based on deep learning in an embodiment of the present invention includes:

[0053] Step S100, obtaining a two-dimensional image of the sperm to be processed, and performing image preprocessing on the two-dimensional image of the sperm to be processed to obtain a preprocessed image;

[0054] Step S200: input the pre-processed image into a trained sperm segmentation model to obtain a binary image of sperm flagella; the sperm segmentation model is a YOLOv8 model;

[0055] Step S300: skeletonize the binary image of the sperm flagellum to obtain the center line of the sperm flagellum; based on the center line of the sperm flagellum, obtain N equidistant points and the position coordinates corresponding to each equidistant point;

[0056] Step S400: Based on N equidistant points, N sperm flagella local images are obtained, and each sperm flagella local image is input into a trained depth prediction model to obtain depth information of each sperm flagella local image; the depth prediction model is a convolutional neural network model;

[0057] Step S500: reconstructing a three-dimensional model of the sperm flagellum based on the depth information of each sperm flagellum local image and the position coordinates corresponding to each equidistant point.

[0058] In this embodiment, Matlab is used to reconstruct the three-dimensional model.

[0059] The technical solution in this embodiment combines advanced deep learning technology to establish a sperm segmentation model and a depth prediction model. By inputting the two-dimensional image of sperm into the sperm segmentation model and the depth prediction model, the three-dimensional coordinate points of the sperm flagellum can be obtained, thereby efficiently and accurately reconstructing the three-dimensional model of the sperm flagellum, providing a reliable solution for the study of sperm motility behavior and the development of assisted reproductive technology.

[0060] Furthermore, sperm motility is a key indicator for assessing male fertility. 3D reconstruction allows for a more accurate assessment of sperm motility and quality, such as determining whether sperm have a normal swimming pattern. This detailed assessment helps physicians develop personalized treatment plans, improve the success rate of artificial reproductive technologies like in vitro fertilization, and effectively guide drug development and treatment optimization.

[0061] Example 2

[0062] See also Figure 7 , a method for reconstructing three-dimensional motion of sperm flagella based on deep learning in this embodiment includes:

[0063] Step S100: Obtain a two-dimensional image of the sperm to be processed, such as Figure 2 As shown, the two-dimensional image of the sperm to be processed is subjected to image preprocessing to obtain a preprocessed image;

[0064] Step S200: input the pre-processed image into a trained sperm segmentation model to obtain a binary image of sperm flagella; the sperm segmentation model is a YOLOv8 model;

[0065] Step S300: skeletonize the binary image of the sperm flagellum to obtain the center line of the sperm flagellum; based on the center line of the sperm flagellum, obtain N equidistant points and the position coordinates corresponding to each equidistant point;

[0066] Step S400: Based on N equidistant points, N sperm flagella local images are obtained, and each sperm flagella local image is input into a trained depth prediction model to obtain depth information of each sperm flagella local image; the depth prediction model is a convolutional neural network model;

[0067] Step S500: reconstructing a three-dimensional model of the sperm flagellum based on the depth information of each sperm flagellum local image and the position coordinates corresponding to each equidistant point.

[0068] In this embodiment, by reconstructing the three-dimensional model of the sperm flagellum, the three-dimensional motion trajectory of the sperm flagellum can be obtained, such as Figure 6 shown.

[0069] In a specific implementation process, step S100 includes:

[0070] The two-dimensional image of the sperm to be processed is enhanced by using a filter to obtain a pre-processed image, such as Figure 4 As shown in (a); the two-dimensional image of the sperm to be processed is acquired by bright field microscopy.

[0071] For example, to reduce noise and smooth an image, a Gaussian filter is used for image processing. A Gaussian filter removes high-frequency noise by averaging each pixel and its neighborhood in a two-dimensional sperm image using Gaussian-distributed weights. The Gaussian kernel of the Gaussian filter is typically selected based on the resolution of the two-dimensional sperm image and the expected noise level.

[0072] In addition, after filter enhancement, in order to further optimize the outline of the sperm, morphological operations such as dilation and erosion can be applied to help remove small noise points and broken connections, making the characteristics of the sperm flagellum more obvious.

[0073] In this embodiment, in step S300,

[0074] N equidistant points are obtained by intercepting the center line of the sperm flagellum at equal distances every X μm; X is a preset distance.

[0075] In this embodiment, X is set to 0.27.

[0076] Optionally, in step S400, obtaining N sperm flagella local images based on N equidistant points includes:

[0077] Taking each of the N equidistant points as the center, N sperm flagella partial images are intercepted; each of the N sperm flagella partial images contains 1-2 μm sperm flagella image information.

[0078] In the specific implementation process, the sperm flagella binary image is an image with a black background and white sperm flagella, such as Figure 4 (b) As shown; In this embodiment, the OpenCV algorithm is used to skeletonize the binary image of the sperm flagellum. The skeletonized image has processed the sperm flagellum into a line with a pixel width, as shown in FIG. Figure 4 (c) shown.

[0079] In this embodiment, equidistant points are taken along the center line of the sperm flagellum, that is, a point is selected every 0.27 μm as an equidistant point. Assuming that the total length of the sperm flagellum is 55 mm, 203 equidistant points can be obtained; see Figure 4 (d) Based on the above equidistant points, a sperm flagella local image with a size of 101×21 pixels is captured with each equidistant point as the center. At this time, each sperm flagella local image contains 1-2μm sperm flagella image information, such as Figure 3 shown.

[0080] In this embodiment, before step S100, the following steps are further included:

[0081] Step S000: Use a training data set to train a sperm segmentation model and a depth prediction model to obtain a trained sperm segmentation model and a depth prediction model; the training data set includes: a sperm segmentation model training data set and a depth prediction model training data set.

[0082] Optionally, the sperm segmentation model training set includes: sperm images and bounding box coordinates corresponding to each sperm image;

[0083] Sperm images were acquired using a bright-field microscope; the bounding box coordinates corresponding to each sperm image were obtained by annotating the sperm image using Labelbox or LabelMe.

[0084] Optionally, the depth prediction model training data set includes: local images of sperm flagella at different focal planes and depth information corresponding to each local image of sperm flagella;

[0085] The partial images of sperm flagella at different focal planes and the depth information corresponding to each partial image of sperm flagella are obtained by capturing sperm flagella images at different focal planes using a piezoelectric platform and intercepting the captured sperm flagella images; the partial images of sperm flagella contain sperm flagella image information of approximately 1-2 μm. Step S000 specifically includes:

[0086] Step S010: input each data in the sperm segmentation model training set into the corresponding sperm segmentation model, and use the target loss function and Adam optimizer to obtain a trained sperm segmentation model;

[0087] The target loss function includes a bounding box regression loss function, a target confidence loss function and a category classification loss function;

[0088] Step S020: Input each data in the depth prediction model training set into the corresponding depth prediction model, use the mean square error loss function and Adam optimizer, with an initial learning rate of K, until M iterations are performed to obtain a trained depth prediction model;

[0089] The K and M are pre-set according to the amount of data in the depth prediction model training set.

[0090] In the specific implementation process, K is set to 0.001 and M is set to 150 times.

[0091] In this embodiment, the sperm segmentation model divides the data set into a training set, a validation set, and a test set in a ratio of 8:1:1; the training and validation of the depth prediction model adopt a 10-fold cross-validation method.

[0092] In step S010, each data point from the training and validation sets in the sperm segmentation model training set is first input into a pre-set initialized sperm segmentation model. The input image is passed through the network via forward propagation to generate predicted bounding box coordinates and class probabilities. Next, the difference between the predicted bounding box coordinates and the actual bounding box coordinates is calculated using a loss function, which includes bounding box regression loss, object confidence loss, and class classification loss. Backpropagation is then used to calculate the gradient of the loss function with respect to the model parameters, propagating back layer by layer from the output layer. Finally, the model parameters are updated using the Adam optimization algorithm to minimize the loss function. This process is repeated with each training iteration to gradually optimize the parameters and improve the model's detection performance. An early stopping strategy is also employed during the training of the sperm segmentation model. Specifically, if the performance metrics on the validation set do not significantly improve within 50 consecutive iterations, training is terminated, resulting in a sperm segmentation model. This strategy helps prevent model overfitting and ensures that training is terminated at the optimal moment. After the training and validation process is completed, each data in the test set is input into the sperm segmentation model, and the average precision, recall rate, and accuracy of the model at this time are obtained to evaluate the final performance of the sperm segmentation model.

[0093] In step S020, a 10-fold cross-validation method is used to divide the depth prediction model training set into 10 subsets of equal size. In each iteration, 9 folds are used as training sets, and the remaining 1 fold is used as a validation set to evaluate the performance of the model.

[0094] In the implementation, each data point in the depth prediction model training set is input into a pre-set initialized depth prediction model, and the prediction error is measured using the mean squared error loss function. Through continuous iteration, the model parameters are adjusted to minimize the loss function, resulting in an optimized depth prediction model. During the entire training process, a total of 150 training rounds are performed, and the accuracy of the depth prediction model on the validation set is calculated. After the training and validation process is complete, the mean squared error and coefficient of determination (R²) of the final model are calculated to evaluate the model's final performance.

[0095] For example, there are 2000 data in the depth prediction model training set. After the training and verification process is completed, the determination coefficient R obtained by linear fitting the final actual value and the predicted value obtained by the depth prediction model is 2 The value reaches 0.99, such as Figure 5 As shown, this shows that the model can accurately predict the depth information of the image. The depth prediction value obtained by the depth prediction model is highly correlated with the actual value, verifying the reliability and stability of the model.

[0096] In this embodiment, the Adam optimizer can also be changed to an SGD optimizer.

[0097] In step S400,

[0098] The depth prediction model consists of a cascaded input layer, convolutional layer, pooling layer, fully connected layer, and output layer.

[0099] The input layer of the depth prediction model receives a partial image of a sperm flagellum of a specified size. The convolutional layer extracts basic and high-level features from the image by gradually increasing the number and depth of convolution kernels. The pooling layer reduces the spatial size of the feature map while retaining important feature information. The fully connected layer processes and refines the features through linear combinations and nonlinear activation functions. Finally, the output layer performs regression and predicts the target value. The entire depth prediction model is trained and optimized to minimize the mean square error loss function, resulting in an accurate prediction model.

[0100] For example, the input layer receives a single-channel partial image of a sperm flagellum of size 101×21 pixels. The model then consists of three convolutional layers, each using a 3×3 convolution kernel. The first layer uses 16 kernels, the second layer uses 32 kernels, and the third layer uses 64 kernels. The pooling layer includes two layers of maximum pooling. The first layer uses a 3×3 pooling window with a stride of [3, 3], and the second layer also uses a 3×3 pooling window with a stride of [3, 3]. The fully connected layer contains 256 neurons with the ReLU activation function. The following layers contain 128 neurons, 64 neurons, and 32 neurons, all using the ReLU activation function. Finally, the output layer contains 1 neuron for regression tasks to predict depth.

[0101] The present embodiment provides a method for reconstructing the three-dimensional motion of sperm flagella using deep learning. By leveraging the rapid processing capabilities of a deep-learned sperm segmentation model and a depth prediction model, image analysis and three-dimensional reconstruction can be completed in a short period of time, improving real-time performance and reducing computing time and resource consumption. Furthermore, the method can accurately segment and predict the focal position of sperm flagella, thereby achieving high-precision three-dimensional reconstruction. Furthermore, the present invention primarily relies on a brightfield microscope and a computer, avoiding the need for expensive, complex equipment. The equipment is simple, and maintenance and operating costs are relatively low, making it suitable for large-scale application and promotion. Furthermore, the use of a brightfield microscope for image acquisition eliminates the need for complex optical setup and calibration processes, simplifying experimental operations and making it applicable to a variety of microscope types, thus having a wide range of applicability.

[0102] Furthermore, this embodiment utilizes label-free imaging technology, avoiding the need for staining sperm samples and ensuring sperm activity and integrity. This technology is suitable for sperm assessment and selection in clinical settings and assisted reproductive technology, demonstrating its practical application. Furthermore, by generating a three-dimensional model of the sperm flagellum, the present invention can determine the three-dimensional motion trajectory of the sperm flagellum, providing more comprehensive information on sperm dynamics. This allows doctors to more accurately assess and select sperm, thereby improving the success rate of assisted reproductive technology.

[0103] Example 3

[0104] A computer device of this embodiment includes a memory and a processor;

[0105] The computer program is stored in the memory, and the processor executes the computer program in the memory, specifically executing a method for reconstructing the three-dimensional motion of sperm flagella based on deep learning in Example 1 or Example 2.

[0106] A computer device of the present invention can reconstruct the three-dimensional model of sperm flagella more efficiently and accurately by executing a method for reconstructing the three-dimensional motion of sperm flagella based on deep learning, providing a reliable solution for the study of sperm movement behavior and the development of assisted reproductive technology.

[0107] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0108] In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; and internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0109] In the present invention, unless otherwise expressly specified or limited, when a first feature is "above" or "below" a second feature, it may mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Furthermore, when a first feature is "above," "above," or "above" a second feature, it may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is at a higher level than the second feature. When a first feature is "below," "below," or "below" a second feature, it may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is at a lower level than the second feature.

[0110] In the description of this specification, the terms "one embodiment", "some embodiments", "embodiments", "examples", "specific examples" or "some examples" refer to the specific features, structures, materials or characteristics described in conjunction with the embodiment or example and included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and features of different embodiments or examples, unless they are mutually inconsistent.

[0111] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may alter, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for reconstructing the three-dimensional motion of sperm flagella based on deep learning, characterized in that: include: S100, obtaining a two-dimensional image of sperm to be processed, and performing image preprocessing on the two-dimensional image of sperm to be processed to obtain a preprocessed image; S200, inputting the pre-processed image into a trained sperm segmentation model to obtain a sperm flagellum binary image; the sperm segmentation model is a YOLOv8 model; S300, skeletonizing the binary image of the sperm flagellum to obtain the centerline of the sperm flagellum; based on the centerline of the sperm flagellum, obtaining N equidistant points and the position coordinates corresponding to each equidistant point; S400, based on the N equidistant points, obtaining N sperm flagella partial images, and inputting each sperm flagella partial image into a trained depth prediction model to obtain depth information of each sperm flagella partial image; the depth prediction model is a convolutional neural network model; The depth prediction model is trained using a depth prediction model training data set; The depth prediction model training data set includes: sperm flagella local images at different focal planes and depth information corresponding to each sperm flagella local image; The local images of sperm flagella at different focal planes and the depth information corresponding to each local image of sperm flagella are obtained by photographing sperm flagella images at different focal planes using a piezoelectric platform and intercepting the photographed sperm flagella images; the local images of sperm flagella include sperm flagella image information of 1-2 μm; S500 : Reconstructing a three-dimensional model of the sperm flagellum based on the depth information of each sperm flagellum local image and the position coordinates corresponding to each equidistant point.

2. The method for reconstructing the three-dimensional motion of sperm flagella based on deep learning according to claim 1, characterized in that: The S100 includes: Performing filter enhancement on the two-dimensional image of the sperm to be processed using a filter to obtain a pre-processed image; The two-dimensional image of the sperm to be processed is acquired by bright field microscopy.

3. The method for reconstructing the three-dimensional motion of sperm flagella based on deep learning according to claim 1, wherein In the S300, The N equidistant points are obtained by intercepting the center line of the sperm flagellum at equal distances every X μm; X is a preset distance.

4. The method for reconstructing the three-dimensional motion of sperm flagella based on deep learning according to claim 1, characterized in that In S400, obtaining N sperm flagella local images based on the N equidistant points includes: Taking each of the N equidistant points as the center, N sperm flagella partial images are intercepted; each of the N sperm flagella partial images contains 1-2 μm sperm flagella image information.

5. The method for reconstructing the three-dimensional motion of sperm flagella based on deep learning according to claim 1, characterized in that: The S100 also includes: S000. Use a training data set to train the sperm segmentation model and the depth prediction model to obtain a trained sperm segmentation model and a depth prediction model; the training data set includes: a sperm segmentation model training data set and a depth prediction model training data set.

6. The method for reconstructing the three-dimensional motion of sperm flagella based on deep learning according to claim 5, characterized in that: The sperm segmentation model training set includes: sperm images and bounding box coordinates corresponding to each sperm image; The sperm image is obtained by bright field microscopy; the bounding box coordinates corresponding to each sperm image are obtained by annotating the sperm image using Labelbox or LabelMe.

7. The method for reconstructing the three-dimensional motion of sperm flagella based on deep learning according to claim 5, characterized in that: The S000 specifically includes: S010, inputting each data in the sperm segmentation model training set into the corresponding sperm segmentation model, using the target loss function and Adam optimizer to obtain a trained sperm segmentation model; The target loss function includes a bounding box regression loss function, a target confidence loss function and a category classification loss function; S020, input each data in the depth prediction model training set into the corresponding depth prediction model, use the mean square error loss function and Adam optimizer, with an initial learning rate of K, until M iterations are performed to obtain a trained depth prediction model; The K and M are pre-set according to the amount of data in the depth prediction model training set.

8. The method for reconstructing the three-dimensional motion of sperm flagella based on deep learning according to claim 1, characterized in that: In the above S400, The depth prediction model includes an input layer, a convolutional layer, a pooling layer, a fully connected layer and an output layer that are cascaded to each other in sequence; Among them, the input layer receives the local image of the sperm flagellum; the convolution layer extracts features in the image by gradually increasing the number and depth of the convolution kernel; the pooling layer is used to reduce the spatial size of the features extracted by the convolution layer, and the fully connected layer processes the features pooled by the pooling layer through linear combination and nonlinear activation function; the output layer is used to perform regression tasks, obtain and output the depth information corresponding to each local image of the sperm flagellum.

9. A computer device, characterized in that: include: memory and processor; The memory stores a computer program, and the processor executes the computer program in the memory, specifically performing the method for reconstructing the three-dimensional motion of sperm flagella based on deep learning as described in any one of claims 1 to 8.

Images